RVC-Project/Retrieval-based-Voice-Conversion-WebUI

Fork 0

mirror of https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI.git synced 2026-11-05 00:27:30 +00:00

[GH-ISSUE #1542] HuBERT ExtraLarge instead of HuBert base model? #1017

New issue

Open

opened 2026-10-20 10:05:07 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-20 10:05:07 +00:00

Owner

Originally created by @ghost on GitHub (Nov 14, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1542

Not an Issue:
Just wanted to know that Can we use HuBERT Extra Large model instead of HuBERT base model?

Will it perform better and generate better quality results?

Reference : https://github.com/facebookresearch/fairseq/tree/main/examples/hubert

Originally created by @ghost on GitHub (Nov 14, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1542 Not an Issue: Just wanted to know that Can we use HuBERT Extra Large model instead of HuBERT base model? Will it perform better and generate better quality results? Reference : https://github.com/facebookresearch/fairseq/tree/main/examples/hubert

bsakaguchi added the

good first issue

label

2026-10-20 10:05:07 +00:00

bsakaguchi commented

2026-10-20 10:05:08 +00:00

Author

Owner

@blakejrobinson commented on GitHub (Nov 15, 2023):

I did experiment with this. One obvious thing is that you won't be able to run models other people make (unless they also use hubert extra large) as inference requires the same HuBERT that was used to train.

Not sure I could hear much quality difference between trained models, but I didn't go into much detail once I'd realised the above.

@blakejrobinson commented on GitHub (Nov 15, 2023): I did experiment with this. One obvious thing is that you won't be able to run models other people make (unless they also use hubert extra large) as inference requires the same HuBERT that was used to train. Not sure I could hear much quality difference between trained models, but I didn't go into much detail once I'd realised the above.

bsakaguchi commented

2026-10-20 10:05:09 +00:00

Author

Owner

@Tps-F commented on GitHub (Nov 16, 2023):

Support for other HuBerts, including ExtraLarge, is planned.

@Tps-F commented on GitHub (Nov 16, 2023): Support for other HuBerts, including ExtraLarge, is planned.

bsakaguchi commented

2026-10-20 10:05:09 +00:00

Author

Owner

@ghost commented on GitHub (Nov 17, 2023):

@Tps-F Looking forward to it,

@ghost commented on GitHub (Nov 17, 2023): @Tps-F Looking forward to it,

bsakaguchi commented

2026-10-20 10:05:09 +00:00

Author

Owner

@Mixomo commented on GitHub (Nov 22, 2023):

Looking forward! 👀

@Mixomo commented on GitHub (Nov 22, 2023): Looking forward! 👀

bsakaguchi commented

2026-10-20 10:05:09 +00:00

Author

Owner

@thepowerfuldeez commented on GitHub (Dec 2, 2023):

Hi! Adding some comment here, correct me if I'm wrong. Modern SVC systems use ContentVec instead of plain Hubert, which is special pre-training of Hubert that disentangles speaker features making it better suited for voice conversion task.
Using just larger Hubert might not make better quality out of the end-to-end system.
Instead, combination of Whisper-large-v3 and ContentVec gives better results, as described in this paper: https://arxiv.org/abs/2310.11160

@thepowerfuldeez commented on GitHub (Dec 2, 2023): Hi! Adding some comment here, correct me if I'm wrong. Modern SVC systems use ContentVec instead of plain Hubert, which is special pre-training of Hubert that disentangles speaker features making it better suited for voice conversion task. Using just larger Hubert might not make better quality out of the end-to-end system. Instead, combination of Whisper-large-v3 and ContentVec gives better results, as described in this paper: https://arxiv.org/abs/2310.11160

bsakaguchi commented

2026-10-20 10:05:09 +00:00

Author

Owner

@ghost commented on GitHub (Dec 8, 2023):

@thepowerfuldeez Have you tried this integration?

@ghost commented on GitHub (Dec 8, 2023): @thepowerfuldeez Have you tried this integration?

bsakaguchi commented

2026-10-20 10:05:09 +00:00

Author

Owner

@hakku615 commented on GitHub (Apr 24, 2026):

Support for other HuBerts, including ExtraLarge, is planned.
is this still TBD?

@hakku615 commented on GitHub (Apr 24, 2026): > Support for other HuBerts, including ExtraLarge, is planned. is this still TBD?