[GH-ISSUE #1344] [Important Feature] AI based best model search function #867

New issue

Closed

opened 2026-10-20 10:03:49 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-20 10:03:49 +00:00

Owner

Originally created by @Pevernow on GitHub (Sep 29, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1344

For individuals with poor ears and difficulty in distinguishing whether there is overfitting or underfitting between different epoch models through sound, it is best to provide an AI based audio similarity score to select the best model function.

For some reason, I don't have time to implement it. I can only explain this idea in detail as much as possible and look forward to your implementation.

The algorithm flow of this function is as follows:

Firstly, users need to upload audio for two training characters, one not in the training set and the other in the training set.

Then, for each model, perform the following operations:

For audio within the training set, change the sound and use AI scoring tools to evaluate the similarity between the changed sound and the original audio.

For audio outside the training set, change the sound and use AI scoring tools to evaluate the similarity between the changed sound and the original audio.

Then calculate the sum of the two similarities and divide by two to obtain the model score.

Generate a table with the model and scores, and indicate the one with the highest score. (At this point, there is neither overfitting nor underfitting)

For AI models that score similarity between two given audio, my suggestion is ERes2Net or CAM++，or other better models.

https://modelscope.cn/models/damo/speech_eres2net_sv_zh-cn_16k-common/summary

https://modelscope.cn/models/damo/speech_campplus_sv_zh-cn_16k-common/summary

Originally created by @Pevernow on GitHub (Sep 29, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1344 For individuals with poor ears and difficulty in distinguishing whether there is overfitting or underfitting between different epoch models through sound, it is best to provide an AI based audio similarity score to select the best model function. For some reason, I don't have time to implement it. I can only explain this idea in detail as much as possible and look forward to your implementation. The algorithm flow of this function is as follows: Firstly, users need to upload audio for two training characters, one not in the training set and the other in the training set. Then, for each model, perform the following operations: For audio within the training set, change the sound and use AI scoring tools to evaluate the similarity between the changed sound and the original audio. For audio outside the training set, change the sound and use AI scoring tools to evaluate the similarity between the changed sound and the original audio. Then calculate the sum of the two similarities and divide by two to obtain the model score. Generate a table with the model and scores, and indicate the one with the highest score. (At this point, there is neither overfitting nor underfitting) For AI models that score similarity between two given audio, my suggestion is ERes2Net or CAM++，or other better models. https://modelscope.cn/models/damo/speech_eres2net_sv_zh-cn_16k-common/summary https://modelscope.cn/models/damo/speech_campplus_sv_zh-cn_16k-common/summary

bsakaguchi

2026-10-20 10:03:49 +00:00

closed this issue
added the
enhancement

stale
labels

bsakaguchi commented

2026-10-20 10:03:49 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Apr 28, 2024):

This issue was closed because it has been inactive for 15 days since being marked as stale.

@github-actions[bot] commented on GitHub (Apr 28, 2024): This issue was closed because it has been inactive for 15 days since being marked as stale.