[GH-ISSUE #502] [Feature request] Segmentation of audio based on parts to improve vocal output of long songs #282

New issue

Closed

opened 2026-10-20 09:57:35 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-20 09:57:35 +00:00

Owner

Originally created by @kalomaze on GitHub (Jun 11, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/502

There could be an advanced setting that detects where silence happens based on a threshold, and does each inference of long vocal pieces separately, then properly combines it into one full output audio. This would help for some songs, especially ones with long quiet sections, as those can have issues with strange raspiness and cutting in and out when returning to vocals after long silent periods (though it is quite rare.)

There would also likely be positive mild improvements in the non-extreme cases of better inference audio, as it would more closely match the training data (in length). The silence detection could be somewhat long, and require, let's say, over 1 seconds of silence under -40db in order for it to start another segmented 'piece.'

This would also prevent errors on longer pieces of audio in the vast majority of cases. If possible, a segment could be 'force split up' if it exceeds an extremely long length (Let's say, 5 minutes of uninterrupted speaking.)

You could call this setting 'Auto-segment' and mention in the blurb that it is meant to improve inference quality and increase possibilities of longer audio. You could also mention that it may take longer due to processing separately.

Originally created by @kalomaze on GitHub (Jun 11, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/502 There could be an advanced setting that detects where silence happens based on a threshold, and does each inference of long vocal pieces separately, then properly combines it into one full output audio. This would help for some songs, especially ones with long quiet sections, as those can have issues with strange raspiness and cutting in and out when returning to vocals after long silent periods (though it is quite rare.) There would also likely be positive mild improvements in the non-extreme cases of better inference audio, as it would more closely match the training data (in length). The silence detection could be somewhat long, and require, let's say, over 1 seconds of silence under -40db in order for it to start another segmented 'piece.' ![image](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/assets/66376113/f1951291-e1f4-41a6-9d4e-8742a28e364b) This would also prevent errors on longer pieces of audio in the vast majority of cases. If possible, a segment could be 'force split up' if it exceeds an extremely long length (Let's say, 5 minutes of uninterrupted speaking.) You could call this setting 'Auto-segment' and mention in the blurb that it is meant to improve inference quality and increase possibilities of longer audio. You could also mention that it may take longer due to processing separately.

bsakaguchi

2026-10-20 09:57:35 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-20 09:57:36 +00:00

Author

Owner

@RVC-Boss commented on GitHub (Jun 11, 2023):

Good idea.

@RVC-Boss commented on GitHub (Jun 11, 2023): Good idea.

bsakaguchi commented

2026-10-20 09:57:36 +00:00

Author

Owner

@kalomaze commented on GitHub (Jun 11, 2023):

One thing I might add, for non GPU / limited RAM systems, there could be a 'compatibility mode' that will segment as much as possible to prevent errors and has lower criteria for segmenting. Let's say 0.5s of silence is the segment threshold, for example. And the maximum length of a segment is 15s or 30s, or something around that length. To prevent abrupt cutoffs, you could make it cut at the center of the quietest average 0.25s in the segment and start the next segment from that point. Either that, or the numbers are directly configurable within the GUI.

One thing I do worry about is having so many options in the main inference tab might get cluttered. Maybe an 'advanced settings' at the bottom of the page could be made in the future to move things like vocal consonant, volume envelope, median filter, etc as their own category, that is used for both batch conversions and single conversions.

@kalomaze commented on GitHub (Jun 11, 2023): One thing I might add, for non GPU / limited RAM systems, there could be a 'compatibility mode' that will segment as much as possible to prevent errors and has lower criteria for segmenting. Let's say 0.5s of silence is the segment threshold, for example. And the maximum length of a segment is 15s or 30s, or something around that length. To prevent abrupt cutoffs, you could make it cut at the center of the quietest average 0.25s in the segment and start the next segment from that point. Either that, or the numbers are directly configurable within the GUI. One thing I do worry about is having so many options in the main inference tab might get cluttered. Maybe an 'advanced settings' at the bottom of the page could be made in the future to move things like vocal consonant, volume envelope, median filter, etc as their own category, that is used for both batch conversions and single conversions.

bsakaguchi commented

2026-10-20 09:57:36 +00:00

Author

Owner

@RVC-Boss commented on GitHub (Jun 12, 2023):

'advanced settings' good idea

@RVC-Boss commented on GitHub (Jun 12, 2023): 'advanced settings' good idea

bsakaguchi commented

2026-10-20 09:57:36 +00:00

Author

Owner

@epietrocola commented on GitHub (Feb 26, 2024):

Is there any news regarding this? I tried to infer a 6 minutes voice file with a lot of silence between sentences and the inference quality of the result is way lower than the quality I get when using a single wavefile of 4 seconds extracted from the same previous long file

@epietrocola commented on GitHub (Feb 26, 2024): Is there any news regarding this? I tried to infer a 6 minutes voice file with a lot of silence between sentences and the inference quality of the result is way lower than the quality I get when using a single wavefile of 4 seconds extracted from the same previous long file

bsakaguchi commented

2026-10-20 09:57:37 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Apr 28, 2024):

This issue was closed because it has been inactive for 15 days since being marked as stale.

@github-actions[bot] commented on GitHub (Apr 28, 2024): This issue was closed because it has been inactive for 15 days since being marked as stale.