[PR #116] [MERGED] Feature: RVC module, applying voice conversion to audio sent by ST #199

New issue

Closed

opened 2026-10-15 18:05:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:05:29 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/SillyTavern/SillyTavern-Extras/pull/116
Author: @Tony-sama
Created: 8/10/2023
Status: ✅ Merged
Merged: 8/10/2023
Merged by: @Cohee1207

Base: neo ← Head: neo

📝 Commits (7)

0d00b5f Add module managing RVC extension to convert audio sent by ST using voice model
28024c5 Merge branch 'SillyTavern:neo' into neo
60a8e5c Add monkey patched fairseq package to run on python 3.11 (what is needed for our use of RVC at least)
efd2408 Add fairseq MIT licence file
240684c Add temporary requirements file for RVC as additional file to be install after complete
3fbb1b6 Continued monkey patching fairseq for RVC python 3.11
1bf4a55 add rvc MIT licence

📊 Changes

404 files changed (+90425 additions, -2 deletions)

View changed files

➕ data/models/rvc/.placeholder (+0 -0)
➕ data/tmp/.placeholder (+0 -0)
➕ modules/voice_conversion/fairseq/LICENSE (+21 -0)
➕ modules/voice_conversion/fairseq/__init__.py (+45 -0)
➕ modules/voice_conversion/fairseq/binarizer.py (+381 -0)
➕ modules/voice_conversion/fairseq/checkpoint_utils.py (+905 -0)
➕ modules/voice_conversion/fairseq/data/__init__.py (+130 -0)
➕ modules/voice_conversion/fairseq/data/add_target_dataset.py (+83 -0)
➕ modules/voice_conversion/fairseq/data/append_token_dataset.py (+41 -0)
➕ modules/voice_conversion/fairseq/data/audio/__init__.py (+93 -0)
➕ modules/voice_conversion/fairseq/data/audio/audio_utils.py (+389 -0)
➕ modules/voice_conversion/fairseq/data/audio/data_cfg.py (+387 -0)
➕ modules/voice_conversion/fairseq/data/audio/dataset_transforms/__init__.py (+53 -0)
➕ modules/voice_conversion/fairseq/data/audio/dataset_transforms/concataugment.py (+61 -0)
➕ modules/voice_conversion/fairseq/data/audio/dataset_transforms/noisyoverlapaugment.py (+105 -0)
➕ modules/voice_conversion/fairseq/data/audio/feature_transforms/__init__.py (+43 -0)
➕ modules/voice_conversion/fairseq/data/audio/feature_transforms/delta_deltas.py (+37 -0)
➕ modules/voice_conversion/fairseq/data/audio/feature_transforms/global_cmvn.py (+29 -0)
➕ modules/voice_conversion/fairseq/data/audio/feature_transforms/specaugment.py (+131 -0)
➕ modules/voice_conversion/fairseq/data/audio/feature_transforms/utterance_cmvn.py (+41 -0)

...and 80 more files

📄 Description

This extension perform voice conversion of audio file sent by ST using (a monkey patched) RVC.

Features

a small script getting TTS audio from request, applying requested RVC model on it and sending back the new audio

What changed

Needed to correct coqui extension scripts to keep current directory path
RVC and fairseq package dirty patched version for now in modules/voice_conversion/rvc and modules/voice_conversion/fairseq

Limitations

Need the RVC models to be stored in data/models/rvc (index file optional)
RVC code is a strip version from original repository with only what is needed for performing voice changing
RVC code relies onf fairseq that is not compatible with python 3.11, made a monkey patched version of fairseq that run fine in module modules\voice_conversion\fairseq
Their is a theorytical dependency conflict between TTS and Torchcrepe over librosa, cannot install everything with one requirement file, made an additional requirement file to be call after for now

Testing

The extension works fine alongside coqui when installed as follow:

conda install python=3.11
pip install -r requirements-complete.txt
pip install -r requirements-rvc.txt
put RVC models into "data/models/rvc/" index files are optional
example extras call: python server.py --enable-modules=rvc,live2d,classify,coqui-tts --coqui-gpu

For ST:
RVC voice map example:
Seraphina=guanguanV1, Coding Sensei=kiki

Coqui voice map example:
Seraphina:tts_models--en--ljspeech--glow-tts\model_file.pth, Coding Sensei:tts_models--multilingual--multi-dataset--your_tts\model_file.pth[3][0]

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/SillyTavern/SillyTavern-Extras/pull/116 **Author:** [@Tony-sama](https://github.com/Tony-sama) **Created:** 8/10/2023 **Status:** ✅ Merged **Merged:** 8/10/2023 **Merged by:** [@Cohee1207](https://github.com/Cohee1207) **Base:** `neo` ← **Head:** `neo` --- ### 📝 Commits (7) - [`0d00b5f`](https://github.com/SillyTavern/SillyTavern-Extras/commit/0d00b5fdd0b99cbd4ac44f5e7703a531819fbc53) Add module managing RVC extension to convert audio sent by ST using voice model - [`28024c5`](https://github.com/SillyTavern/SillyTavern-Extras/commit/28024c5649182bb0ed72fbd80ba7185abc33201f) Merge branch 'SillyTavern:neo' into neo - [`60a8e5c`](https://github.com/SillyTavern/SillyTavern-Extras/commit/60a8e5c9c638ca592e80fdbd6f6fdbf705f3824a) Add monkey patched fairseq package to run on python 3.11 (what is needed for our use of RVC at least) - [`efd2408`](https://github.com/SillyTavern/SillyTavern-Extras/commit/efd2408051e12a6f3157129386f4196599304f5c) Add fairseq MIT licence file - [`240684c`](https://github.com/SillyTavern/SillyTavern-Extras/commit/240684c8c4c030feb14c76ad4fb06620f858e172) Add temporary requirements file for RVC as additional file to be install after complete - [`3fbb1b6`](https://github.com/SillyTavern/SillyTavern-Extras/commit/3fbb1b6ede365450bb1ab734a201c49946e0db35) Continued monkey patching fairseq for RVC python 3.11 - [`1bf4a55`](https://github.com/SillyTavern/SillyTavern-Extras/commit/1bf4a5539f81e0892412b0d272125cdb328a9ba4) add rvc MIT licence ### 📊 Changes **404 files changed** (+90425 additions, -2 deletions) <details> <summary>View changed files</summary> ➕ `data/models/rvc/.placeholder` (+0 -0) ➕ `data/tmp/.placeholder` (+0 -0) ➕ `modules/voice_conversion/fairseq/LICENSE` (+21 -0) ➕ `modules/voice_conversion/fairseq/__init__.py` (+45 -0) ➕ `modules/voice_conversion/fairseq/binarizer.py` (+381 -0) ➕ `modules/voice_conversion/fairseq/checkpoint_utils.py` (+905 -0) ➕ `modules/voice_conversion/fairseq/data/__init__.py` (+130 -0) ➕ `modules/voice_conversion/fairseq/data/add_target_dataset.py` (+83 -0) ➕ `modules/voice_conversion/fairseq/data/append_token_dataset.py` (+41 -0) ➕ `modules/voice_conversion/fairseq/data/audio/__init__.py` (+93 -0) ➕ `modules/voice_conversion/fairseq/data/audio/audio_utils.py` (+389 -0) ➕ `modules/voice_conversion/fairseq/data/audio/data_cfg.py` (+387 -0) ➕ `modules/voice_conversion/fairseq/data/audio/dataset_transforms/__init__.py` (+53 -0) ➕ `modules/voice_conversion/fairseq/data/audio/dataset_transforms/concataugment.py` (+61 -0) ➕ `modules/voice_conversion/fairseq/data/audio/dataset_transforms/noisyoverlapaugment.py` (+105 -0) ➕ `modules/voice_conversion/fairseq/data/audio/feature_transforms/__init__.py` (+43 -0) ➕ `modules/voice_conversion/fairseq/data/audio/feature_transforms/delta_deltas.py` (+37 -0) ➕ `modules/voice_conversion/fairseq/data/audio/feature_transforms/global_cmvn.py` (+29 -0) ➕ `modules/voice_conversion/fairseq/data/audio/feature_transforms/specaugment.py` (+131 -0) ➕ `modules/voice_conversion/fairseq/data/audio/feature_transforms/utterance_cmvn.py` (+41 -0) _...and 80 more files_ </details> ### 📄 Description This extension perform voice conversion of audio file sent by ST using (a monkey patched) RVC. # Features - a small script getting TTS audio from request, applying requested RVC model on it and sending back the new audio # What changed - Needed to correct coqui extension scripts to keep current directory path - RVC and fairseq package dirty patched version for now in modules/voice_conversion/rvc and modules/voice_conversion/fairseq # Limitations - Need the RVC models to be stored in data/models/rvc (index file optional) - RVC code is a strip version from original repository with only what is needed for performing voice changing - RVC code relies onf fairseq that is not compatible with python 3.11, made a monkey patched version of fairseq that run fine in module modules\voice_conversion\fairseq - Their is a theorytical dependency conflict between TTS and Torchcrepe over librosa, cannot install everything with one requirement file, made an additional requirement file to be call after for now # Testing The extension works fine alongside coqui when installed as follow: - conda install python=3.11 - pip install -r requirements-complete.txt - pip install -r requirements-rvc.txt - put RVC models into "data/models/rvc/" index files are optional - example extras call: python server.py --enable-modules=rvc,live2d,classify,coqui-tts --coqui-gpu For ST: RVC voice map example: Seraphina=guanguanV1, Coding Sensei=kiki Coqui voice map example: Seraphina:tts_models--en--ljspeech--glow-tts\model_file.pth, Coding Sensei:tts_models--multilingual--multi-dataset--your_tts\model_file.pth[3][0] --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-15 18:05:29 +00:00