[GH-ISSUE #2855] ## CHAPTER 5: CRITICAL FILE SYSTEM AND BLUEPRINT RE-ARCHITECTURING #2053

New issue

Open

opened 2026-10-20 10:14:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-20 10:14:11 +00:00

Owner

Originally created by @LebedevIV on GitHub (Sep 10, 2026).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/2855

Feature 13: Replacing Positional `sys.argv` with Robust `argparse` Flags in Dataset Tools

The Problem:
Currently, standalone CLI modules like train/dataset/extract_f0.py and extract_hubert_feature.py rely on raw positional command-line arguments via sys.argv. Any slight deviation in the wrapper pipeline call logic or an attempt by a user to run the script directly via a shell causes fatal argument shifts. For instance, passing an unexpected extraction string causes the script to interpret a directory path as a hardware backend, resulting in a silent crash or a misleading ValueError: Unsupported F0 extraction mode.
Suggested Solution:
Refactor all dataset processing entry points to utilize Python's native argparse library. Forcing explicit, named flags (e.g., --backend cuda, --input_dir ./logs/project) instead of brittle array indices will secure the interface from argument corruption across different OS shells and Python 3.12 subprocess configurations.

Bug 14: Fragile Directory Initialization and Safe Storage Isolation

The Problem:
The pitch extraction core (extract_f0.py) assumes that its target directories (2a_f0, 2b-f0nsf, 3_feature768) already exist as physical folders on the drive. If a user shifts to a new custom experiment name (e.g., creating a separate model configuration profile), the pipeline immediately crashes with FileNotFoundError or NotADirectoryError (WinError 267). This happens because the framework fails to execute a recursive folder layout check and occasionally creates flat, broken meta-strings instead of actual physical directory nodes on Windows environments.
Suggested Solution:
Implement an immutable, top-level initialization pass using os.makedirs(..., exist_ok=True) inside all feature extraction entry points. Before attempting to open loggers or stream tensor data, the framework must explicitly declare and force-verify that the required nested project folder branches physically exist as genuine directory objects.

Feature 15: Dynamic Validation-Driven `filelist.txt` Generation (Anti-Crash Mapping)

The Problem:
The current dataset preprocessing utility (train/preprocess.py via audio-slicer) can occasionally skip indices or name sliced chunks non-sequentially based on the source voice profile's underlying envelope and silences (e.g., rendering files starting directly from 0_10.npy while omitting 0_1.npy). RVC's blueprint compiler generates the filelist.txt training map "blindly," assuming a perfect arithmetic index sequence. When the PyTorch DataLoader worker processes step one, it attempts to load missing baseline tensors, resulting in an unrecoverable FileNotFoundError that halts training.
Suggested Solution:
Rework the blueprint compiling script to perform a dynamic, validation-driven directory scan. Instead of guessing file rows mathematically, the script should physically read the contents of the 3_feature768 and 2a_f0 folders on the SSD and map rows into filelist.txt only if all three required AI components (WAV, HuBERT features, and F0 pitch maps) are verified to coexist simultaneously on the drive. This failsafe mechanism will make the training engine completely bulletproof against irregular dataset slicing behaviors.

Originally created by @LebedevIV on GitHub (Sep 10, 2026). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/2855 ### Feature 13: Replacing Positional `sys.argv` with Robust `argparse` Flags in Dataset Tools * **The Problem:** Currently, standalone CLI modules like `train/dataset/extract_f0.py` and `extract_hubert_feature.py` rely on raw positional command-line arguments via `sys.argv`. Any slight deviation in the wrapper pipeline call logic or an attempt by a user to run the script directly via a shell causes fatal argument shifts. For instance, passing an unexpected extraction string causes the script to interpret a directory path as a hardware backend, resulting in a silent crash or a misleading `ValueError: Unsupported F0 extraction mode`. * **Suggested Solution:** Refactor all dataset processing entry points to utilize Python's native `argparse` library. Forcing explicit, named flags (e.g., `--backend cuda`, `--input_dir ./logs/project`) instead of brittle array indices will secure the interface from argument corruption across different OS shells and Python 3.12 subprocess configurations. --- ### Bug 14: Fragile Directory Initialization and Safe Storage Isolation * **The Problem:** The pitch extraction core (`extract_f0.py`) assumes that its target directories (`2a_f0`, `2b-f0nsf`, `3_feature768`) already exist as physical folders on the drive. If a user shifts to a new custom experiment name (e.g., creating a separate model configuration profile), the pipeline immediately crashes with `FileNotFoundError` or `NotADirectoryError (WinError 267)`. This happens because the framework fails to execute a recursive folder layout check and occasionally creates flat, broken meta-strings instead of actual physical directory nodes on Windows environments. * **Suggested Solution:** Implement an immutable, top-level initialization pass using `os.makedirs(..., exist_ok=True)` inside all feature extraction entry points. Before attempting to open loggers or stream tensor data, the framework must explicitly declare and force-verify that the required nested project folder branches physically exist as genuine directory objects. --- ### Feature 15: Dynamic Validation-Driven `filelist.txt` Generation (Anti-Crash Mapping) * **The Problem:** The current dataset preprocessing utility (`train/preprocess.py` via `audio-slicer`) can occasionally skip indices or name sliced chunks non-sequentially based on the source voice profile's underlying envelope and silences (e.g., rendering files starting directly from `0_10.npy` while omitting `0_1.npy`). RVC's blueprint compiler generates the `filelist.txt` training map "blindly," assuming a perfect arithmetic index sequence. When the PyTorch `DataLoader` worker processes step one, it attempts to load missing baseline tensors, resulting in an unrecoverable `FileNotFoundError` that halts training. * **Suggested Solution:** Rework the blueprint compiling script to perform a dynamic, validation-driven directory scan. Instead of guessing file rows mathematically, the script should physically read the contents of the `3_feature768` and `2a_f0` folders on the SSD and map rows into `filelist.txt` **only if** all three required AI components (WAV, HuBERT features, and F0 pitch maps) are verified to coexist simultaneously on the drive. This failsafe mechanism will make the training engine completely bulletproof against irregular dataset slicing behaviors.