low-VRAM GPUs #2049

New issue

Open

opened 2026-10-20 10:14:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-20 10:14:09 +00:00

Owner

Originally created by @LebedevIV on GitHub (Aug 22, 2026).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/2848

When running Batch Inference on full-length audio tracks using a 3GB GPU (NVIDIA GeForce GTX 1060 3GB), the output audio turns into a permanent synthetic/robotic buzzing sound (resembling an electric shaver). The system does not throw a CUDA out-of-memory error, but the inference breaks because of underlying PyTorch 2.7+ compatibility quirks with legacy Pascal architectures.

The problem is twofold:

FP16 Breakdown on 3GB: The framework automatically forces FP16 (Half Precision), which silently corrupts tensors into NaN zeros on older 3GB cards.
VRAM Overload: Long audio files completely choke the available 3GB buffer when loaded all at once.

I solved this by modifying configs/config.py to force FP32 precision and offload the data caching pipeline to the host system RAM via CPU environment flags:

if torch.cuda.is_available():
    infer_device = torch.device("cuda:0")
    infer_dtype = torch.float32
    infer_gpu_mem = 3.0
    # FIX FOR 3GB/4GB GPUS:
    is_half = False
    os.environ["RVC_AUDIO_FORCE_CPU"] = "1"

I request to expose these two parameters directly into the WebUI "Model Inference" / "Batch Inference" tabs as checkboxes or advanced toggles:

Force FP32 Precision (Disable is_half)
Offload DataLoader Caching to System RAM (CPU)

This will instantly allow thousands of users with budget or older GPUs (GTX 1060 3GB, 1050Ti, 960) to safely process full-length songs in near real-time using Windows Shared GPU Memory without crashing the pipeline.

Additional context

OS: Windows 10 / 11 x64
Python: 3.12+ (Python 3.12 branch)
PyTorch: 2.7.1+cu118
GPU: NVIDIA GeForce GTX 1060 3GB

Originally created by @LebedevIV on GitHub (Aug 22, 2026). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/2848 When running Batch Inference on full-length audio tracks using a 3GB GPU (NVIDIA GeForce GTX 1060 3GB), the output audio turns into a permanent synthetic/robotic buzzing sound (resembling an electric shaver). The system does not throw a CUDA out-of-memory error, but the inference breaks because of underlying PyTorch 2.7+ compatibility quirks with legacy Pascal architectures. The problem is twofold: 1. **FP16 Breakdown on 3GB:** The framework automatically forces FP16 (Half Precision), which silently corrupts tensors into `NaN` zeros on older 3GB cards. 2. **VRAM Overload:** Long audio files completely choke the available 3GB buffer when loaded all at once. I solved this by modifying `configs/config.py` to force FP32 precision and offload the data caching pipeline to the host system RAM via CPU environment flags: ```python if torch.cuda.is_available(): infer_device = torch.device("cuda:0") infer_dtype = torch.float32 infer_gpu_mem = 3.0 # FIX FOR 3GB/4GB GPUS: is_half = False os.environ["RVC_AUDIO_FORCE_CPU"] = "1" ``` I request to expose these two parameters directly into the WebUI "Model Inference" / "Batch Inference" tabs as checkboxes or advanced toggles: * `Force FP32 Precision (Disable is_half)` * `Offload DataLoader Caching to System RAM (CPU)` This will instantly allow thousands of users with budget or older GPUs (GTX 1060 3GB, 1050Ti, 960) to safely process full-length songs in near real-time using Windows Shared GPU Memory without crashing the pipeline. ### Additional context * **OS:** Windows 10 / 11 x64 * **Python:** 3.12+ (Python 3.12 branch) * **PyTorch:** 2.7.1+cu118 * **GPU:** NVIDIA GeForce GTX 1060 3GB

bsakaguchi added the

following up

label

2026-10-20 10:14:09 +00:00