[GH-ISSUE #1622] "torch.distributed.DistNetworkError: Unknown error" when training #1079

New issue

Closed

opened 2026-10-20 10:05:45 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-20 10:05:45 +00:00

Owner

Originally created by @PedroSophiaaa on GitHub (Dec 16, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1622

Hello, I've inserted the path correctely.
But, when i then start training, it throws pytorch error "Unknown error"
whole log:

INFO:Vini:{'train': {'log_interval': 200, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0001, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 4, 'fp16_run': False, 'lr_decay': 0.999875, 'segment_size': 12800, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'max_wav_value': 32768.0, 'sampling_rate': 40000, 'filter_length': 2048, 'hop_length': 400, 'win_length': 2048, 'n_mel_channels': 125, 'mel_fmin': 0.0, 'mel_fmax': None, 'training_files': './logs\\Vini/filelist.txt'}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': [[1, 3, 5], [1, 3, 5], [1, 3, 5]], 'upsample_rates': [10, 10, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'use_spectral_norm': False, 'gin_channels': 256, 'spk_embed_dim': 109}, 'model_dir': './logs\\Vini', 'experiment_dir': './logs\\Vini', 'save_every_epoch': 5, 'name': 'Vini', 'total_epoch': 20, 'pretrainG': 'pretrained/f0G40k.pth', 'pretrainD': 'pretrained/f0D40k.pth', 'version': 'v1', 'gpus': '0', 'sample_rate': '40k', 'if_f0': 1, 'if_latest': 0, 'save_every_weights': '0', 'if_cache_data_in_gpu': 0}
Process Process-1:
Traceback (most recent call last):
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Main\RVC\train_nsf_sim_cache_sid_load_pretrain.py", line 100, in run
    dist.init_process_group(
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\c10d_logger.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\c10d_logger.py", line 89, in wrapper
    func_return = func(*args, **kwargs)
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\distributed_c10d.py", line 1228, in init_process_group
    store, rank, world_size = next(rendezvous_iterator)
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\rendezvous.py", line 246, in _env_rendezvous_handler
    store = _create_c10d_store(master_addr, master_port, rank, world_size, timeout, use_libuv)
  File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\rendezvous.py", line 174, in _create_c10d_store
    return TCPStore(
torch.distributed.DistNetworkError: Unknown error

What should I do?

system: windows 11 64bit

GPU: Nvidia RTX 3070 TI laptop

Originally created by @PedroSophiaaa on GitHub (Dec 16, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1622 Hello, I've inserted the path correctely. But, when i then start training, it throws pytorch error "Unknown error" whole log: ``` INFO:Vini:{'train': {'log_interval': 200, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0001, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 4, 'fp16_run': False, 'lr_decay': 0.999875, 'segment_size': 12800, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'max_wav_value': 32768.0, 'sampling_rate': 40000, 'filter_length': 2048, 'hop_length': 400, 'win_length': 2048, 'n_mel_channels': 125, 'mel_fmin': 0.0, 'mel_fmax': None, 'training_files': './logs\\Vini/filelist.txt'}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': [[1, 3, 5], [1, 3, 5], [1, 3, 5]], 'upsample_rates': [10, 10, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'use_spectral_norm': False, 'gin_channels': 256, 'spk_embed_dim': 109}, 'model_dir': './logs\\Vini', 'experiment_dir': './logs\\Vini', 'save_every_epoch': 5, 'name': 'Vini', 'total_epoch': 20, 'pretrainG': 'pretrained/f0G40k.pth', 'pretrainD': 'pretrained/f0D40k.pth', 'version': 'v1', 'gpus': '0', 'sample_rate': '40k', 'if_f0': 1, 'if_latest': 0, 'save_every_weights': '0', 'if_cache_data_in_gpu': 0} Process Process-1: Traceback (most recent call last): File "C:\Users\pedro\anaconda3\envs\RVC\lib\multiprocessing\process.py", line 315, in _bootstrap self.run() File "C:\Users\pedro\anaconda3\envs\RVC\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "C:\Main\RVC\train_nsf_sim_cache_sid_load_pretrain.py", line 100, in run dist.init_process_group( File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\c10d_logger.py", line 75, in wrapper return func(*args, **kwargs) File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\c10d_logger.py", line 89, in wrapper func_return = func(*args, **kwargs) File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\distributed_c10d.py", line 1228, in init_process_group store, rank, world_size = next(rendezvous_iterator) File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\rendezvous.py", line 246, in _env_rendezvous_handler store = _create_c10d_store(master_addr, master_port, rank, world_size, timeout, use_libuv) File "C:\Users\pedro\anaconda3\envs\RVC\lib\site-packages\torch\distributed\rendezvous.py", line 174, in _create_c10d_store return TCPStore( torch.distributed.DistNetworkError: Unknown error ``` What should I do? system: windows 11 64bit GPU: Nvidia RTX 3070 TI laptop

bsakaguchi

2026-10-20 10:05:45 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-20 10:05:46 +00:00

Author

Owner

@Scheigster commented on GitHub (Dec 19, 2023):

I had the same problem today after I installed another gradio tool.
Most likely this is due to one of our python dependency not being at the correct version.

As a workaround, I installed RVC with Pinokio, it "fixed" my problem for now. If you don't know Pinokio, it's basically an open-source app making it super easy to install many tools like RVC. Good thing is it takes care of creating new environment for each app so not-so-knowledgeable people like me don't have to deal with dependencies and conflicts.

Edit: not an ad for Pinokio btw, I have nothing to do with it. Just putting that out there in case people who stumble upon this page like I did know about it. There is a surely a better way to solve that issue which do not require installing an external tool.

@Scheigster commented on GitHub (Dec 19, 2023): I had the same problem today after I installed another gradio tool. Most likely this is due to one of our python dependency not being at the correct version. As a workaround, I installed RVC with [Pinokio](https://pinokio.computer/), it "fixed" my problem for now. If you don't know Pinokio, it's basically an open-source app making it super easy to install many tools like RVC. Good thing is it takes care of creating new environment for each app so _not-so-knowledgeable_ people like me don't have to deal with dependencies and conflicts. _Edit: not an ad for Pinokio btw, I have nothing to do with it. Just putting that out there in case people who stumble upon this page like I did know about it. There is a surely a better way to solve that issue which do not require installing an external tool._

bsakaguchi commented

2026-10-20 10:05:46 +00:00

Author

Owner

@trivikramak commented on GitHub (Feb 19, 2024):

Getting the same error with Pinokio as well

@trivikramak commented on GitHub (Feb 19, 2024): Getting the same error with Pinokio as well

bsakaguchi commented

2026-10-20 10:05:46 +00:00

Author

Owner

@skeddles commented on GitHub (Feb 20, 2024):

im getting the same error. i tried reinstalling with pinokio, and while the installation was so much easier, which i really appreciate, it unfortunately resulted in the same exact problem.

the training says "All processes have been completed!" after just a few seconds, and the command line says:

Process Process-1:
Traceback (most recent call last):
  File "U:\AI\pinokio\bin\miniconda\lib\multiprocessing\process.py", line 314, in _bootstrap
    self.run()
  File "U:\AI\pinokio\bin\miniconda\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "U:\AI\pinokio\api\rvc.pinokio.git\app\infer\modules\train\train.py", line 129, in run
    dist.init_process_group(
  File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\c10d_logger.py", 
line 86, in wrapper
    func_return = func(*args, **kwargs)
  File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\distributed_c10d.
py", line 1177, in init_process_group
    store, rank, world_size = next(rendezvous_iterator)
  File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\rendezvous.py", l
ine 246, in _env_rendezvous_handler
    store = _create_c10d_store(master_addr, master_port, rank, world_size, timeout, use_libuv)
  File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\rendezvous.py", l
ine 174, in _create_c10d_store
    return TCPStore(
torch.distributed.DistNetworkError: Unknown error

windows 10 / NVIDIA GeForce GTX 1080 Ti

@skeddles commented on GitHub (Feb 20, 2024): im getting the same error. i tried reinstalling with pinokio, and while the installation was so much easier, which i really appreciate, it unfortunately resulted in the same exact problem. the training says "All processes have been completed!" after just a few seconds, and the command line says: ``` Process Process-1: Traceback (most recent call last): File "U:\AI\pinokio\bin\miniconda\lib\multiprocessing\process.py", line 314, in _bootstrap self.run() File "U:\AI\pinokio\bin\miniconda\lib\multiprocessing\process.py", line 108, in run self._target(*self._args, **self._kwargs) File "U:\AI\pinokio\api\rvc.pinokio.git\app\infer\modules\train\train.py", line 129, in run dist.init_process_group( File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\c10d_logger.py", line 86, in wrapper func_return = func(*args, **kwargs) File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\distributed_c10d. py", line 1177, in init_process_group store, rank, world_size = next(rendezvous_iterator) File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\rendezvous.py", l ine 246, in _env_rendezvous_handler store = _create_c10d_store(master_addr, master_port, rank, world_size, timeout, use_libuv) File "U:\AI\pinokio\api\rvc.pinokio.git\app\env\lib\site-packages\torch\distributed\rendezvous.py", l ine 174, in _create_c10d_store return TCPStore( torch.distributed.DistNetworkError: Unknown error ``` windows 10 / NVIDIA GeForce GTX 1080 Ti

bsakaguchi commented

2026-10-20 10:05:46 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Apr 28, 2024):

This issue was closed because it has been inactive for 15 days since being marked as stale.

@github-actions[bot] commented on GitHub (Apr 28, 2024): This issue was closed because it has been inactive for 15 days since being marked as stale.