[GH-ISSUE #437] IndexError and UnpicklingError during model training with pretrained weights #227

Open
opened 2026-10-20 09:56:54 +00:00 by bsakaguchi · 5 comments
Owner

Originally created by @Setmaster on GitHub (Jun 5, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/437

When trying to train I encountered two errors: an IndexError related to the list index being out of range and an _pickle.UnpicklingError regarding an invalid load key.

INFO:mi-test:{'train': {'log_interval': 200, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0001, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 40, 'fp16_run': True, 'lr_decay': 0.999875, 'segment_size': 12800, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'max_wav_value': 32768.0, 'sampling_rate': 40000, 'filter_length': 2048, 'hop_length': 400, 'win_length': 2048, 'n_mel_channels': 125, 'mel_fmin': 0.0, 'mel_fmax': None, 'training_files': './logs/mi-test/filelist.txt'}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': 1, 3, 5], [1, 3, 5], [1, 3, 5, 'upsample_rates': [10, 10, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'use_spectral_norm': False, 'gin_channels': 256, 'spk_embed_dim': 109}, 'model_dir': './logs/mi-test', 'experiment_dir': './logs/mi-test', 'save_every_epoch': 10, 'name': 'mi-test', 'total_epoch': 200, 'pretrainG': 'pretrained/f0G40k.pth', 'pretrainD': 'pretrained/f0D40k.pth', 'version': 'v1', 'gpus': '0', 'sample_rate': '40k', 'if_f0': 1, 'if_latest': 1, 'save_every_weights': '1', 'if_cache_data_in_gpu': 0}
INFO:torch.distributed.distributed_c10d:Added key: store_based_barrier_key:1 to store for rank: 0
INFO:torch.distributed.distributed_c10d:Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes.
gin_channels: 256 self.spk_embed_dim: 109
INFO:mi-test:loaded pretrained pretrained/f0G40k.pth pretrained/f0D40k.pth
Process Process-1:
Traceback (most recent call last):
File "/mnt/c/users/john/documents/Retrieval-based-Voice-Conversion-WebUI/train_nsf_sim_cache_sid_load_pretrain.py", line 178, in run
utils.latest_checkpoint_path(hps.model_dir, "D_*.pth"), net_d, optim_d
File "/mnt/c/users/john/documents/Retrieval-based-Voice-Conversion-WebUI/train/utils.py", line 206, in latest_checkpoint_path
x = f_list[-1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/mnt/c/users/john/documents/Retrieval-based-Voice-Conversion-WebUI/train_nsf_sim_cache_sid_load_pretrain.py", line 197, in run
torch.load(hps.pretrainG, map_location="cpu")["model"]
File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/site-packages/torch/serialization.py", line 815, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

Originally created by @Setmaster on GitHub (Jun 5, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/437 When trying to train I encountered two errors: an `IndexError` related to the list index being out of range and an `_pickle.UnpicklingError` regarding an invalid load key. > INFO:mi-test:{'train': {'log_interval': 200, 'seed': 1234, 'epochs': 20000, 'learning_rate': 0.0001, 'betas': [0.8, 0.99], 'eps': 1e-09, 'batch_size': 40, 'fp16_run': True, 'lr_decay': 0.999875, 'segment_size': 12800, 'init_lr_ratio': 1, 'warmup_epochs': 0, 'c_mel': 45, 'c_kl': 1.0}, 'data': {'max_wav_value': 32768.0, 'sampling_rate': 40000, 'filter_length': 2048, 'hop_length': 400, 'win_length': 2048, 'n_mel_channels': 125, 'mel_fmin': 0.0, 'mel_fmax': None, 'training_files': './logs/mi-test/filelist.txt'}, 'model': {'inter_channels': 192, 'hidden_channels': 192, 'filter_channels': 768, 'n_heads': 2, 'n_layers': 6, 'kernel_size': 3, 'p_dropout': 0, 'resblock': '1', 'resblock_kernel_sizes': [3, 7, 11], 'resblock_dilation_sizes': [[1, 3, 5], [1, 3, 5], [1, 3, 5]], 'upsample_rates': [10, 10, 2, 2], 'upsample_initial_channel': 512, 'upsample_kernel_sizes': [16, 16, 4, 4], 'use_spectral_norm': False, 'gin_channels': 256, 'spk_embed_dim': 109}, 'model_dir': './logs/mi-test', 'experiment_dir': './logs/mi-test', 'save_every_epoch': 10, 'name': 'mi-test', 'total_epoch': 200, 'pretrainG': 'pretrained/f0G40k.pth', 'pretrainD': 'pretrained/f0D40k.pth', 'version': 'v1', 'gpus': '0', 'sample_rate': '40k', 'if_f0': 1, 'if_latest': 1, 'save_every_weights': '1', 'if_cache_data_in_gpu': 0} > INFO:torch.distributed.distributed_c10d:Added key: store_based_barrier_key:1 to store for rank: 0 > INFO:torch.distributed.distributed_c10d:Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes. > gin_channels: 256 self.spk_embed_dim: 109 > INFO:mi-test:loaded pretrained pretrained/f0G40k.pth pretrained/f0D40k.pth > Process Process-1: > Traceback (most recent call last): > File "/mnt/c/users/john/documents/Retrieval-based-Voice-Conversion-WebUI/train_nsf_sim_cache_sid_load_pretrain.py", line 178, in run > utils.latest_checkpoint_path(hps.model_dir, "D_*.pth"), net_d, optim_d > File "/mnt/c/users/john/documents/Retrieval-based-Voice-Conversion-WebUI/train/utils.py", line 206, in latest_checkpoint_path > x = f_list[-1] > IndexError: list index out of range > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap > self.run() > File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/multiprocessing/process.py", line 108, in run > self._target(*self._args, **self._kwargs) > File "/mnt/c/users/john/documents/Retrieval-based-Voice-Conversion-WebUI/train_nsf_sim_cache_sid_load_pretrain.py", line 197, in run > torch.load(hps.pretrainG, map_location="cpu")["model"] > File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/site-packages/torch/serialization.py", line 815, in load > return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) > File "/home/se7dev/miniconda3/envs/RBVC/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, 'v'.
Author
Owner

@fjibj commented on GitHub (Jun 21, 2023):

me too.

<!-- gh-comment-id:1599947168 --> @fjibj commented on GitHub (Jun 21, 2023): me too.
Author
Owner

@brunolm commented on GitHub (Jul 8, 2023):

Got same error when trying to separate voice/instrument

_pickle.UnpicklingError: invalid load key, 'v'.

related?
image


Note: no errors downloading the 7z file and extracting everything instead

<!-- gh-comment-id:1626813994 --> @brunolm commented on GitHub (Jul 8, 2023): Got same error when trying to separate voice/instrument ``` _pickle.UnpicklingError: invalid load key, 'v'. ``` related? ![image](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/assets/706078/bef0bcfe-d285-4855-9e40-a508be359f4c) --- Note: no errors downloading the 7z file and extracting everything instead
Author
Owner

@mattdisk commented on GitHub (Jul 10, 2023):

That is because the pre-trained model is saved as LFS (Large File Storage) on HuggingFace.

If you only use "git clone" or "curl", it will only give you an html file rather than the real document.
You can easily realize that by looking at the file size. Normally the model takes around 50MB ~200MB. However, the html file is only several hundred KB.
That is why it shows something like "_pickle.UnpicklingError: invalid load key, '<'." This "<" is from the html file.

The solution is to use "git lfs".
I think this mistake is very common if someone is a beginner, and we should mention it in the readme document.

<!-- gh-comment-id:1628162159 --> @mattdisk commented on GitHub (Jul 10, 2023): That is because the pre-trained model is saved as LFS (Large File Storage) on HuggingFace. If you only use "git clone" or "curl", it will only give you an html file rather than the real document. You can easily realize that by looking at the file size. Normally the model takes around 50MB ~200MB. However, the html file is only several hundred KB. That is why it shows something like "_pickle.UnpicklingError: invalid load key, '<'." This "<" is from the html file. The solution is to use "[git lfs](https://git-lfs.com/)". I think this mistake is very common if someone is a beginner, and we should mention it in the readme document.
Author
Owner

@EvilFireLee commented on GitHub (Dec 27, 2023):

I encountered this issue today. In my situation, I know it has nothing to do with HTML or FLS because I just finished training my other model. As it turns out, you cannot name your experiment with a name like XXXXXV3. Somehow, the RVC thinks you are attempting to find a Version3 pretrained model that doesn't exist yet. So, my takeaway is not to name your experiment with anything related to numbers and common coding instructions.

<!-- gh-comment-id:1869898236 --> @EvilFireLee commented on GitHub (Dec 27, 2023): I encountered this issue today. In my situation, I know it has nothing to do with HTML or FLS because I just finished training my other model. As it turns out, you cannot name your experiment with a name like XXXXXV3. Somehow, the RVC thinks you are attempting to find a Version3 pretrained model that doesn't exist yet. So, my takeaway is not to name your experiment with anything related to numbers and common coding instructions.
Author
Owner

@augustohsales commented on GitHub (Apr 20, 2024):

I resolved this error by following this tutorial https://www.youtube.com/watch?v=nXpBlC6OBw4&t=1119s

The error occurs because you must have some virtual environment with the name env in anaconda. This causes the env subfolder to not be created in Retrieval-based-Voice-Conversion-WebUI. Replace the command "python - m venv env" with python - m venv [your_name] then type your_name\Scripts\activate before executing the other commands:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 our cu118 and
ip install -r requirements.txt and enjoy

<!-- gh-comment-id:2067668426 --> @augustohsales commented on GitHub (Apr 20, 2024): I resolved this error by following this tutorial https://www.youtube.com/watch?v=nXpBlC6OBw4&t=1119s The error occurs because you must have some virtual environment with the name env in anaconda. This causes the env subfolder to not be created in Retrieval-based-Voice-Conversion-WebUI. Replace the command "python - m venv env" with python - m venv [your_name] then type your_name\Scripts\activate before executing the other commands: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 our cu118 and ip install -r requirements.txt and enjoy
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
RVC-Project/Retrieval-based-Voice-Conversion-WebUI#227
No description provided.