RVC-Project/Retrieval-based-Voice-Conversion-WebUI

Fork 0

mirror of https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI.git synced 2026-11-05 00:27:30 +00:00

[GH-ISSUE #1678] Warning: Grad strides do not match bucket view strides DDP. (Can impact performance) #1117

New issue

Open

opened 2026-10-20 10:06:06 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-20 10:06:06 +00:00

Owner

Originally created by @blaisewf on GitHub (Jan 3, 2024).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1678

Warning when attempting to start training:

...\lib\site-packages\torch\autograd\__init__.py:251: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed.  This is not an error, but may impair performance.
grad.sizes() = [64, 1, 4], strides() = [4, 1, 1]
bucket_view.sizes() = [64, 1, 4], strides() = [4, 4, 1] (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\distributed\c10d\reducer.cpp:334.)
  Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass

Maybe this can help https://github.com/pytorch/pytorch/issues/47163 & https://github.com/noahzn/Lite-Mono/issues/43

Originally created by @blaisewf on GitHub (Jan 3, 2024). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1678 Warning when attempting to start training: ``` ...\lib\site-packages\torch\autograd\__init__.py:251: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance. grad.sizes() = [64, 1, 4], strides() = [4, 1, 1] bucket_view.sizes() = [64, 1, 4], strides() = [4, 4, 1] (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\distributed\c10d\reducer.cpp:334.) Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass ``` Maybe this can help https://github.com/pytorch/pytorch/issues/47163 & https://github.com/noahzn/Lite-Mono/issues/43

bsakaguchi added the

help wanted

bug

question

labels

2026-10-20 10:06:06 +00:00

bsakaguchi commented

2026-10-20 10:06:06 +00:00

Author

Owner

@BitCalSaul commented on GitHub (Jan 13, 2024):

I encounter the same problem when i used ddp with batch size =2.

UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed.  This is not an error, but may impair performance.
grad.sizes() = [180, 6, 1, 1], strides() = [6, 1, 6, 6]
bucket_view.sizes() = [180, 6, 1, 1], strides() = [6, 1, 1, 1] (Triggered internally at `/opt/conda/conda-bld/pytorch_1695392020195/work/torch/csrc/distributed/c10d/reducer.cpp:320.)
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass

@BitCalSaul commented on GitHub (Jan 13, 2024): I encounter the same problem when i used ddp with batch size =2. ``` UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance. grad.sizes() = [180, 6, 1, 1], strides() = [6, 1, 6, 6] bucket_view.sizes() = [180, 6, 1, 1], strides() = [6, 1, 1, 1] (Triggered internally at `/opt/conda/conda-bld/pytorch_1695392020195/work/torch/csrc/distributed/c10d/reducer.cpp:320.) Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass ```

bsakaguchi commented

2026-10-20 10:06:07 +00:00

Author

Owner

@blaisewf commented on GitHub (Apr 25, 2024):

@fumiama sorry but do you know anything about this warnings? seems to be in a lot of VITS implementations

@blaisewf commented on GitHub (Apr 25, 2024): @fumiama sorry but do you know anything about this warnings? seems to be in a lot of VITS implementations

bsakaguchi commented

2026-10-20 10:06:07 +00:00

Author

Owner

@fumiama commented on GitHub (Apr 28, 2024):

Well, sorry but I also have no idea about it because I have never met this error 😂.

@fumiama commented on GitHub (Apr 28, 2024): Well, sorry but I also have no idea about it because I have never met this error 😂.

bsakaguchi commented

2026-10-20 10:06:07 +00:00

Author

Owner

@blaisewf commented on GitHub (Apr 29, 2024):

Well, sorry but I also have no idea about it because I have never met this error 😂.

It happens to everyone at the beginning of training, are you sure it never happened to you?

@blaisewf commented on GitHub (Apr 29, 2024): > Well, sorry but I also have no idea about it because I have never met this error 😂. It happens to everyone at the beginning of training, are you sure it never happened to you?

bsakaguchi commented

2026-10-20 10:06:07 +00:00

Author

Owner

@fumiama commented on GitHub (Apr 29, 2024):

Well, maybe I have missed some logs😂. I will re-check it later.

@fumiama commented on GitHub (Apr 29, 2024): Well, maybe I have missed some logs😂. I will re-check it later.

bsakaguchi commented

2026-10-20 10:06:07 +00:00

Author

Owner

@blaisewf commented on GitHub (May 16, 2024):

Well, maybe I have missed some logs😂. I will re-check it later.

Maybe this is useful https://chat.openai.com/share/bf7a4bd0-3799-47aa-98bb-666ec4785a4b

@blaisewf commented on GitHub (May 16, 2024): > Well, maybe I have missed some logs😂. I will re-check it later. Maybe this is useful https://chat.openai.com/share/bf7a4bd0-3799-47aa-98bb-666ec4785a4b

bsakaguchi commented

2026-10-20 10:06:07 +00:00

Author

Owner

@marktellez commented on GitHub (Aug 29, 2024):

version 2 does this for me, v1 works

@marktellez commented on GitHub (Aug 29, 2024): version 2 does this for me, v1 works