[GH-ISSUE #1743] Why Very Big Spike in Learning Curve #1156

New issue

Open

opened 2026-10-20 10:06:29 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-20 10:06:29 +00:00

Owner

Originally created by @Localcrack on GitHub (Jan 19, 2024).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1743

As you see in the below image the curve is almost ok till 10K steps then it again goes to 9K steps backward and then again goes forward.
why this happened ? is something wrong ?

This spike is shows in all graphs.

Originally created by @Localcrack on GitHub (Jan 19, 2024). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/1743 As you see in the below image the curve is almost ok till 10K steps then it again goes to 9K steps backward and then again goes forward. why this happened ? is something wrong ? ![Screenshot 2024-01-19 152016](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/assets/139331483/4349cd84-634f-4f6b-9960-adcff36bafe1) This spike is shows in all graphs. ![Screenshot 2024-01-19 152102](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/assets/139331483/2cbea2ce-0f24-4a17-83f8-b2a76e4aa1ec)

bsakaguchi added the

help wanted

bug

labels

2026-10-20 10:06:29 +00:00

bsakaguchi commented

2026-10-20 10:06:30 +00:00

Author

Owner

@arbazw commented on GitHub (Jun 21, 2024):

Check your learning rate schedule. If you are using a constant learning rate, try decreasing it. Also, implement a learning rate decay if it's not already being used.

If you are resuming training from a checkpoint, ensure that the optimizer state is also being restored correctly.

@arbazw commented on GitHub (Jun 21, 2024): Check your learning rate schedule. If you are using a constant learning rate, try decreasing it. Also, implement a learning rate decay if it's not already being used. If you are resuming training from a checkpoint, ensure that the optimizer state is also being restored correctly.