[GH-ISSUE #860] Prodigy Optimizer. #555

New issue

Open

opened 2026-10-20 10:00:35 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-20 10:00:35 +00:00

Owner

Originally created by @artificialguybr on GitHub (Jul 25, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/860

Hello!

First of all, I would like to thank the devs for their effort in maintaining this repository.

I would like to suggest an update.

Update the optimizer to Prodigy.

Prodigy is a new optimizer that converges much faster and easier than Adam. It is the direct evolution of Dadptation.

Prodigy has been widely used in Stable Diffusion now for delivering faster results and allowing the choice of LR to be automatic.

I implemented PRODIGY in my local environment and it worked very well.

It's not very difficult to implement and modify.

In this repository, you can find the paper and the requirements to use Prodigy.

From optimize args I used the standards we are currently using in SD.
"decouple=True" "weight_decay=0.01" "d_coef=2" "use_bias_correction=True" "safeguard_warmup=True"

I really think this can speed up model training and deliver better results. It would also be recommended to trigger an early stop system to lock the training at the moment it starts to want to overfit.

I also implemented in my local env a wandb integration, but for that i removed the tensorboard. So i dont know if it valid for here, but i like more Wandb since we can acess it from anywhere.

https://github.com/konstmish/prodigy

So that's it.

Any doubts you can contact me!

Have a good day!

Originally created by @artificialguybr on GitHub (Jul 25, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/860 Hello! First of all, I would like to thank the devs for their effort in maintaining this repository. I would like to suggest an update. Update the optimizer to Prodigy. Prodigy is a new optimizer that converges much faster and easier than Adam. It is the direct evolution of Dadptation. Prodigy has been widely used in Stable Diffusion now for delivering faster results and allowing the choice of LR to be automatic. I implemented PRODIGY in my local environment and it worked very well. It's not very difficult to implement and modify. In this repository, you can find the paper and the requirements to use Prodigy. From optimize args I used the standards we are currently using in SD. "decouple=True" "weight_decay=0.01" "d_coef=2" "use_bias_correction=True" "safeguard_warmup=True" I really think this can speed up model training and deliver better results. It would also be recommended to trigger an early stop system to lock the training at the moment it starts to want to overfit. I also implemented in my local env a wandb integration, but for that i removed the tensorboard. So i dont know if it valid for here, but i like more Wandb since we can acess it from anywhere. https://github.com/konstmish/prodigy So that's it. Any doubts you can contact me! Have a good day!

bsakaguchi added the

good first issue

label

2026-10-20 10:00:35 +00:00

bsakaguchi commented

2026-10-20 10:00:35 +00:00

Author

Owner

@TechInterMezzo commented on GitHub (Oct 21, 2023):

It would also be recommended to trigger an early stop system to lock the training at the moment it starts to want to overfit.

How would you do the detection of overfitting? Any tips?

@TechInterMezzo commented on GitHub (Oct 21, 2023): > It would also be recommended to trigger an early stop system to lock the training at the moment it starts to want to overfit. How would you do the detection of overfitting? Any tips?