[GH-ISSUE #952] Change the Optimization Method (AdamW --> diffGrad) #613

New issue

Open

opened 2026-10-20 10:01:15 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-20 10:01:15 +00:00

Owner

Originally created by @jhchase on GitHub (Aug 6, 2023).
Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/952

Adamw was replaced by a new optimization method - diffgrad. What do you think about it? Will you add it to increase the accuracy in the experiment?

Change

    optim_g = torch.optim.AdamW(
        net_g.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )
    optim_d = torch.optim.AdamW(
        net_d.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )

pip install torch-optimizer
...

import torch_optimizer as optimizer
...

    optim_g = optimizer.DiffGrad(
        net_g.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )
    optim_d = optimizer.DiffGrad(
        net_d.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )

Source diffGrad: https://github.com/shivram1987/diffGrad

Originally created by @jhchase on GitHub (Aug 6, 2023). Original GitHub issue: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/issues/952 Adamw was replaced by a new optimization method - **diffgrad**. What do you think about it? Will you add it to increase the accuracy in the experiment? **Change** ``` optim_g = torch.optim.AdamW( net_g.parameters(), hps.train.learning_rate, betas=hps.train.betas, eps=hps.train.eps, ) optim_d = torch.optim.AdamW( net_d.parameters(), hps.train.learning_rate, betas=hps.train.betas, eps=hps.train.eps, ) ``` **To** ``` pip install torch-optimizer ... import torch_optimizer as optimizer ... optim_g = optimizer.DiffGrad( net_g.parameters(), hps.train.learning_rate, betas=hps.train.betas, eps=hps.train.eps, ) optim_d = optimizer.DiffGrad( net_d.parameters(), hps.train.learning_rate, betas=hps.train.betas, eps=hps.train.eps, ) ``` **Source diffGrad:** https://github.com/shivram1987/diffGrad

bsakaguchi added the

good first issue

label

2026-10-20 10:01:15 +00:00

bsakaguchi commented

2026-10-20 10:01:15 +00:00

Author

Owner

@ghost commented on GitHub (Nov 23, 2023):

@jhchase In which file are you changing this? Can you guide please?

@ghost commented on GitHub (Nov 23, 2023): @jhchase In which file are you changing this? Can you guide please?