Jaret Burkett
|
894374b2e9
|
Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use)
|
2024-11-20 09:16:55 -07:00 |
|
Jaret Burkett
|
58f9d01c2b
|
Added adafactor implementation that handles stochastic rounding of update and accumulation
|
2024-10-30 05:25:57 -06:00 |
|
Jaret Burkett
|
e72b59a8e9
|
Added experimental 8bit version of prodigy with stochastic rounding and stochastic gradient accumulation. Still testing.
|
2024-10-29 14:28:28 -06:00 |
|
Jaret Burkett
|
69aa92bce5
|
Added support for AdEMAMix8bit
|
2024-09-28 14:33:51 -06:00 |
|
Jaret Burkett
|
3f3636b788
|
Bug fixes and little improvements here and there.
|
2024-06-08 06:24:20 -06:00 |
|
Jaret Burkett
|
833c833f28
|
WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements
|
2024-05-27 10:50:24 -06:00 |
|
Jaret Burkett
|
b01e8d889a
|
Added stochastic rounding to adafactor. ILora adjustments
|
2024-03-05 07:07:09 -07:00 |
|
Jaret Burkett
|
a899ec91c8
|
Added some split prompting started code, adamw8bit, replacements improving, learnable snr gos. A lot of good stuff.
|
2023-11-01 06:52:21 -06:00 |
|
Jaret Burkett
|
73c8b50975
|
Added ability to use adagrad from transformers
|
2023-10-24 11:16:01 -06:00 |
|
Jaret Burkett
|
bd758ff203
|
Cleanup and small bug fixes
|
2023-08-29 05:45:49 -06:00 |
|
Jaret Burkett
|
8b8d53888d
|
Added Model rescale and prepared a release upgrade
|
2023-08-01 13:49:54 -06:00 |
|
Jaret Burkett
|
1e50b39442
|
Work on slider rework
|
2023-07-28 18:11:10 -06:00 |
|
Jaret Burkett
|
e6fb0229bf
|
Added better optimizer chooised and param support
|
2023-07-24 09:21:58 -06:00 |
|
Jaret Burkett
|
0761656a90
|
Added my good ole pattern loss. God I love that thing, conv transpose pattern instantly wiped from vae
|
2023-07-20 15:44:16 -06:00 |
|
Jaret Burkett
|
557732e7ff
|
Added Critic support to VAE training. Still tweaking and working on it. Many other fixes
|
2023-07-19 15:57:32 -06:00 |
|