ostris/ai-toolkit
1
0
Fork 0
mirror of https://github.com/ostris/ai-toolkit.git synced 2026-01-26 16:39:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
955 Commits 10 Branches 0 Tags
main
Commit Graph

18 Commits

Author SHA1 Message Date
Jaret Burkett
8c12977891 Fixed adafactor eps 2025-10-26 05:47:25 -06:00
Jaret Burkett
41edc18750 Removed unnessary import 2025-03-25 11:54:42 -06:00
Jaret Burkett
cbe31eaf0a Initial work on a auto adjusting optimizer 2024-11-29 04:48:58 -07:00
Jaret Burkett
894374b2e9 Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use) 2024-11-20 09:16:55 -07:00
Jaret Burkett
58f9d01c2b Added adafactor implementation that handles stochastic rounding of update and accumulation 2024-10-30 05:25:57 -06:00
Jaret Burkett
e72b59a8e9 Added experimental 8bit version of prodigy with stochastic rounding and stochastic gradient accumulation. Still testing. 2024-10-29 14:28:28 -06:00
Jaret Burkett
69aa92bce5 Added support for AdEMAMix8bit 2024-09-28 14:33:51 -06:00
Jaret Burkett
3f3636b788 Bug fixes and little improvements here and there. 2024-06-08 06:24:20 -06:00
Jaret Burkett
833c833f28 WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements 2024-05-27 10:50:24 -06:00
Jaret Burkett
b01e8d889a Added stochastic rounding to adafactor. ILora adjustments 2024-03-05 07:07:09 -07:00
Jaret Burkett
a899ec91c8 Added some split prompting started code, adamw8bit, replacements improving, learnable snr gos. A lot of good stuff. 2023-11-01 06:52:21 -06:00
Jaret Burkett
73c8b50975 Added ability to use adagrad from transformers 2023-10-24 11:16:01 -06:00
Jaret Burkett
bd758ff203 Cleanup and small bug fixes 2023-08-29 05:45:49 -06:00
Jaret Burkett
8b8d53888d Added Model rescale and prepared a release upgrade 2023-08-01 13:49:54 -06:00
Jaret Burkett
1e50b39442 Work on slider rework 2023-07-28 18:11:10 -06:00
Jaret Burkett
e6fb0229bf Added better optimizer chooised and param support 2023-07-24 09:21:58 -06:00
Jaret Burkett
0761656a90 Added my good ole pattern loss. God I love that thing, conv transpose pattern instantly wiped from vae 2023-07-20 15:44:16 -06:00
Jaret Burkett
557732e7ff Added Critic support to VAE training. Still tweaking and working on it. Many other fixes 2023-07-19 15:57:32 -06:00