11 Commits

Author	SHA1	Message	Date
Jaret Burkett	3f3636b788	Bug fixes and little improvements here and there.	2024-06-08 06:24:20 -06:00
Jaret Burkett	833c833f28	WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements	2024-05-27 10:50:24 -06:00
Jaret Burkett	b01e8d889a	Added stochastic rounding to adafactor. ILora adjustments	2024-03-05 07:07:09 -07:00
Jaret Burkett	a899ec91c8	Added some split prompting started code, adamw8bit, replacements improving, learnable snr gos. A lot of good stuff.	2023-11-01 06:52:21 -06:00
Jaret Burkett	73c8b50975	Added ability to use adagrad from transformers	2023-10-24 11:16:01 -06:00
Jaret Burkett	bd758ff203	Cleanup and small bug fixes	2023-08-29 05:45:49 -06:00
Jaret Burkett	8b8d53888d	Added Model rescale and prepared a release upgrade	2023-08-01 13:49:54 -06:00
Jaret Burkett	1e50b39442	Work on slider rework	2023-07-28 18:11:10 -06:00
Jaret Burkett	e6fb0229bf	Added better optimizer chooised and param support	2023-07-24 09:21:58 -06:00
Jaret Burkett	0761656a90	Added my good ole pattern loss. God I love that thing, conv transpose pattern instantly wiped from vae	2023-07-20 15:44:16 -06:00
Jaret Burkett	557732e7ff	Added Critic support to VAE training. Still tweaking and working on it. Many other fixes	2023-07-19 15:57:32 -06:00