10 Commits

Author	SHA1	Message	Date
Jaret Burkett	25341c4613	Got wan 14b training to work on 24GB card.	2025-03-07 17:04:10 -07:00
Jaret Burkett	4fe33f51c1	Fix issue with picking layers for quantization, adjust layers fo better quantization of cogview4	2025-03-05 13:44:40 -07:00
Jaret Burkett	6f6fb90812	Added cogview4. Loss still needs work.	2025-03-04 18:43:52 -07:00
Jaret Burkett	acc79956aa	WIP create new class to add new models more easily	2025-03-01 13:49:02 -07:00
Jaret Burkett	58f9d01c2b	Added adafactor implementation that handles stochastic rounding of update and accumulation	2024-10-30 05:25:57 -06:00
Jaret Burkett	5d47244c57	Added support for pixart sigma loras	2024-06-16 11:56:30 -06:00
Jaret Burkett	3f3636b788	Bug fixes and little improvements here and there.	2024-06-08 06:24:20 -06:00
Jaret Burkett	5a70b7f38d	Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded.	2024-04-20 10:46:56 -06:00
Jaret Burkett	427847ac4c	Small tweaks and fixes for specialized ip adapter training	2024-03-26 11:35:26 -06:00
Jaret Burkett	b01e8d889a	Added stochastic rounding to adafactor. ILora adjustments	2024-03-05 07:07:09 -07:00