18 Commits

Author	SHA1	Message	Date
Jaret Burkett	a2301cf28c	Amall bug fixes	2024-07-18 10:39:55 -06:00
Jaret Burkett	cab8a1c7b8	WIP to add the caption_proj weight to pixart sigma TE adapter	2024-07-06 13:00:21 -06:00
Jaret Burkett	5d47244c57	Added support for pixart sigma loras	2024-06-16 11:56:30 -06:00
Jaret Burkett	ada722c9e4	Fixed issue with heads not being added	2024-06-15 08:34:33 -06:00
Jaret Burkett	e3410413b9	Rework head on ilora	2024-06-14 16:21:26 -06:00
Jaret Burkett	37cebd9458	WIP Ilora	2024-06-14 09:31:01 -06:00
Jaret Burkett	bd10d2d668	Some work on sd3 training. Not working	2024-06-13 12:19:16 -06:00
Jaret Burkett	cb5d28cba9	Added working ilora trainer	2024-06-12 09:33:45 -06:00
Jaret Burkett	833c833f28	WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements	2024-05-27 10:50:24 -06:00
Jaret Burkett	68b7e159bc	Bug Fixes	2024-05-17 08:41:20 -06:00
Jaret Burkett	5a45c709cd	Work on ipadapters and custom adapters	2024-05-13 06:37:54 -06:00
Jaret Burkett	7284aab7c0	Added specialized scaler training to ip adapters	2024-04-05 08:17:09 -06:00
Jaret Burkett	016687bda1	Adapter work. Bug fixes. Auto adjust LR when resuming optimizer.	2024-03-17 10:21:47 -06:00
Jaret Burkett	b01e8d889a	Added stochastic rounding to adafactor. ILora adjustments	2024-03-05 07:07:09 -07:00
Jaret Burkett	1325613583	rework ilora	2024-02-29 07:55:52 -07:00
Jaret Burkett	e074058faa	Work on additional image embedding methods. Finalized zipper resampler. It works amazing	2024-02-10 09:00:05 -07:00
Jaret Burkett	e18e0cb5f8	Added comparitive loss when training clip encoder. Allow selecting clip layer. on ip adapter. Improvements to prior prediction	2024-02-05 07:40:03 -07:00
Jaret Burkett	92b9c71d44	Many bug fixes. Ip adapter bug fixes. Added noise to unconditional, it works better. added an ilora adapter for 1 shotting LoRAs	2024-01-28 08:20:03 -07:00