473 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Jaret Burkett	4c8a9e1b88	Added example config to train Flex	2025-01-18 18:03:20 -07:00
Jaret Burkett	fadb2f3a76	Allow quantizing the te independently on flux. added lognorm_blend timestep schedule	2025-01-18 18:02:31 -07:00
Jaret Burkett	4723f23c0d	Added ability to split up flux across gpus (experimental). Changed the way timestep scheduling works to prep for more specific schedules.	2024-12-31 07:06:55 -07:00
Jaret Burkett	8ef07a9c36	Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications	2024-12-15 08:59:27 -07:00
Jaret Burkett	92ce93140e	Adjustments to defaults for automagic	2024-11-29 10:28:06 -07:00
Jaret Burkett	f213996aa5	Fixed saving and displaying for automagic	2024-11-29 08:00:22 -07:00
Jaret Burkett	cbe31eaf0a	Initial work on a auto adjusting optimizer	2024-11-29 04:48:58 -07:00
Jaret Burkett	67c2e44edb	Added support for training flux redux adapters	2024-11-21 20:01:52 -07:00
Jaret Burkett	96d418bb95	Added support for full finetuning flux with randomized param activation. Examples coming soon	2024-11-21 13:05:32 -07:00
Jaret Burkett	894374b2e9	Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use)	2024-11-20 09:16:55 -07:00
Jaret Burkett	6509ba4484	Fix seed generation to make it deterministic so it is consistant from gpu to gpu	2024-11-15 12:11:13 -07:00
Jaret Burkett	025ee3dd3d	Added ability for adafactor to fully fine tune quantized model.	2024-10-30 16:38:07 -06:00
Jaret Burkett	58f9d01c2b	Added adafactor implementation that handles stochastic rounding of update and accumulation	2024-10-30 05:25:57 -06:00
Jaret Burkett	e72b59a8e9	Added experimental 8bit version of prodigy with stochastic rounding and stochastic gradient accumulation. Still testing.	2024-10-29 14:28:28 -06:00
Jaret Burkett	4aa19b5c1d	Only quantize flux T5 is also quantizing model. Load TE from original name and path if fine tuning.	2024-10-29 14:25:31 -06:00
Jaret Burkett	4747716867	Fixed issue with adapters not providing gradients with new grad activator	2024-10-29 14:22:10 -06:00
Jaret Burkett	22cd40d7b9	Improvements for full tuning flux. Added debugging launch config for vscode	2024-10-29 04:54:08 -06:00
Jaret Burkett	3400882a80	Added preliminary support for SD3.5-large lora training	2024-10-22 12:21:36 -06:00
Jaret Burkett	9f94c7b61e	Added experimental param multiplier to the ema module	2024-10-22 09:25:52 -06:00
Jaret Burkett	bedb8197a2	Fixed issue with sizes for some images being loaded sideways resulting in squished images.	2024-10-20 11:51:29 -06:00
Jaret Burkett	e3ebd73610	Add a projection layer on vision direct when doing image embeds	2024-10-20 10:48:23 -06:00
Jaret Burkett	dd931757cd	Merge branch 'main' of github.com:ostris/ai-toolkit	2024-10-20 07:04:29 -06:00
Jaret Burkett	0640cdf569	Handle errors in loading size database	2024-10-20 07:04:19 -06:00
Jaret Burkett	0b048d0dde	Locked version of quanto as it breaks in later versions	2024-10-16 22:41:04 +00:00
Jaret Burkett	473d455f44	Process empty clip image if there is not one for reg images when training a custom adapter	2024-10-15 08:28:04 -06:00
Jaret Burkett	ce759ebd8c	Normalize the image embeddings on vd adapter forward	2024-10-12 15:09:48 +00:00
Jaret Burkett	628a7923a3	Remove norm on image embeds on custom adapter	2024-10-12 00:43:18 +00:00
Jaret Burkett	3922981996	Added some additional experimental things to the vision direct encoder	2024-10-10 19:42:26 +00:00
Jaret Burkett	ab22674980	Allow for a default caption file in the folder. Minor bug fixes.	2024-10-10 07:31:33 -06:00
Jaret Burkett	9452929300	Apply a mask to the embeds for SD if using T5 encoder	2024-10-04 10:55:20 -06:00
Jaret Burkett	a800c9d19e	Add a method to have an inference only lora	2024-10-04 10:06:53 -06:00
Jaret Burkett	28e6f00790	Fixed bug in returning clip image embed to actually return it	2024-10-03 10:49:09 -06:00
Jaret Burkett	67e0aca750	Added ability to load clip pairs randomly from folder. Other small bug fixes	2024-10-03 10:03:49 -06:00
Jaret Burkett	f05224970f	Added Vision Languate Adapter usage for pixtral vd adapter	2024-09-29 19:39:56 -06:00
Jaret Burkett	b4f64de4c2	Quick patch to scope xformer imports until a better solution	2024-09-28 15:36:42 -06:00
Jaret Burkett	2e5f6668dc	Add xformers ad a dependency	2024-09-28 15:30:14 -06:00
Jaret Burkett	e4c82803e1	Handle random resizing for pixtral input on direct vision adapter	2024-09-28 14:53:38 -06:00
Jaret Burkett	69aa92bce5	Added support for AdEMAMix8bit	2024-09-28 14:33:51 -06:00
Jaret Burkett	a508caad1d	Change pixtral to crop based on number of pixels instead of largest dimension	2024-09-28 13:05:26 -06:00
Jaret Burkett	58537fc92b	Added initial direct vision pixtral support	2024-09-28 10:47:51 -06:00
Jaret Burkett	86b5938cf3	Fixed the webp bug finally.	2024-09-25 13:56:00 -06:00
Jaret Burkett	6b4034122f	REmove layers from direct vision resampler	2024-09-24 15:08:29 -06:00
Jaret Burkett	10817696fb	Fixed issue where direct vision was not passing additional params from resampler when it is added	2024-09-24 10:34:11 -06:00
Jaret Burkett	037ce11740	Always return vision encoder in state dict	2024-09-24 07:43:17 -06:00
Jaret Burkett	04424fe2d6	Added config setting to set the timestep type	2024-09-24 06:53:59 -06:00
Jaret Burkett	40a8ff5731	Load local hugging face packages for assistant adapter	2024-09-23 10:37:12 -06:00
Jaret Burkett	2776221497	Added option to cache empty prompt or trigger and unload text encoders while training	2024-09-21 20:54:09 -06:00
Jaret Burkett	f85ad452c6	Added initial support for pixtral vision as a vision encoder.	2024-09-21 15:21:14 -06:00
Jaret Burkett	dd889086f4	Updates to the docker file for jupyterlab	2024-09-21 12:07:07 -06:00
apolinário	bc693488eb	fix diffusers codebase (#183)	2024-09-21 11:50:29 -06:00