ostris/ai-toolkit
1
0
Fork 0
mirror of https://github.com/ostris/ai-toolkit.git synced 2026-01-26 16:39:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
483 Commits 10 Branches 0 Tags
5e663746b83de64d5ca239fa699cf59a8a268ba5
Commit Graph

388 Commits

Author SHA1 Message Date
Jaret Burkett
5e663746b8 Working multi gpu training. Still need a lot of tweaks and testing. 2025-01-25 16:46:20 -07:00
Jaret Burkett
bbfba0c188 Added v2 of dfp 2025-01-22 16:32:13 -07:00
Jaret Burkett
e1549ad54d Update dfe model arch 2025-01-22 10:37:23 -07:00
Jaret Burkett
04abe57c76 Added weighing to DFE 2025-01-22 08:50:57 -07:00
Jaret Burkett
89dd041b97 Added ability to pair samples with a closer noise with optimal_noise_pairing_samples 2025-01-21 18:30:10 -07:00
Jaret Burkett
29122b1a54 Added code to handle diffusion feature extraction loss 2025-01-21 14:21:34 -07:00
Jaret Burkett
fadb2f3a76 Allow quantizing the te independently on flux. added lognorm_blend timestep schedule 2025-01-18 18:02:31 -07:00
Jaret Burkett
4723f23c0d Added ability to split up flux across gpus (experimental). Changed the way timestep scheduling works to prep for more specific schedules. 2024-12-31 07:06:55 -07:00
Jaret Burkett
8ef07a9c36 Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications 2024-12-15 08:59:27 -07:00
Jaret Burkett
92ce93140e Adjustments to defaults for automagic 2024-11-29 10:28:06 -07:00
Jaret Burkett
f213996aa5 Fixed saving and displaying for automagic 2024-11-29 08:00:22 -07:00
Jaret Burkett
cbe31eaf0a Initial work on a auto adjusting optimizer 2024-11-29 04:48:58 -07:00
Jaret Burkett
67c2e44edb Added support for training flux redux adapters 2024-11-21 20:01:52 -07:00
Jaret Burkett
96d418bb95 Added support for full finetuning flux with randomized param activation. Examples coming soon 2024-11-21 13:05:32 -07:00
Jaret Burkett
894374b2e9 Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use) 2024-11-20 09:16:55 -07:00
Jaret Burkett
6509ba4484 Fix seed generation to make it deterministic so it is consistant from gpu to gpu 2024-11-15 12:11:13 -07:00
Jaret Burkett
025ee3dd3d Added ability for adafactor to fully fine tune quantized model. 2024-10-30 16:38:07 -06:00
Jaret Burkett
58f9d01c2b Added adafactor implementation that handles stochastic rounding of update and accumulation 2024-10-30 05:25:57 -06:00
Jaret Burkett
e72b59a8e9 Added experimental 8bit version of prodigy with stochastic rounding and stochastic gradient accumulation. Still testing. 2024-10-29 14:28:28 -06:00
Jaret Burkett
4aa19b5c1d Only quantize flux T5 is also quantizing model. Load TE from original name and path if fine tuning. 2024-10-29 14:25:31 -06:00
Jaret Burkett
4747716867 Fixed issue with adapters not providing gradients with new grad activator 2024-10-29 14:22:10 -06:00
Jaret Burkett
22cd40d7b9 Improvements for full tuning flux. Added debugging launch config for vscode 2024-10-29 04:54:08 -06:00
Jaret Burkett
3400882a80 Added preliminary support for SD3.5-large lora training 2024-10-22 12:21:36 -06:00
Jaret Burkett
9f94c7b61e Added experimental param multiplier to the ema module 2024-10-22 09:25:52 -06:00
Jaret Burkett
bedb8197a2 Fixed issue with sizes for some images being loaded sideways resulting in squished images. 2024-10-20 11:51:29 -06:00
Jaret Burkett
e3ebd73610 Add a projection layer on vision direct when doing image embeds 2024-10-20 10:48:23 -06:00
Jaret Burkett
0640cdf569 Handle errors in loading size database 2024-10-20 07:04:19 -06:00
Jaret Burkett
ce759ebd8c Normalize the image embeddings on vd adapter forward 2024-10-12 15:09:48 +00:00
Jaret Burkett
628a7923a3 Remove norm on image embeds on custom adapter 2024-10-12 00:43:18 +00:00
Jaret Burkett
3922981996 Added some additional experimental things to the vision direct encoder 2024-10-10 19:42:26 +00:00
Jaret Burkett
ab22674980 Allow for a default caption file in the folder. Minor bug fixes. 2024-10-10 07:31:33 -06:00
Jaret Burkett
9452929300 Apply a mask to the embeds for SD if using T5 encoder 2024-10-04 10:55:20 -06:00
Jaret Burkett
a800c9d19e Add a method to have an inference only lora 2024-10-04 10:06:53 -06:00
Jaret Burkett
28e6f00790 Fixed bug in returning clip image embed to actually return it 2024-10-03 10:49:09 -06:00
Jaret Burkett
67e0aca750 Added ability to load clip pairs randomly from folder. Other small bug fixes 2024-10-03 10:03:49 -06:00
Jaret Burkett
f05224970f Added Vision Languate Adapter usage for pixtral vd adapter 2024-09-29 19:39:56 -06:00
Jaret Burkett
b4f64de4c2 Quick patch to scope xformer imports until a better solution 2024-09-28 15:36:42 -06:00
Jaret Burkett
e4c82803e1 Handle random resizing for pixtral input on direct vision adapter 2024-09-28 14:53:38 -06:00
Jaret Burkett
69aa92bce5 Added support for AdEMAMix8bit 2024-09-28 14:33:51 -06:00
Jaret Burkett
a508caad1d Change pixtral to crop based on number of pixels instead of largest dimension 2024-09-28 13:05:26 -06:00
Jaret Burkett
58537fc92b Added initial direct vision pixtral support 2024-09-28 10:47:51 -06:00
Jaret Burkett
86b5938cf3 Fixed the webp bug finally. 2024-09-25 13:56:00 -06:00
Jaret Burkett
6b4034122f REmove layers from direct vision resampler 2024-09-24 15:08:29 -06:00
Jaret Burkett
10817696fb Fixed issue where direct vision was not passing additional params from resampler when it is added 2024-09-24 10:34:11 -06:00
Jaret Burkett
037ce11740 Always return vision encoder in state dict 2024-09-24 07:43:17 -06:00
Jaret Burkett
04424fe2d6 Added config setting to set the timestep type 2024-09-24 06:53:59 -06:00
Jaret Burkett
40a8ff5731 Load local hugging face packages for assistant adapter 2024-09-23 10:37:12 -06:00
Jaret Burkett
2776221497 Added option to cache empty prompt or trigger and unload text encoders while training 2024-09-21 20:54:09 -06:00
Jaret Burkett
f85ad452c6 Added initial support for pixtral vision as a vision encoder. 2024-09-21 15:21:14 -06:00
Plat
79b4e04b80 Feat: Wandb logging (#95) 
* wandb logging

* fix: start logging before train loop

* chore: add wandb dir to gitignore

* fix: wrap wandb functions

* fix: forget to send last samples

* chore: use valid type

* chore: use None when not type-checking

* chore: resolved complicated logic

* fix: follow log_every

---------

Co-authored-by: Plat <github@p1at.dev>
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
 2024-09-19 20:01:01 -06:00