ostris/ai-toolkit
1
0
Fork 0
mirror of https://github.com/ostris/ai-toolkit.git synced 2026-02-07 05:59:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
431 Commits 10 Branches 0 Tags
10817696fb3e06fa6eca6d7b2dbbe24d2b2ca912
Commit Graph

431 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jaret Burkett
10817696fb Fixed issue where direct vision was not passing additional params from resampler when it is added 2024-09-24 10:34:11 -06:00
Jaret Burkett
037ce11740 Always return vision encoder in state dict 2024-09-24 07:43:17 -06:00
Jaret Burkett
04424fe2d6 Added config setting to set the timestep type 2024-09-24 06:53:59 -06:00
Jaret Burkett
40a8ff5731 Load local hugging face packages for assistant adapter 2024-09-23 10:37:12 -06:00
Jaret Burkett
2776221497 Added option to cache empty prompt or trigger and unload text encoders while training 2024-09-21 20:54:09 -06:00
Jaret Burkett
f85ad452c6 Added initial support for pixtral vision as a vision encoder. 2024-09-21 15:21:14 -06:00
Jaret Burkett
dd889086f4 Updates to the docker file for jupyterlab 2024-09-21 12:07:07 -06:00
apolinário
bc693488eb fix diffusers codebase (#183) 2024-09-21 11:50:29 -06:00
Jaret Burkett
d97c55cd96 Updated requirements to lock version of albucore, which had breaking changes. 2024-09-21 11:19:13 -06:00
Plat
79b4e04b80 Feat: Wandb logging (#95) 
* wandb logging

* fix: start logging before train loop

* chore: add wandb dir to gitignore

* fix: wrap wandb functions

* fix: forget to send last samples

* chore: use valid type

* chore: use None when not type-checking

* chore: resolved complicated logic

* fix: follow log_every

---------

Co-authored-by: Plat <github@p1at.dev>
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
 2024-09-19 20:01:01 -06:00
Jaret Burkett
951e223481 Added support to disable single transformers in vision direct adapter 2024-09-11 08:54:51 -06:00
Jaret Burkett
fc34a69bec Ignore guidance embed when full tuning flux. adjust block scaler to decat to 1.0. Add MLP resampler for reducing vision adapter tokens 2024-09-09 16:24:46 -06:00
Jaret Burkett
279ee65177 Remove block scaler 2024-09-06 08:28:17 -06:00
Jaret Burkett
3a1f464132 Added support for training vision direct weight adapters 2024-09-05 10:11:44 -06:00
Jaret Burkett
5c8fcc8a4e Fix bug with zeroing out gradients when accumulating 2024-09-03 08:29:15 -06:00
Jaret Burkett
121a760c19 Added proper grad accumulation 2024-09-03 07:24:18 -06:00
Jaret Burkett
e5fadddd45 Added ability to do prompt attn masking for flux 2024-09-02 17:29:36 -06:00
Jaret Burkett
d44d4eb61a Added a new experimental linear weighing technique 2024-09-02 09:22:13 -06:00
Jaret Burkett
7d9ab22405 Rework ip adapter and vision direct adapters to apply to the single transformer blocks even though they are not cross attn. 2024-09-01 10:40:42 -06:00
Jaret Burkett
7ed8c51f20 Readme cleanup 2024-09-01 07:06:09 -06:00
Jaret Burkett
6df33156f0 Add information about specific weight targeting in the README 2024-09-01 06:59:47 -06:00
Jaret Burkett
40f5c59da0 Fixes for training ilora on flux 2024-08-31 16:55:26 -06:00
Jaret Burkett
3e71a99df0 Check for contains only against clean name for lora, not the adjusted one 2024-08-31 07:44:13 -06:00
apolinário
562405923f Update README.md for push_to_hub (#143) 
Add diffusers examples and clarify how to use the model locally
 2024-08-30 16:34:28 -06:00
apolinário
f84bd6d7a6 Add Gradio UI for ai-toolkit (#141) 
* Add Gradio UI for FLUX.1

* small text changes

* no flash-attn? no problem!

* bye flash-attn!

* fixes for windows

---------

Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com>
 2024-08-30 06:29:51 -06:00
Jaret Burkett
60232def91 Made peleminary arch for flux ip adapter training 2024-08-28 08:55:39 -06:00
Jaret Burkett
3843e0d148 Added support for vision direct adapter for flux 2024-08-26 16:27:28 -06:00
liaoliaojun
e127c079da fix: print out the path where the image encode failed (#107) 2024-08-22 21:34:35 -06:00
martintomov
34db804c76 Modal cloud training support, fixed typo in toolkit/scheduler.py, Schnell training support for Colab, issue #92 , issue #114 (#115) 
* issue #76, load_checkpoint_and_dispatch() 'force_hooks'

https://github.com/ostris/ai-toolkit/issues/76

* RunPod cloud config

https://github.com/ostris/ai-toolkit/issues/90

* change 2x A40 to 1x A40 and price per hour

referring to https://github.com/ostris/ai-toolkit/issues/90#issuecomment-2294894929

* include missed FLUX.1-schnell setup guide in last commit

* huggingface-cli login required auth

* #92 peft, #114 colab, schnell training in colab

* modal cloud - run_modal.py and .yaml configs

* run_modal.py mount path example

* modal_examples renamed to modal

* Training in Modal README.md setup guide

* rename run command in title for consistency
 2024-08-22 21:25:44 -06:00
apolinário
4d35a29c97 Add push_to_hub to the trainer (#109) 
* add push_to_hub

* fix indentation

* indent again

* model_config

* allow samples to not exist

* repo creation fix

* dont show empty [] if widget doesnt exist

* dont submit the config and optimizer

* Unsafe to have tokens saved in the yaml file

* make sure to catch only the latest samples

* change name to slug

* formatting

* formatting

---------

Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com>
 2024-08-22 21:18:56 -06:00
Jaret Burkett
b322d05fa3 Added tutorial link to readme 2024-08-22 16:25:32 -06:00
Jaret Burkett
8577849eeb Fixed wrong discord link. Woops. 2024-08-22 14:49:03 -06:00
Jaret Burkett
338c77d677 Fixed breaking change with diffusers. Allow flowmatch on normal stable diffusion models. 2024-08-22 14:36:22 -06:00
Jaret Burkett
e07a98a50c Bugfixes for full finetuning at bf16 2024-08-22 05:15:33 -06:00
Jaret Burkett
6a754b2710 Merge branch 'main' of github.com:ostris/ai-toolkit 2024-08-22 04:36:50 -06:00
Jaret Burkett
a939cf3730 WIP - adding support for flux DoRA and ip adapter training 2024-08-22 04:36:39 -06:00
Jaret Burkett
169dbd22ba Finaized bug reports 2024-08-18 16:21:48 -06:00
Jaret Burkett
6e7d721382 More issues testing 2024-08-18 16:20:08 -06:00
Jaret Burkett
dc6f36cd82 Testing github bug reporting stuff 2024-08-18 16:09:52 -06:00
martintomov
5603f9e004 issue #76, and RunPod cloud training setup #90 (#80) 
* issue #76, load_checkpoint_and_dispatch() 'force_hooks'

https://github.com/ostris/ai-toolkit/issues/76

* RunPod cloud config

https://github.com/ostris/ai-toolkit/issues/90

* change 2x A40 to 1x A40 and price per hour

referring to https://github.com/ostris/ai-toolkit/issues/90#issuecomment-2294894929

* include missed FLUX.1-schnell setup guide in last commit

* huggingface-cli login required auth
 2024-08-18 15:43:45 -06:00
Jaret Burkett
c45887192a Unload interum weights when doing multi lora fuse 2024-08-18 09:35:10 -06:00
Jaret Burkett
13a965a26c Fixed bad key naming on lora fuse I just pushed 2024-08-18 09:33:31 -06:00
Jaret Burkett
77ee7090e8 Update FAQ.md 2024-08-18 09:26:22 -06:00
Jaret Burkett
078396ceac Added a basic FAQ 2024-08-18 09:21:51 -06:00
Jaret Burkett
f944eeaa4d Fuse flux schnell assistant adapter in pieces when doing lowvram to drastically speed ip up from minutes to seconds. 2024-08-18 09:09:11 -06:00
Jaret Burkett
81899310f8 Added support for training on flux schnell. Added example config and instructions for training on flux schnell 2024-08-17 06:58:39 -06:00
Jaret Burkett
f9179540d2 Flush after sampling 2024-08-16 17:29:42 -06:00
Jaret Burkett
452e0e286d For lora assisted training, merge in before quantizing then sample with schnell at -1 weight. Almost doubles training speed with lora adapter. 2024-08-16 17:28:44 -06:00
Jaret Burkett
165510ace2 Dumb typo 2024-08-15 12:59:32 -06:00
Jaret Burkett
0355662e8e Added support for polarity guidance for flow matching models 2024-08-15 12:22:00 -06:00