ostris/ai-toolkit
1
0
Fork 0
mirror of https://github.com/ostris/ai-toolkit.git synced 2026-02-09 23:19:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
375 Commits 10 Branches 0 Tags
f7cf2f866fd4eb01f758f2d6a7aaa1e1738176fa
Commit Graph

311 Commits

Author SHA1 Message Date
Jaret Burkett
f7cf2f866f Make 100% sure lora alpha matches for flux 2024-08-13 14:24:03 -06:00
Jaret Burkett
418f5f7e8c Added new experimental time step weighing that should solve a lot of issues with distribution. Updated example. Removed a warning 2024-08-13 12:02:11 -06:00
Jaret Burkett
9ee1ef2a0a Added experimental modified sigma sqrt weight mapping for linear timestep scheduling for flowmatching 2024-08-12 17:03:09 -06:00
Jaret Burkett
599fafe01f Allow user to have the full flux checkpoint local 2024-08-12 09:57:16 -06:00
Jaret Burkett
af108bb964 Bug fix with dataloader. Added a flag to completly disable sampling 2024-08-12 09:19:40 -06:00
Jaret Burkett
a6aa4b2c7d Added ability to set timesteps to linear for flowmatching schedule 2024-08-11 13:06:08 -06:00
Jaret Burkett
7f0ecdb377 Merge branch 'main' of github.com:ostris/ai-toolkit 2024-08-11 11:10:45 -06:00
Jaret Burkett
fbed8568fb Actually use the correct timestep sampling instead of calculating it and moving on lol. Tested a few with it and it seems to work better. 2024-08-11 11:10:37 -06:00
Jaret Burkett
6d31c6db73 Added a fix for windows dataloader 2024-08-11 10:48:24 -06:00
Jaret Burkett
6490a326e5 Fixed issue for vaes without a shift 2024-08-11 10:30:55 -06:00
Jaret Burkett
ec1ea7aa0e Added support for training on primary gpu with low_vram flag. Updated example script to remove creepy horse sample at that seed 2024-08-11 09:54:30 -06:00
Jaret Burkett
2308ef2868 Added flux training instructions 2024-08-10 14:10:02 -06:00
Jaret Burkett
b3e03295ad Reworked flux pred. Again 2024-08-08 13:06:34 -06:00
Jaret Burkett
e69a520616 Reworked timestep distribution on flowmatch sampler when training. 2024-08-08 06:01:45 -06:00
Jaret Burkett
acafe9984f Adjustments to loading of flux. Added a feedback to ema 2024-08-07 13:17:26 -06:00
Jaret Burkett
653fe60f16 Updates to flow matching algo 2024-08-07 15:04:17 +00:00
Jaret Burkett
c2424087d6 8 bit training working on flux 2024-08-06 11:53:27 -06:00
Jaret Burkett
272c8608c2 Make a CFG version of flux pipeline 2024-08-05 16:35:53 -06:00
Jaret Burkett
187663ab55 Use peft format for flux loras so they are compatible with diffusers. allow loading an assistant lora 2024-08-05 14:34:37 -06:00
Jaret Burkett
edb7e827ee Adjusted flow matching so target noise multiplier works properly with it. 2024-08-05 11:40:05 -06:00
Jaret Burkett
0ea27011d5 Bug fix 2024-08-04 11:07:19 -06:00
Jaret Burkett
f321de7bdb Setup to retrain guidance embedding for flux. Use defualt timestep distribution for flux 2024-08-04 10:37:23 -06:00
Jaret Burkett
9beea1c268 Flux training should work now... maybe 2024-08-03 09:17:34 -06:00
Jaret Burkett
369aa143bc Only train a few blocks on flux (for now) 2024-08-03 07:02:27 -06:00
Jaret Burkett
87ba867fdc Added flux training. Still a WIP. Wont train right without rectified flow working right 2024-08-02 15:00:30 -06:00
Jaret Burkett
03613c523f Bugfixes and cleanup 2024-08-01 11:45:12 -06:00
Jaret Burkett
47744373f2 Change img multiplier math 2024-07-30 11:33:41 -06:00
Jaret Burkett
8f0f467c20 Switch back to old ilora 2024-07-29 07:22:05 -06:00
Jaret Burkett
e81e19fd0f Added target_norm_std which is a game changer 2024-07-28 16:08:33 -06:00
Jaret Burkett
0bc4d555c7 A lot of pixart sigma training tweaks 2024-07-28 11:23:18 -06:00
Jaret Burkett
80aa2dbb80 New image generation img2img. various tweaks and fixes 2024-07-24 04:13:41 -06:00
Jaret Burkett
6e92922c14 Add a mergable linear to the mid of ilora 2024-07-20 21:17:53 -06:00
Jaret Burkett
c51235c486 Fixed misnamed var 2024-07-20 23:00:20 +00:00
Jaret Burkett
c2c4e8cf34 Added ability to target parts of lora for ilora 2024-07-20 22:45:52 +00:00
Jaret Burkett
4c249cf607 Added ilora2 2024-07-20 16:40:57 -06:00
Jaret Burkett
c2d5f712a3 Reworked ilora arch 2024-07-20 15:35:59 -06:00
Jaret Burkett
22d2f6e28f Fixed issue with grad scaling 2024-07-20 08:21:57 -06:00
Jaret Burkett
a2301cf28c Amall bug fixes 2024-07-18 10:39:55 -06:00
Jaret Burkett
11e426fdf1 Various features and fixes. Too much brain fog to do a proper description 2024-07-18 07:34:14 -06:00
Jaret Burkett
58dffd43a8 Added caching to image sizes so we dont do it every time. 2024-07-15 19:07:41 -06:00
Jaret Burkett
e4558dff4b Partial implementation for training auraflow. 2024-07-12 12:11:38 -06:00
Jaret Burkett
c062b7716c Varous bug fixes 2024-07-10 15:20:04 -06:00
Jaret Burkett
93e5df1d59 Merge branch 'main' of github.com:ostris/ai-toolkit 2024-07-07 07:56:56 -06:00
Jaret Burkett
045e4a6e15 Save entire pixart model again 2024-07-07 07:56:48 -06:00
Jaret Burkett
76f225a467 Fixed issue with TE adapter caption projection 2024-07-06 19:09:58 +00:00
Jaret Burkett
cab8a1c7b8 WIP to add the caption_proj weight to pixart sigma TE adapter 2024-07-06 13:00:21 -06:00
Jaret Burkett
acb06d6ff3 Bug fixes 2024-07-03 10:56:34 -06:00
Jaret Burkett
bb57623a35 Merge branch 'main' of github.com:ostris/ai-toolkit 2024-06-29 15:53:40 -06:00
Jaret Burkett
3072d20f17 Add ability to include conv_in and conv_out to full train when doing a lora 2024-06-29 14:54:50 -06:00
Jaret Burkett
f6b21f47bb Increased the number of heads for ip adapters. 2024-06-28 16:09:52 +00:00