ostris/ai-toolkit
1
0
Fork 0
mirror of https://github.com/ostris/ai-toolkit.git synced 2026-01-31 10:59:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
457 Commits 10 Branches 0 Tags
22cd40d7b95ff9b3a01be669c0da88960d045017
Commit Graph

65 Commits

Author SHA1 Message Date
Jaret Burkett
e3ebd73610 Add a projection layer on vision direct when doing image embeds 2024-10-20 10:48:23 -06:00
Jaret Burkett
ce759ebd8c Normalize the image embeddings on vd adapter forward 2024-10-12 15:09:48 +00:00
Jaret Burkett
3922981996 Added some additional experimental things to the vision direct encoder 2024-10-10 19:42:26 +00:00
Jaret Burkett
67e0aca750 Added ability to load clip pairs randomly from folder. Other small bug fixes 2024-10-03 10:03:49 -06:00
Jaret Burkett
f05224970f Added Vision Languate Adapter usage for pixtral vd adapter 2024-09-29 19:39:56 -06:00
Jaret Burkett
b4f64de4c2 Quick patch to scope xformer imports until a better solution 2024-09-28 15:36:42 -06:00
Jaret Burkett
e4c82803e1 Handle random resizing for pixtral input on direct vision adapter 2024-09-28 14:53:38 -06:00
Jaret Burkett
a508caad1d Change pixtral to crop based on number of pixels instead of largest dimension 2024-09-28 13:05:26 -06:00
Jaret Burkett
58537fc92b Added initial direct vision pixtral support 2024-09-28 10:47:51 -06:00
Jaret Burkett
6b4034122f REmove layers from direct vision resampler 2024-09-24 15:08:29 -06:00
Jaret Burkett
f85ad452c6 Added initial support for pixtral vision as a vision encoder. 2024-09-21 15:21:14 -06:00
Jaret Burkett
951e223481 Added support to disable single transformers in vision direct adapter 2024-09-11 08:54:51 -06:00
Jaret Burkett
fc34a69bec Ignore guidance embed when full tuning flux. adjust block scaler to decat to 1.0. Add MLP resampler for reducing vision adapter tokens 2024-09-09 16:24:46 -06:00
Jaret Burkett
279ee65177 Remove block scaler 2024-09-06 08:28:17 -06:00
Jaret Burkett
3a1f464132 Added support for training vision direct weight adapters 2024-09-05 10:11:44 -06:00
Jaret Burkett
7d9ab22405 Rework ip adapter and vision direct adapters to apply to the single transformer blocks even though they are not cross attn. 2024-09-01 10:40:42 -06:00
Jaret Burkett
40f5c59da0 Fixes for training ilora on flux 2024-08-31 16:55:26 -06:00
Jaret Burkett
3843e0d148 Added support for vision direct adapter for flux 2024-08-26 16:27:28 -06:00
Jaret Burkett
a939cf3730 WIP - adding support for flux DoRA and ip adapter training 2024-08-22 04:36:39 -06:00
Jaret Burkett
6e92922c14 Add a mergable linear to the mid of ilora 2024-07-20 21:17:53 -06:00
Jaret Burkett
c51235c486 Fixed misnamed var 2024-07-20 23:00:20 +00:00
Jaret Burkett
c2c4e8cf34 Added ability to target parts of lora for ilora 2024-07-20 22:45:52 +00:00
Jaret Burkett
c2d5f712a3 Reworked ilora arch 2024-07-20 15:35:59 -06:00
Jaret Burkett
a2301cf28c Amall bug fixes 2024-07-18 10:39:55 -06:00
Jaret Burkett
11e426fdf1 Various features and fixes. Too much brain fog to do a proper description 2024-07-18 07:34:14 -06:00
Jaret Burkett
58dffd43a8 Added caching to image sizes so we dont do it every time. 2024-07-15 19:07:41 -06:00
Jaret Burkett
e4558dff4b Partial implementation for training auraflow. 2024-07-12 12:11:38 -06:00
Jaret Burkett
c062b7716c Varous bug fixes 2024-07-10 15:20:04 -06:00
Jaret Burkett
76f225a467 Fixed issue with TE adapter caption projection 2024-07-06 19:09:58 +00:00
Jaret Burkett
cab8a1c7b8 WIP to add the caption_proj weight to pixart sigma TE adapter 2024-07-06 13:00:21 -06:00
Jaret Burkett
7165f2d25a Work to omprove pixart training 2024-06-23 20:46:48 +00:00
Jaret Burkett
5d47244c57 Added support for pixart sigma loras 2024-06-16 11:56:30 -06:00
Jaret Burkett
ada722c9e4 Fixed issue with heads not being added 2024-06-15 08:34:33 -06:00
Jaret Burkett
e3410413b9 Rework head on ilora 2024-06-14 16:21:26 -06:00
Jaret Burkett
37cebd9458 WIP Ilora 2024-06-14 09:31:01 -06:00
Jaret Burkett
bd10d2d668 Some work on sd3 training. Not working 2024-06-13 12:19:16 -06:00
Jaret Burkett
cb5d28cba9 Added working ilora trainer 2024-06-12 09:33:45 -06:00
Jaret Burkett
833c833f28 WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements 2024-05-27 10:50:24 -06:00
Jaret Burkett
68b7e159bc Bug Fixes 2024-05-17 08:41:20 -06:00
Jaret Burkett
5a45c709cd Work on ipadapters and custom adapters 2024-05-13 06:37:54 -06:00
Jaret Burkett
10e1ecf1e8 Added single value adapter training 2024-04-28 06:04:47 -06:00
Jaret Burkett
5da3613e0b Bug fixes and minor features 2024-04-25 06:14:31 -06:00
Jaret Burkett
5a70b7f38d Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded. 2024-04-20 10:46:56 -06:00
Jaret Burkett
7284aab7c0 Added specialized scaler training to ip adapters 2024-04-05 08:17:09 -06:00
Jaret Burkett
016687bda1 Adapter work. Bug fixes. Auto adjust LR when resuming optimizer. 2024-03-17 10:21:47 -06:00
Jaret Burkett
d87b49882c Work on embedding adapters 2024-03-11 15:18:42 -06:00
Jaret Burkett
b01e8d889a Added stochastic rounding to adafactor. ILora adjustments 2024-03-05 07:07:09 -07:00
Jaret Burkett
1325613583 rework ilora 2024-02-29 07:55:52 -07:00
Jaret Burkett
f965a1299f Fixed Dora implementation. Still highly experimental 2024-02-24 10:26:01 -07:00
Jaret Burkett
1bd94f0f01 Added early DoRA support, but will change shortly. Dont use right now. 2024-02-23 05:55:41 -07:00