ai-toolkit

ostris/ai-toolkit

Fork 0

mirror of https://github.com/ostris/ai-toolkit.git synced 2026-02-02 11:59:46 +00:00

Commit Graph

Select branches

Hide Pull Requests

WIP

dev

development

kohya-sdxl

lumina2

main

multi-gpu

qwen_image_edit

sdxl

wavelet_loss

#107

#109

#115

#12

#120

#124

#125

#128

#128

#138

#138

#141

#143

#155

#155

#156

#156

#158

#158

#160

#160

#168

#168

#173

#176

#177

#179

#179

#183

#184

#184

#185

#185

#191

#191

#196

#224

#224

#226

#226

#232

#233

#233

#244

#244

#245

#245

#246

#246

#249

#25

#25

#251

#251

#253

#253

#256

#263

#263

#264

#265

#265

#271

#273

#273

#274

#275

#275

#278

#283

#283

#290

#290

#300

#300

#302

#302

#303

#303

#304

#307

#318

#318

#326

#326

#331

#331

#333

#333

#341

#341

#342

#342

#343

#350

#350

#353

#353

#354

#354

#359

#362

#362

#373

#373

#375

#375

#377

#381

#381

#382

#382

#383

#385

#385

#392

#392

#395

#407

#407

#41

#417

#418

#420

#420

#422

#426

#427

#427

#430

#430

#431

#431

#432

#432

#434

#442

#442

#446

#447

#452

#455

#459

#459

#461

#461

#464

#465

#465

#467

#469

#469

#473

#473

#474

#488

#488

#489

#489

#492

#492

#494

#494

#496

#498

#498

#499

#499

#500

#509

#511

#513

#519

#519

#520

#520

#521

#522

#523

#525

#527

#527

#540

#547

#547

#551

#551

#555

#558

#559

#559

#563

#563

#564

#564

#567

#578

#578

#580

#580

#581

#581

#583

#594

#599

#60

#600

#600

#601

#601

#606

#606

#608

#608

#61

#610

#611

#614

#614

#616

#616

#629

#629

#630

#630

#633

#633

#637

#637

#639

#639

#644

#646

#648

#649

#649

#655

#655

#658

#658

#660

#661

#661

#666

#666

#668

#668

#673

#673

#682

#685

#685

#687

#687

#69

#69

#692

#692

#694

#694

#695

#695

#7

#80

#86

#86

#9

#95

86c70a2a1f Added an experimental clip fusion model that is showing promise for embedding concepts Jaret Burkett 2024-01-17 13:13:04 -07:00
655533d4c7 More work on custom adapter Jaret Burkett 2024-01-16 17:41:26 -07:00
eebd3c8212 Initial training script for photomaker training. Needs a little more work. Jaret Burkett 2024-01-15 18:46:26 -07:00
5276975fb0 Added additional config options for custom plugins I needed Jaret Burkett 2024-01-15 08:31:09 -07:00
e190fbaeb8 Prepwork for ilora Jaret Burkett 2024-01-12 06:41:15 -07:00
290393f7ae Imporvements to ip weight adaptation. Bug fixes. Added masking to direct guidance loss. Allow importing a file for random triggers. Handle bas meta images with improper sizing. Jaret Burkett 2024-01-11 12:22:16 -07:00
b2a54c8f36 Added siglip support Jaret Burkett 2024-01-09 20:52:21 -07:00
b767d29b3c Adjustments to the clip preprocessor. Allow merging in new weights for ip adapters so you can change the arcitecture while maintaining as much data as possible Jaret Burkett 2024-01-06 11:56:53 -07:00
645b27f97a Bug fixes with ip adapter training. Made a clip pre processor that can be trained with ip adapter to help augment the clip input to squeeze in more detail from a larget input. moved clip processing to the dataloader for speed. Jaret Burkett 2024-01-04 12:59:38 -07:00
65c08b09c3 Added ability to do cfg during training. Various bug fixes Jaret Burkett 2024-01-02 11:29:57 -07:00
afc231efc1 Added reference adapters, many bug fixes, more ip adapter work and customizability Jaret Burkett 2024-01-01 17:15:53 -07:00
bafacf3b65 Initial commit Jaret Burkett 2023-12-29 13:07:35 -07:00
0892dec4a5 Fixed some new bugs i added. woops Jaret Burkett 2023-12-28 14:03:42 -07:00
eeee4a1620 Created a size agnostic feature encoder (SAFE) model to be trained in replace of CLIP for ip adapters. It is mostly conv layers so will hopefully be able to handle facial features better than clip can. Also bug fixes Jaret Burkett 2023-12-28 12:20:27 -07:00
d11ed7f66c Big fixes and added method to standardize values in both latent and pixel space before feeding into the network. Target values were determined over huge generated regularization sets. Jaret Burkett 2023-12-26 06:19:48 -07:00
27ad79053e Added SDXL support for clip vision embedder trainer Jaret Burkett 2023-12-24 14:31:29 -07:00
05ae95ca89 Added a clip vision adapter trainer. Only works for sd15 for now Jaret Burkett 2023-12-24 13:26:04 -07:00
0f8daa5612 Bug fixes, work on maing IP adapters more customizable. Jaret Burkett 2023-12-24 08:32:39 -07:00
7703e3a15e Fixes for sdxl ip adapter training. Bug fixes Jaret Burkett 2023-12-21 11:15:58 -07:00
0f597f453e Switched ip adapter dataloader to clip_image paths so the control paths can be used for training assistant adapters while training ip adapters Jaret Burkett 2023-12-20 10:32:24 -07:00
dfb64b5957 Allow ip adapters to be much more variable in their creation Jaret Burkett 2023-12-20 06:18:33 -07:00
82098e5d6e Added more functionality for ip adapters Jaret Burkett 2023-12-19 09:54:56 -07:00
b653906715 Fixed ip adapter training. Works now Jaret Burkett 2023-12-17 08:22:59 -07:00
13d32423f6 Added a polarity balancer to guidance Jaret Burkett 2023-12-15 15:19:14 -07:00
39870411d8 More guidance work. Improved LoRA module resolver for unet. Added vega mappings and LoRA training for it. Various other bigfixes and changes Jaret Burkett 2023-12-15 06:02:10 -07:00
e5177833b2 Targeted guidance work Jaret Burkett 2023-12-09 19:06:18 -07:00
eaa0fb6253 Tons of bug fixes and improvements to special training. Fixed slider training. Jaret Burkett 2023-12-09 16:38:10 -07:00
eaec2f5a52 Added guidance mentiods. WIP Jaret Burkett 2023-12-08 08:39:21 -07:00
92cb5ae096 Reworked targeted guidance algo Jaret Burkett 2023-12-01 06:30:52 -07:00
bd2bce9b92 Switched to trailing timestep spacing to make timesteps for consistant across schedulers. Honed in on targeted guidance. It is finally perfect. (I think) Jaret Burkett 2023-11-29 14:32:48 -07:00
537af79b0d Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2023-11-29 10:13:55 -07:00
7624241032 More fixes for noise schedules and fixed targeted guidance inverted masked prior Jaret Burkett 2023-11-29 10:13:31 -07:00
0d5943af91 Update readme for torch requirements Jaret Burkett 2023-11-28 12:52:55 -07:00
be815f9c47 updated requirements Jaret Burkett 2023-11-28 10:43:17 -07:00
3443d6aafa Updated Readme Jaret Burkett 2023-11-28 10:41:16 -07:00
bef10a639c Merge remote-tracking branch 'origin/development' Jaret Burkett 2023-11-28 10:40:05 -07:00
c2a4b8e058 More timestep fixes development Jaret Burkett 2023-11-28 10:34:08 -07:00
792a5e37e2 Numerous fixes for time sampling. Still not perfect Jaret Burkett 2023-11-28 07:34:43 -07:00
d7e55b6ad4 Bug fixes, negative prompting during training, hardened catching Jaret Burkett 2023-11-24 07:25:11 -07:00
fbec68681d Added timestep modifications to lcm scheduler for more evenly spaced timesteps Jaret Burkett 2023-11-17 23:26:52 -07:00
6280284d8b Fixed cleanup of emebddings. Jaret Burkett 2023-11-16 20:26:11 -07:00
ad50921c41 Sampling tests and added fixes for cleanups Jaret Burkett 2023-11-16 08:33:23 -07:00
e47006ed70 Added some features for an LCM condenser plugin Jaret Burkett 2023-11-15 08:56:45 -07:00
4f9cdd916a added prompt dropout to happen indempendently on each TE Jaret Burkett 2023-11-14 05:26:51 -07:00
7782caa468 Varous bug fixes. Finalized targeted guidance algo Jaret Burkett 2023-11-10 12:18:08 -07:00
fa6d91ba76 Diffirential guidance working, but I may have a better way Jaret Burkett 2023-11-09 06:09:36 -07:00
1ee62562a4 diffirential guidance is WORKING (from what I can tell) Jaret Burkett 2023-11-07 19:24:12 -07:00
dc8448d958 Added a way pass refiner ratio to sample config Jaret Burkett 2023-11-06 09:22:58 -07:00
a8b3b8b8da Fixed weight mapping for refiner Jaret Burkett 2023-11-06 07:37:47 -07:00
93ea955d7c Added refiner fine tuning. Works, but needs some polish. Jaret Burkett 2023-11-05 17:15:03 -07:00
8a9e8f708f Added base for using guidance during training. Still not working right. Jaret Burkett 2023-11-05 04:03:32 -07:00
d35733ac06 Added support for training ssd-1B. Added support for saving models into diffusers format. We can currently save in safetensors format for ssd-1b, but diffusers cannot load it yet. Jaret Burkett 2023-11-03 05:01:16 -06:00
ceaf1d9454 Various bug fixes, wip stuff, and tweaks Jaret Burkett 2023-11-02 18:19:20 -06:00
7d707b2fe6 Added masking to slider training. Something is still weird though Jaret Burkett 2023-11-01 14:51:29 -06:00
a899ec91c8 Added some split prompting started code, adamw8bit, replacements improving, learnable snr gos. A lot of good stuff. Jaret Burkett 2023-11-01 06:52:21 -06:00
436a09430e Added flat snr gamma vs min. Fixes timestep timing Jaret Burkett 2023-10-29 15:41:55 -06:00
3097865203 Fix llava import.. again Jaret Burkett 2023-10-29 12:52:17 -06:00
b84e3260cb Fix llava import Jaret Burkett 2023-10-29 12:50:54 -06:00
48a9bac22d Added doffusers schedulers Jaret Burkett 2023-10-29 12:39:50 -06:00
298001439a Added gradient accumulation finally Jaret Burkett 2023-10-28 13:14:29 -06:00
6f3e0d5af2 Improved lorm extraction and training Jaret Burkett 2023-10-28 08:21:59 -06:00
0a79ac9604 Added lorm. WIP Jaret Burkett 2023-10-26 18:23:51 -06:00
9636194c09 Added fuyu captioning Jaret Burkett 2023-10-25 14:14:53 -06:00
d742792ee4 Fixed issue with not loading short prompt Jaret Burkett 2023-10-24 16:19:32 -06:00
002279cec3 Allow short and long caption combinations like form the new captioning system. Merge the network into the model before inference and reextract when done. Doubles inference speed on locon models during inference. allow splitting a batch into individual components and run them through alone. Basicallt gradient accumulation with single batch size. Jaret Burkett 2023-10-24 16:02:07 -06:00
73c8b50975 Added ability to use adagrad from transformers Jaret Burkett 2023-10-24 11:16:01 -06:00
34eb563d55 Added dataset tagging and management tools using llava Jaret Burkett 2023-10-24 10:11:47 -06:00
dc36bbb3c8 Added long prompts to general training Jaret Burkett 2023-10-23 08:12:58 -06:00
9905a1e205 Fixes and longer prompts Jaret Burkett 2023-10-22 08:57:37 -06:00
0e9fc42816 Allow for inverted masked prior Jaret Burkett 2023-10-21 06:50:17 -06:00
d46112a354 Fixes for aug pipeline Jaret Burkett 2023-10-20 13:24:07 -06:00
07bf7bd7de Allow augmentations and targeting different loss types fron the config file Jaret Burkett 2023-10-18 03:04:57 -06:00
da6302ada8 added a method to apply multipliers to noise and latents prior to combining Jaret Burkett 2023-10-17 06:09:16 -06:00
a05459afaf Fixed issue with adapters that only had 1 input channel. Added ability to set the percentage chance of adapter matching Jaret Burkett 2023-10-15 15:13:35 -06:00
b1a22d0b3e hardened reading prompts from json Jaret Burkett 2023-10-15 07:20:33 -06:00
7909b50d24 Added adapter assistance to SD training Jaret Burkett 2023-10-14 08:44:53 -06:00
38e441a29c allow flipping for point of interesting autocropping. allow num repeats. Fixed some bugs with new free u Jaret Burkett 2023-10-12 21:02:47 -06:00
4e3b2c2569 Allow for alpha to be used as a mask Jaret Burkett 2023-10-10 17:08:59 -06:00
239addba51 Fixed memory leak when cachine latents to disk Jaret Burkett 2023-10-10 13:31:47 -06:00
63ceffae24 Massive speed increases and ram optimizations Jaret Burkett 2023-10-10 06:07:55 -06:00
f4c90bb589 Added config to set the min value of a mask Jaret Burkett 2023-10-09 15:47:54 -06:00
bb1d3793e3 Added ability to add masks to dataloader and sd trainer to adjust weight of image Jaret Burkett 2023-10-09 11:21:00 -06:00
1d3de678aa fixed bug with trigger word embedding. Allow control images to load from the dataloader or legacy way Jaret Burkett 2023-10-09 06:21:49 -06:00
b1cfafa0c6 always prompt both even when training only one encoder Jaret Burkett 2023-10-07 11:23:09 -06:00
cac8754399 Allow for training loras on onle one text encoder for sdxl Jaret Burkett 2023-10-06 08:11:56 -06:00
f73402473b Bug fixes. Added some functionality to help with private extensions Jaret Burkett 2023-10-05 07:09:34 -06:00
579650eaf8 Fixed big issue with bucketing dataloader and added random cripping to a point of interest Jaret Burkett 2023-10-02 18:31:08 -06:00
320e109c5f Allow loading from a json detail file for captions Jaret Burkett 2023-10-02 13:25:09 -06:00
560251a24f fixed issue with down block residuals when doing slider cfg on sdxl with t2i adapter assisted training Jaret Burkett 2023-10-01 07:32:48 -06:00
085787b799 Allow loading auxillery images from dataloader Jaret Burkett 2023-09-30 07:28:23 -06:00
8d9450ad7c Compatability fixes Jaret Burkett 2023-09-29 14:07:37 -06:00
8509da60cb Added a way to add a t2i adapter guided slider training for more consitant images Jaret Burkett 2023-09-28 14:08:56 -06:00
c5d49ba661 Allow adapter image to be cropped to match bucket cropping Jaret Burkett 2023-09-25 10:38:54 -06:00
76c764af49 Fixed issues with adapter and disabled telementry for now Jaret Burkett 2023-09-24 12:51:29 -06:00
abf7cd221d allow setting adapter weight in prompts Jaret Burkett 2023-09-24 06:51:54 -06:00
e5153d87c9 Fixed issues with dataloader bucketing. Allow using standard base image for t2i adapters. Jaret Burkett 2023-09-24 05:19:57 -06:00
830e87cb87 Added IP adapter training. Not functioning correctly yet Jaret Burkett 2023-09-24 02:39:43 -06:00
19255cdc7c Bugfixes. Added small augmentations to dataloader. Will switch to abluminations soon though. Added ability to adjust step count on start to override what is in the file Jaret Burkett 2023-09-20 05:30:10 -06:00
0f105690cc Added some further extendability for plugins Jaret Burkett 2023-09-19 05:41:44 -06:00
61badf85a7 t2i training working from what I can tell at least Jaret Burkett 2023-09-17 15:56:43 -06:00