ai-toolkit

ostris/ai-toolkit

Fork 0

mirror of https://github.com/ostris/ai-toolkit.git synced 2026-05-11 08:20:35 +00:00

Commit Graph

Select branches

Hide Pull Requests

WIP

dev

development

hidream_o1

kohya-sdxl

lumina2

main

multi-gpu

qwen_image_edit

sdxl

wavelet_loss

#107

#109

#115

#12

#120

#124

#125

#128

#138

#141

#143

#155

#156

#158

#160

#160

#168

#173

#176

#177

#179

#183

#184

#185

#191

#196

#224

#226

#232

#233

#244

#245

#246

#249

#25

#251

#253

#253

#256

#263

#264

#265

#271

#273

#274

#275

#278

#283

#290

#300

#302

#302

#303

#304

#307

#318

#326

#326

#331

#333

#341

#342

#343

#350

#353

#354

#359

#362

#373

#375

#377

#381

#382

#383

#385

#392

#392

#395

#407

#41

#417

#418

#420

#422

#426

#427

#430

#431

#431

#432

#434

#442

#446

#447

#452

#455

#459

#461

#464

#465

#467

#469

#473

#474

#488

#489

#492

#494

#496

#498

#499

#500

#509

#511

#513

#519

#520

#521

#522

#523

#525

#527

#540

#547

#551

#555

#558

#559

#563

#563

#564

#564

#567

#578

#580

#581

#583

#594

#599

#60

#600

#601

#601

#606

#608

#61

#610

#611

#614

#616

#629

#630

#633

#637

#639

#644

#646

#648

#649

#655

#658

#660

#661

#666

#666

#668

#668

#673

#682

#685

#687

#69

#692

#694

#694

#695

#697

#698

#699

#7

#702

#703

#705

#706

#707

#708

#709

#709

#713

#713

#717

#718

#718

#719

#720

#721

#724

#724

#726

#727

#727

#732

#732

#733

#738

#742

#743

#743

#745

#748

#749

#754

#759

#760

#761

#762

#763

#765

#765

#766

#766

#767

#768

#769

#770

#774

#774

#777

#777

#778

#778

#782

#782

#783

#785

#793

#80

#800

#803

#803

#805

#805

#807

#807

#808

#808

#809

#811

#811

#812

#812

#813

#813

#814

#814

#819

#819

#825

#825

#827

#828

#828

#829

#829

#86

#9

#95

37cebd9458 WIP Ilora Jaret Burkett 2024-06-14 09:31:01 -06:00
bd10d2d668 Some work on sd3 training. Not working Jaret Burkett 2024-06-13 12:19:16 -06:00
cb5d28cba9 Added working ilora trainer Jaret Burkett 2024-06-12 09:33:45 -06:00
3f3636b788 Bug fixes and little improvements here and there. Jaret Burkett 2024-06-08 06:24:20 -06:00
833c833f28 WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements Jaret Burkett 2024-05-27 10:50:24 -06:00
68b7e159bc Bug Fixes Jaret Burkett 2024-05-17 08:41:20 -06:00
5a45c709cd Work on ipadapters and custom adapters Jaret Burkett 2024-05-13 06:37:54 -06:00
10e1ecf1e8 Added single value adapter training Jaret Burkett 2024-04-28 06:04:47 -06:00
b96913d73c Improvements to dataloader Jaret Burkett 2024-04-27 09:28:28 -06:00
5da3613e0b Bug fixes and minor features Jaret Burkett 2024-04-25 06:14:31 -06:00
5a70b7f38d Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded. Jaret Burkett 2024-04-20 10:46:56 -06:00
377b81ee3e Adjustments to guidance Jaret Burkett 2024-04-19 15:00:35 -06:00
2d0a1be59d Bug fixes Jaret Burkett 2024-04-16 03:48:13 -06:00
7284aab7c0 Added specialized scaler training to ip adapters Jaret Burkett 2024-04-05 08:17:09 -06:00
427847ac4c Small tweaks and fixes for specialized ip adapter training Jaret Burkett 2024-03-26 11:35:26 -06:00
9c1cc9641e Added keep tokens to keep so many tokens in a prompt when dropping Jaret Burkett 2024-03-18 13:18:25 -06:00
89f4bcad2e Lock diffusers to 0.26.3 until I can figure out why future versions break LoRA code Jaret Burkett 2024-03-18 10:17:55 -06:00
016687bda1 Adapter work. Bug fixes. Auto adjust LR when resuming optimizer. Jaret Burkett 2024-03-17 10:21:47 -06:00
72de68d8aa WIP on clip vision encoder Jaret Burkett 2024-03-13 07:24:08 -06:00
d87b49882c Work on embedding adapters Jaret Burkett 2024-03-11 15:18:42 -06:00
f415bac7b5 Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2024-03-06 09:32:38 -07:00
f1cb87fe9e fixed bug the kept learning rates the same Jaret Burkett 2024-03-06 09:23:32 -07:00
8f9cd823d1 Create LICENSE Jaret Burkett 2024-03-06 07:54:55 -07:00
b01e8d889a Added stochastic rounding to adafactor. ILora adjustments Jaret Burkett 2024-03-05 07:07:09 -07:00
1325613583 rework ilora Jaret Burkett 2024-02-29 07:55:52 -07:00
337945de9a Added this not that guidance. Added ability to replace prompts. Jaret Burkett 2024-02-28 20:10:14 -07:00
561914d8e6 Removed old code for fixing multistep sampler that is no longer needed Jaret Burkett 2024-02-25 11:53:35 -07:00
b0a0f28191 Bug fixes Jaret Burkett 2024-02-25 08:28:29 -07:00
f965a1299f Fixed Dora implementation. Still highly experimental Jaret Burkett 2024-02-24 10:26:01 -07:00
1bd94f0f01 Added early DoRA support, but will change shortly. Dont use right now. Jaret Burkett 2024-02-23 05:55:41 -07:00
9ffa8c3711 Fixed issue when there is no adapter Jaret Burkett 2024-02-22 02:59:59 -07:00
b68c3ef734 Added te aug adapter Jaret Burkett 2024-02-21 21:30:26 -07:00
49c41e6a5f Bug fixes. allow for random negative prompts Jaret Burkett 2024-02-21 04:51:52 -07:00
2478554c95 Bug fixes. Added IP adapter training for Pixart Jaret Burkett 2024-02-17 10:06:57 -07:00
93b52932c1 Added training for pixart-a Jaret Burkett 2024-02-13 16:00:04 -07:00
4ec4025cbb Added adapter modules for text encoders and direct vision Jaret Burkett 2024-02-12 08:46:18 -07:00
e074058faa Work on additional image embedding methods. Finalized zipper resampler. It works amazing Jaret Burkett 2024-02-10 09:00:05 -07:00
a8481c1670 randomly adjust scale of unconditional noise on ip adapters if training with cfg Jaret Burkett 2024-02-06 03:44:54 -07:00
e18e0cb5f8 Added comparitive loss when training clip encoder. Allow selecting clip layer. on ip adapter. Improvements to prior prediction Jaret Burkett 2024-02-05 07:40:03 -07:00
177c7130ec improved correction of pred norm by targeting the prior Jaret Burkett 2024-02-01 06:31:04 -07:00
1ae1017748 Bug fixes. added ability to use l1 loss. varous other tests and improvements Jaret Burkett 2024-01-31 06:30:54 -07:00
92b9c71d44 Many bug fixes. Ip adapter bug fixes. Added noise to unconditional, it works better. added an ilora adapter for 1 shotting LoRAs Jaret Burkett 2024-01-28 08:20:03 -07:00
f17ad8d794 various bug fixes. Created an contextual alpha mask module to calculate alpha mask Jaret Burkett 2024-01-18 16:34:27 -07:00
86c70a2a1f Added an experimental clip fusion model that is showing promise for embedding concepts Jaret Burkett 2024-01-17 13:13:04 -07:00
655533d4c7 More work on custom adapter Jaret Burkett 2024-01-16 17:41:26 -07:00
eebd3c8212 Initial training script for photomaker training. Needs a little more work. Jaret Burkett 2024-01-15 18:46:26 -07:00
5276975fb0 Added additional config options for custom plugins I needed Jaret Burkett 2024-01-15 08:31:09 -07:00
e190fbaeb8 Prepwork for ilora Jaret Burkett 2024-01-12 06:41:15 -07:00
290393f7ae Imporvements to ip weight adaptation. Bug fixes. Added masking to direct guidance loss. Allow importing a file for random triggers. Handle bas meta images with improper sizing. Jaret Burkett 2024-01-11 12:22:16 -07:00
b2a54c8f36 Added siglip support Jaret Burkett 2024-01-09 20:52:21 -07:00
b767d29b3c Adjustments to the clip preprocessor. Allow merging in new weights for ip adapters so you can change the arcitecture while maintaining as much data as possible Jaret Burkett 2024-01-06 11:56:53 -07:00
645b27f97a Bug fixes with ip adapter training. Made a clip pre processor that can be trained with ip adapter to help augment the clip input to squeeze in more detail from a larget input. moved clip processing to the dataloader for speed. Jaret Burkett 2024-01-04 12:59:38 -07:00
65c08b09c3 Added ability to do cfg during training. Various bug fixes Jaret Burkett 2024-01-02 11:29:57 -07:00
afc231efc1 Added reference adapters, many bug fixes, more ip adapter work and customizability Jaret Burkett 2024-01-01 17:15:53 -07:00
bafacf3b65 Initial commit Jaret Burkett 2023-12-29 13:07:35 -07:00
0892dec4a5 Fixed some new bugs i added. woops Jaret Burkett 2023-12-28 14:03:42 -07:00
eeee4a1620 Created a size agnostic feature encoder (SAFE) model to be trained in replace of CLIP for ip adapters. It is mostly conv layers so will hopefully be able to handle facial features better than clip can. Also bug fixes Jaret Burkett 2023-12-28 12:20:27 -07:00
d11ed7f66c Big fixes and added method to standardize values in both latent and pixel space before feeding into the network. Target values were determined over huge generated regularization sets. Jaret Burkett 2023-12-26 06:19:48 -07:00
27ad79053e Added SDXL support for clip vision embedder trainer Jaret Burkett 2023-12-24 14:31:29 -07:00
05ae95ca89 Added a clip vision adapter trainer. Only works for sd15 for now Jaret Burkett 2023-12-24 13:26:04 -07:00
0f8daa5612 Bug fixes, work on maing IP adapters more customizable. Jaret Burkett 2023-12-24 08:32:39 -07:00
7703e3a15e Fixes for sdxl ip adapter training. Bug fixes Jaret Burkett 2023-12-21 11:15:58 -07:00
0f597f453e Switched ip adapter dataloader to clip_image paths so the control paths can be used for training assistant adapters while training ip adapters Jaret Burkett 2023-12-20 10:32:24 -07:00
dfb64b5957 Allow ip adapters to be much more variable in their creation Jaret Burkett 2023-12-20 06:18:33 -07:00
82098e5d6e Added more functionality for ip adapters Jaret Burkett 2023-12-19 09:54:56 -07:00
b653906715 Fixed ip adapter training. Works now Jaret Burkett 2023-12-17 08:22:59 -07:00
13d32423f6 Added a polarity balancer to guidance Jaret Burkett 2023-12-15 15:19:14 -07:00
39870411d8 More guidance work. Improved LoRA module resolver for unet. Added vega mappings and LoRA training for it. Various other bigfixes and changes Jaret Burkett 2023-12-15 06:02:10 -07:00
e5177833b2 Targeted guidance work Jaret Burkett 2023-12-09 19:06:18 -07:00
eaa0fb6253 Tons of bug fixes and improvements to special training. Fixed slider training. Jaret Burkett 2023-12-09 16:38:10 -07:00
eaec2f5a52 Added guidance mentiods. WIP Jaret Burkett 2023-12-08 08:39:21 -07:00
92cb5ae096 Reworked targeted guidance algo Jaret Burkett 2023-12-01 06:30:52 -07:00
bd2bce9b92 Switched to trailing timestep spacing to make timesteps for consistant across schedulers. Honed in on targeted guidance. It is finally perfect. (I think) Jaret Burkett 2023-11-29 14:32:48 -07:00
537af79b0d Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2023-11-29 10:13:55 -07:00
7624241032 More fixes for noise schedules and fixed targeted guidance inverted masked prior Jaret Burkett 2023-11-29 10:13:31 -07:00
0d5943af91 Update readme for torch requirements Jaret Burkett 2023-11-28 12:52:55 -07:00
be815f9c47 updated requirements Jaret Burkett 2023-11-28 10:43:17 -07:00
3443d6aafa Updated Readme Jaret Burkett 2023-11-28 10:41:16 -07:00
bef10a639c Merge remote-tracking branch 'origin/development' Jaret Burkett 2023-11-28 10:40:05 -07:00
c2a4b8e058 More timestep fixes development Jaret Burkett 2023-11-28 10:34:08 -07:00
792a5e37e2 Numerous fixes for time sampling. Still not perfect Jaret Burkett 2023-11-28 07:34:43 -07:00
d7e55b6ad4 Bug fixes, negative prompting during training, hardened catching Jaret Burkett 2023-11-24 07:25:11 -07:00
fbec68681d Added timestep modifications to lcm scheduler for more evenly spaced timesteps Jaret Burkett 2023-11-17 23:26:52 -07:00
6280284d8b Fixed cleanup of emebddings. Jaret Burkett 2023-11-16 20:26:11 -07:00
ad50921c41 Sampling tests and added fixes for cleanups Jaret Burkett 2023-11-16 08:33:23 -07:00
e47006ed70 Added some features for an LCM condenser plugin Jaret Burkett 2023-11-15 08:56:45 -07:00
4f9cdd916a added prompt dropout to happen indempendently on each TE Jaret Burkett 2023-11-14 05:26:51 -07:00
7782caa468 Varous bug fixes. Finalized targeted guidance algo Jaret Burkett 2023-11-10 12:18:08 -07:00
fa6d91ba76 Diffirential guidance working, but I may have a better way Jaret Burkett 2023-11-09 06:09:36 -07:00
1ee62562a4 diffirential guidance is WORKING (from what I can tell) Jaret Burkett 2023-11-07 19:24:12 -07:00
dc8448d958 Added a way pass refiner ratio to sample config Jaret Burkett 2023-11-06 09:22:58 -07:00
a8b3b8b8da Fixed weight mapping for refiner Jaret Burkett 2023-11-06 07:37:47 -07:00
93ea955d7c Added refiner fine tuning. Works, but needs some polish. Jaret Burkett 2023-11-05 17:15:03 -07:00
8a9e8f708f Added base for using guidance during training. Still not working right. Jaret Burkett 2023-11-05 04:03:32 -07:00
d35733ac06 Added support for training ssd-1B. Added support for saving models into diffusers format. We can currently save in safetensors format for ssd-1b, but diffusers cannot load it yet. Jaret Burkett 2023-11-03 05:01:16 -06:00
ceaf1d9454 Various bug fixes, wip stuff, and tweaks Jaret Burkett 2023-11-02 18:19:20 -06:00
7d707b2fe6 Added masking to slider training. Something is still weird though Jaret Burkett 2023-11-01 14:51:29 -06:00
a899ec91c8 Added some split prompting started code, adamw8bit, replacements improving, learnable snr gos. A lot of good stuff. Jaret Burkett 2023-11-01 06:52:21 -06:00
436a09430e Added flat snr gamma vs min. Fixes timestep timing Jaret Burkett 2023-10-29 15:41:55 -06:00
3097865203 Fix llava import.. again Jaret Burkett 2023-10-29 12:52:17 -06:00

... 7 8 9 10 11 ...