ai-toolkit

ostris/ai-toolkit

Fork 0

mirror of https://github.com/ostris/ai-toolkit.git synced 2026-02-02 03:49:47 +00:00

Commit Graph

Select branches

Hide Pull Requests

WIP

dev

development

kohya-sdxl

lumina2

main

multi-gpu

qwen_image_edit

sdxl

wavelet_loss

#107

#109

#115

#12

#120

#124

#125

#128

#128

#138

#138

#141

#143

#155

#155

#156

#156

#158

#158

#160

#160

#168

#168

#173

#176

#177

#179

#179

#183

#184

#184

#185

#185

#191

#191

#196

#224

#224

#226

#226

#232

#233

#233

#244

#244

#245

#245

#246

#246

#249

#25

#25

#251

#251

#253

#253

#256

#263

#263

#264

#265

#265

#271

#273

#273

#274

#275

#275

#278

#283

#283

#290

#290

#300

#300

#302

#302

#303

#303

#304

#307

#318

#318

#326

#326

#331

#331

#333

#333

#341

#341

#342

#342

#343

#350

#350

#353

#353

#354

#354

#359

#362

#362

#373

#373

#375

#375

#377

#381

#381

#382

#382

#383

#385

#385

#392

#392

#395

#407

#407

#41

#417

#418

#420

#420

#422

#426

#427

#427

#430

#430

#431

#431

#432

#432

#434

#442

#442

#446

#447

#452

#455

#459

#459

#461

#461

#464

#465

#465

#467

#469

#469

#473

#473

#474

#488

#488

#489

#489

#492

#492

#494

#494

#496

#498

#498

#499

#499

#500

#509

#511

#513

#519

#519

#520

#520

#521

#522

#523

#525

#527

#527

#540

#547

#547

#551

#551

#555

#558

#559

#559

#563

#563

#564

#564

#567

#578

#578

#580

#580

#581

#581

#583

#594

#599

#60

#600

#600

#601

#601

#606

#606

#608

#608

#61

#610

#611

#614

#614

#616

#616

#629

#629

#630

#630

#633

#633

#637

#637

#639

#639

#644

#646

#648

#649

#649

#655

#655

#658

#658

#660

#661

#661

#666

#666

#668

#668

#673

#673

#682

#685

#685

#687

#687

#69

#69

#692

#692

#694

#694

#695

#695

#7

#80

#86

#86

#9

#95

8d48ad4e85 fixed bug I added to demo config Jaret Burkett 2024-08-11 10:28:39 -06:00
ec1ea7aa0e Added support for training on primary gpu with low_vram flag. Updated example script to remove creepy horse sample at that seed Jaret Burkett 2024-08-11 09:54:30 -06:00
fa02e774b0 Added info about datset Jaret Burkett 2024-08-10 15:08:05 -06:00
2308ef2868 Added flux training instructions Jaret Burkett 2024-08-10 14:10:02 -06:00
b3e03295ad Reworked flux pred. Again Jaret Burkett 2024-08-08 13:06:34 -06:00
e69a520616 Reworked timestep distribution on flowmatch sampler when training. Jaret Burkett 2024-08-08 06:01:45 -06:00
acafe9984f Adjustments to loading of flux. Added a feedback to ema Jaret Burkett 2024-08-07 13:17:26 -06:00
653fe60f16 Updates to flow matching algo Jaret Burkett 2024-08-07 15:04:17 +00:00
c2424087d6 8 bit training working on flux Jaret Burkett 2024-08-06 11:53:27 -06:00
272c8608c2 Make a CFG version of flux pipeline Jaret Burkett 2024-08-05 16:35:53 -06:00
99f24cfb0c Added a conversion script to convert my loras to peft format for flux Jaret Burkett 2024-08-05 14:54:10 -06:00
187663ab55 Use peft format for flux loras so they are compatible with diffusers. allow loading an assistant lora Jaret Burkett 2024-08-05 14:34:37 -06:00
edb7e827ee Adjusted flow matching so target noise multiplier works properly with it. Jaret Burkett 2024-08-05 11:40:05 -06:00
0ea27011d5 Bug fix Jaret Burkett 2024-08-04 11:07:19 -06:00
f321de7bdb Setup to retrain guidance embedding for flux. Use defualt timestep distribution for flux Jaret Burkett 2024-08-04 10:37:23 -06:00
88acc28d7f Prep for runpod docker Jaret Burkett 2024-08-03 12:41:06 -06:00
de2da96a81 Updat4ed requirements Jaret Burkett 2024-08-03 09:50:18 -06:00
9beea1c268 Flux training should work now... maybe Jaret Burkett 2024-08-03 09:17:34 -06:00
369aa143bc Only train a few blocks on flux (for now) Jaret Burkett 2024-08-03 07:02:27 -06:00
87ba867fdc Added flux training. Still a WIP. Wont train right without rectified flow working right Jaret Burkett 2024-08-02 15:00:30 -06:00
03613c523f Bugfixes and cleanup Jaret Burkett 2024-08-01 11:45:12 -06:00
47744373f2 Change img multiplier math Jaret Burkett 2024-07-30 11:33:41 -06:00
443c996e7f Do a noisy unconsitional for vision direct Jaret Burkett 2024-07-29 15:42:26 -06:00
8f0f467c20 Switch back to old ilora Jaret Burkett 2024-07-29 07:22:05 -06:00
e81e19fd0f Added target_norm_std which is a game changer Jaret Burkett 2024-07-28 16:08:33 -06:00
0bc4d555c7 A lot of pixart sigma training tweaks Jaret Burkett 2024-07-28 11:23:18 -06:00
80aa2dbb80 New image generation img2img. various tweaks and fixes Jaret Burkett 2024-07-24 04:13:41 -06:00
8d799031cf Remove reg as prior pred Jaret Burkett 2024-07-21 02:34:12 -06:00
6e92922c14 Add a mergable linear to the mid of ilora Jaret Burkett 2024-07-20 21:17:53 -06:00
c51235c486 Fixed misnamed var Jaret Burkett 2024-07-20 23:00:20 +00:00
c2c4e8cf34 Added ability to target parts of lora for ilora Jaret Burkett 2024-07-20 22:45:52 +00:00
4c249cf607 Added ilora2 Jaret Burkett 2024-07-20 16:40:57 -06:00
c2d5f712a3 Reworked ilora arch Jaret Burkett 2024-07-20 15:35:59 -06:00
22d2f6e28f Fixed issue with grad scaling Jaret Burkett 2024-07-20 08:21:57 -06:00
a2301cf28c Amall bug fixes Jaret Burkett 2024-07-18 10:39:55 -06:00
11e426fdf1 Various features and fixes. Too much brain fog to do a proper description Jaret Burkett 2024-07-18 07:34:14 -06:00
58dffd43a8 Added caching to image sizes so we dont do it every time. Jaret Burkett 2024-07-15 19:07:41 -06:00
e4558dff4b Partial implementation for training auraflow. Jaret Burkett 2024-07-12 12:11:38 -06:00
c062b7716c Varous bug fixes Jaret Burkett 2024-07-10 15:20:04 -06:00
c008405480 Added after model load hook Jaret Burkett 2024-07-09 15:34:48 -06:00
93e5df1d59 Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2024-07-07 07:56:56 -06:00
045e4a6e15 Save entire pixart model again Jaret Burkett 2024-07-07 07:56:48 -06:00
76f225a467 Fixed issue with TE adapter caption projection Jaret Burkett 2024-07-06 19:09:58 +00:00
cab8a1c7b8 WIP to add the caption_proj weight to pixart sigma TE adapter Jaret Burkett 2024-07-06 13:00:21 -06:00
acb06d6ff3 Bug fixes Jaret Burkett 2024-07-03 10:56:34 -06:00
bb57623a35 Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2024-06-29 15:53:40 -06:00
3072d20f17 Add ability to include conv_in and conv_out to full train when doing a lora Jaret Burkett 2024-06-29 14:54:50 -06:00
f6b21f47bb Increased the number of heads for ip adapters. Jaret Burkett 2024-06-28 16:09:52 +00:00
603ceca3ca added ema Jaret Burkett 2024-06-28 10:03:26 -06:00
657fd09f25 Added more control over Sigma sizes Jaret Burkett 2024-06-26 08:57:53 -06:00
8407c4deea Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2024-06-23 14:47:43 -06:00
64f2b085b7 Minor fixes Jaret Burkett 2024-06-23 14:47:40 -06:00
7165f2d25a Work to omprove pixart training Jaret Burkett 2024-06-23 20:46:48 +00:00
5d47244c57 Added support for pixart sigma loras Jaret Burkett 2024-06-16 11:56:30 -06:00
ada722c9e4 Fixed issue with heads not being added Jaret Burkett 2024-06-15 08:34:33 -06:00
696f73c30d Removed variant Jaret Burkett 2024-06-14 17:09:47 -06:00
e3410413b9 Rework head on ilora Jaret Burkett 2024-06-14 16:21:26 -06:00
37cebd9458 WIP Ilora Jaret Burkett 2024-06-14 09:31:01 -06:00
bd10d2d668 Some work on sd3 training. Not working Jaret Burkett 2024-06-13 12:19:16 -06:00
cb5d28cba9 Added working ilora trainer Jaret Burkett 2024-06-12 09:33:45 -06:00
3f3636b788 Bug fixes and little improvements here and there. Jaret Burkett 2024-06-08 06:24:20 -06:00
833c833f28 WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements Jaret Burkett 2024-05-27 10:50:24 -06:00
68b7e159bc Bug Fixes Jaret Burkett 2024-05-17 08:41:20 -06:00
5a45c709cd Work on ipadapters and custom adapters Jaret Burkett 2024-05-13 06:37:54 -06:00
10e1ecf1e8 Added single value adapter training Jaret Burkett 2024-04-28 06:04:47 -06:00
b96913d73c Improvements to dataloader Jaret Burkett 2024-04-27 09:28:28 -06:00
5da3613e0b Bug fixes and minor features Jaret Burkett 2024-04-25 06:14:31 -06:00
5a70b7f38d Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded. Jaret Burkett 2024-04-20 10:46:56 -06:00
377b81ee3e Adjustments to guidance Jaret Burkett 2024-04-19 15:00:35 -06:00
2d0a1be59d Bug fixes Jaret Burkett 2024-04-16 03:48:13 -06:00
7284aab7c0 Added specialized scaler training to ip adapters Jaret Burkett 2024-04-05 08:17:09 -06:00
427847ac4c Small tweaks and fixes for specialized ip adapter training Jaret Burkett 2024-03-26 11:35:26 -06:00
9c1cc9641e Added keep tokens to keep so many tokens in a prompt when dropping Jaret Burkett 2024-03-18 13:18:25 -06:00
89f4bcad2e Lock diffusers to 0.26.3 until I can figure out why future versions break LoRA code Jaret Burkett 2024-03-18 10:17:55 -06:00
016687bda1 Adapter work. Bug fixes. Auto adjust LR when resuming optimizer. Jaret Burkett 2024-03-17 10:21:47 -06:00
72de68d8aa WIP on clip vision encoder Jaret Burkett 2024-03-13 07:24:08 -06:00
d87b49882c Work on embedding adapters Jaret Burkett 2024-03-11 15:18:42 -06:00
f415bac7b5 Merge branch 'main' of github.com:ostris/ai-toolkit Jaret Burkett 2024-03-06 09:32:38 -07:00
f1cb87fe9e fixed bug the kept learning rates the same Jaret Burkett 2024-03-06 09:23:32 -07:00
8f9cd823d1 Create LICENSE Jaret Burkett 2024-03-06 07:54:55 -07:00
b01e8d889a Added stochastic rounding to adafactor. ILora adjustments Jaret Burkett 2024-03-05 07:07:09 -07:00
1325613583 rework ilora Jaret Burkett 2024-02-29 07:55:52 -07:00
337945de9a Added this not that guidance. Added ability to replace prompts. Jaret Burkett 2024-02-28 20:10:14 -07:00
561914d8e6 Removed old code for fixing multistep sampler that is no longer needed Jaret Burkett 2024-02-25 11:53:35 -07:00
b0a0f28191 Bug fixes Jaret Burkett 2024-02-25 08:28:29 -07:00
f965a1299f Fixed Dora implementation. Still highly experimental Jaret Burkett 2024-02-24 10:26:01 -07:00
1bd94f0f01 Added early DoRA support, but will change shortly. Dont use right now. Jaret Burkett 2024-02-23 05:55:41 -07:00
9ffa8c3711 Fixed issue when there is no adapter Jaret Burkett 2024-02-22 02:59:59 -07:00
b68c3ef734 Added te aug adapter Jaret Burkett 2024-02-21 21:30:26 -07:00
49c41e6a5f Bug fixes. allow for random negative prompts Jaret Burkett 2024-02-21 04:51:52 -07:00
2478554c95 Bug fixes. Added IP adapter training for Pixart Jaret Burkett 2024-02-17 10:06:57 -07:00
93b52932c1 Added training for pixart-a Jaret Burkett 2024-02-13 16:00:04 -07:00
4ec4025cbb Added adapter modules for text encoders and direct vision Jaret Burkett 2024-02-12 08:46:18 -07:00
e074058faa Work on additional image embedding methods. Finalized zipper resampler. It works amazing Jaret Burkett 2024-02-10 09:00:05 -07:00
a8481c1670 randomly adjust scale of unconditional noise on ip adapters if training with cfg Jaret Burkett 2024-02-06 03:44:54 -07:00
e18e0cb5f8 Added comparitive loss when training clip encoder. Allow selecting clip layer. on ip adapter. Improvements to prior prediction Jaret Burkett 2024-02-05 07:40:03 -07:00
177c7130ec improved correction of pred norm by targeting the prior Jaret Burkett 2024-02-01 06:31:04 -07:00
1ae1017748 Bug fixes. added ability to use l1 loss. varous other tests and improvements Jaret Burkett 2024-01-31 06:30:54 -07:00
92b9c71d44 Many bug fixes. Ip adapter bug fixes. Added noise to unconditional, it works better. added an ilora adapter for 1 shotting LoRAs Jaret Burkett 2024-01-28 08:20:03 -07:00
f17ad8d794 various bug fixes. Created an contextual alpha mask module to calculate alpha mask Jaret Burkett 2024-01-18 16:34:27 -07:00