Commit Graph

Select branches

Hide Pull Requests

WIP

dev

development

hidream_o1

kohya-sdxl

lumina2

main

multi-gpu

qwen_image_edit

sdxl

wavelet_loss

#107

#109

#115

#12

#120

#124

#125

#128

#138

#141

#143

#155

#156

#158

#160

#160

#168

#173

#176

#177

#179

#183

#184

#185

#191

#196

#224

#226

#232

#233

#244

#245

#246

#249

#25

#251

#253

#253

#256

#263

#264

#265

#271

#273

#274

#275

#278

#283

#290

#300

#302

#302

#303

#304

#307

#318

#326

#326

#331

#333

#341

#342

#343

#350

#353

#354

#359

#362

#373

#375

#377

#381

#382

#383

#385

#392

#392

#395

#407

#41

#417

#418

#420

#422

#426

#427

#430

#431

#431

#432

#434

#442

#446

#447

#452

#455

#459

#461

#464

#465

#467

#469

#473

#474

#488

#489

#492

#494

#496

#498

#499

#500

#509

#511

#513

#519

#520

#521

#522

#523

#525

#527

#540

#547

#551

#555

#558

#559

#563

#563

#564

#564

#567

#578

#580

#581

#583

#594

#599

#60

#600

#601

#601

#606

#608

#61

#610

#611

#614

#616

#629

#630

#633

#637

#639

#644

#646

#648

#649

#655

#658

#660

#661

#666

#666

#668

#668

#673

#682

#685

#687

#69

#692

#694

#694

#695

#697

#698

#699

#7

#702

#703

#705

#706

#707

#708

#709

#709

#713

#713

#717

#718

#718

#719

#720

#721

#724

#724

#726

#727

#727

#732

#732

#733

#738

#742

#743

#743

#745

#748

#749

#754

#759

#760

#761

#762

#763

#765

#765

#766

#766

#767

#768

#769

#770

#774

#774

#777

#777

#778

#778

#782

#782

#783

#785

#793

#80

#800

#803

#803

#805

#805

#807

#807

#808

#808

#809

#811

#811

#812

#812

#813

#813

#814

#814

#819

#819

#825

#825

#827

#828

#828

#829

#829

#86

#9

#95

Mono Color

• b84e3260cb Fix llava import Jaret Burkett 2023-10-29 12:50:54 -06:00
• 48a9bac22d Added doffusers schedulers Jaret Burkett 2023-10-29 12:39:50 -06:00
• 298001439a Added gradient accumulation finally Jaret Burkett 2023-10-28 13:14:29 -06:00
• 6f3e0d5af2 Improved lorm extraction and training Jaret Burkett 2023-10-28 08:21:59 -06:00
• 0a79ac9604 Added lorm. WIP Jaret Burkett 2023-10-26 18:23:51 -06:00
• 9636194c09 Added fuyu captioning Jaret Burkett 2023-10-25 14:14:53 -06:00
• d742792ee4 Fixed issue with not loading short prompt Jaret Burkett 2023-10-24 16:19:32 -06:00
• 002279cec3 Allow short and long caption combinations like form the new captioning system. Merge the network into the model before inference and reextract when done. Doubles inference speed on locon models during inference. allow splitting a batch into individual components and run them through alone. Basicallt gradient accumulation with single batch size. Jaret Burkett 2023-10-24 16:02:07 -06:00
• 73c8b50975 Added ability to use adagrad from transformers Jaret Burkett 2023-10-24 11:16:01 -06:00
• 34eb563d55 Added dataset tagging and management tools using llava Jaret Burkett 2023-10-24 10:11:47 -06:00
• dc36bbb3c8 Added long prompts to general training Jaret Burkett 2023-10-23 08:12:58 -06:00
• 9905a1e205 Fixes and longer prompts Jaret Burkett 2023-10-22 08:57:37 -06:00
• 0e9fc42816 Allow for inverted masked prior Jaret Burkett 2023-10-21 06:50:17 -06:00
• d46112a354 Fixes for aug pipeline Jaret Burkett 2023-10-20 13:24:07 -06:00
• 07bf7bd7de Allow augmentations and targeting different loss types fron the config file Jaret Burkett 2023-10-18 03:04:57 -06:00
• da6302ada8 added a method to apply multipliers to noise and latents prior to combining Jaret Burkett 2023-10-17 06:09:16 -06:00
• a05459afaf Fixed issue with adapters that only had 1 input channel. Added ability to set the percentage chance of adapter matching Jaret Burkett 2023-10-15 15:13:35 -06:00
• b1a22d0b3e hardened reading prompts from json Jaret Burkett 2023-10-15 07:20:33 -06:00
• 7909b50d24 Added adapter assistance to SD training Jaret Burkett 2023-10-14 08:44:53 -06:00
• 38e441a29c allow flipping for point of interesting autocropping. allow num repeats. Fixed some bugs with new free u Jaret Burkett 2023-10-12 21:02:47 -06:00
• 4e3b2c2569 Allow for alpha to be used as a mask Jaret Burkett 2023-10-10 17:08:59 -06:00
• 239addba51 Fixed memory leak when cachine latents to disk Jaret Burkett 2023-10-10 13:31:47 -06:00
• 63ceffae24 Massive speed increases and ram optimizations Jaret Burkett 2023-10-10 06:07:55 -06:00
• f4c90bb589 Added config to set the min value of a mask Jaret Burkett 2023-10-09 15:47:54 -06:00
• bb1d3793e3 Added ability to add masks to dataloader and sd trainer to adjust weight of image Jaret Burkett 2023-10-09 11:21:00 -06:00
• 1d3de678aa fixed bug with trigger word embedding. Allow control images to load from the dataloader or legacy way Jaret Burkett 2023-10-09 06:21:49 -06:00
• b1cfafa0c6 always prompt both even when training only one encoder Jaret Burkett 2023-10-07 11:23:09 -06:00
• cac8754399 Allow for training loras on onle one text encoder for sdxl Jaret Burkett 2023-10-06 08:11:56 -06:00
• f73402473b Bug fixes. Added some functionality to help with private extensions Jaret Burkett 2023-10-05 07:09:34 -06:00
• 579650eaf8 Fixed big issue with bucketing dataloader and added random cripping to a point of interest Jaret Burkett 2023-10-02 18:31:08 -06:00
• 320e109c5f Allow loading from a json detail file for captions Jaret Burkett 2023-10-02 13:25:09 -06:00
• 560251a24f fixed issue with down block residuals when doing slider cfg on sdxl with t2i adapter assisted training Jaret Burkett 2023-10-01 07:32:48 -06:00
• 085787b799 Allow loading auxillery images from dataloader Jaret Burkett 2023-09-30 07:28:23 -06:00
• 8d9450ad7c Compatability fixes Jaret Burkett 2023-09-29 14:07:37 -06:00
• 8509da60cb Added a way to add a t2i adapter guided slider training for more consitant images Jaret Burkett 2023-09-28 14:08:56 -06:00
• c5d49ba661 Allow adapter image to be cropped to match bucket cropping Jaret Burkett 2023-09-25 10:38:54 -06:00
• 76c764af49 Fixed issues with adapter and disabled telementry for now Jaret Burkett 2023-09-24 12:51:29 -06:00
• abf7cd221d allow setting adapter weight in prompts Jaret Burkett 2023-09-24 06:51:54 -06:00
• e5153d87c9 Fixed issues with dataloader bucketing. Allow using standard base image for t2i adapters. Jaret Burkett 2023-09-24 05:19:57 -06:00
• 830e87cb87 Added IP adapter training. Not functioning correctly yet Jaret Burkett 2023-09-24 02:39:43 -06:00
• 19255cdc7c Bugfixes. Added small augmentations to dataloader. Will switch to abluminations soon though. Added ability to adjust step count on start to override what is in the file Jaret Burkett 2023-09-20 05:30:10 -06:00
• 0f105690cc Added some further extendability for plugins Jaret Burkett 2023-09-19 05:41:44 -06:00
• 61badf85a7 t2i training working from what I can tell at least Jaret Burkett 2023-09-17 15:56:43 -06:00
• 181f237a7b added flipping x and y for dataset loader Jaret Burkett 2023-09-17 08:42:54 -06:00
• c698837241 Fixes to esrgan trainer. Moved logic for sd prompt embeddings out of diffusers pipeline so I can manipulate it Jaret Burkett 2023-09-16 17:41:07 -06:00
• 27f343fc08 Added base setup for training t2i adapters. Currently untested, saw something else shiny i wanted to finish sirst. Added content_or_style to the training config. It defaults to balanced, which is standard uniform time step sampling. If style or content is passed, it will use cubic sampling for timesteps to favor timesteps that are beneficial for training them. for style, favor later timesteps. For content, favor earlier timesteps. Jaret Burkett 2023-09-16 08:30:38 -06:00
• 3eb3535683 Merge pull request #12 from bendeguzvaradi/main Jaret Burkett 2023-09-14 15:31:09 -06:00
• 17e4fe40d7 Prevent lycoris network moduels if not training that part of network. Skew timesteps to favor later steps. It performs better Jaret Burkett 2023-09-14 15:13:24 -06:00
• 569d7464d5 implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar Jaret Burkett 2023-09-14 08:31:54 -06:00
• 4e945917df added dropout to LoRA networks Jaret Burkett 2023-09-13 15:23:07 -06:00
• ae70200d3c Bug fixes, speed improvements, compatability adjustments withdiffusers updates Jaret Burkett 2023-09-13 07:03:53 -06:00
• d8d1e6fd1e big fixes Jaret Burkett 2023-09-12 18:48:39 -06:00
• 257da9493d Dont load image if we are cachine latents Jaret Burkett 2023-09-12 18:39:41 -06:00
• b5a2669b74 Fixed memory leak Jaret Burkett 2023-09-12 07:03:10 -06:00
• d74dd636ee Memory optimizations. Default to using cudamalloc when torch 2.0 for mem allocation Jaret Burkett 2023-09-12 04:30:23 -06:00
• e8583860ad Upgraded to dev for t2i on diffusers. Minor migrations to make it work. Jaret Burkett 2023-09-11 14:46:06 -06:00
• 3d387103cd safety checker to None bendeguzvaradi 2023-09-11 17:05:55 +02:00
• 083cefa78c Bugfixes for slider reference Jaret Burkett 2023-09-10 18:36:23 -06:00
• b5ec8e4eb1 Improve reference slider memory and speed Jaret Burkett 2023-09-10 18:26:44 -06:00
• 708b07adb7 Fixed issue with interleaving when doing cfg Jaret Burkett 2023-09-10 10:26:58 -06:00
• a437aed45f bug fix Jaret Burkett 2023-09-10 09:52:14 -06:00
• 34bfeba229 Massive speed increase. Added latent caching both to disk and to memory Jaret Burkett 2023-09-10 08:54:49 -06:00
• 41a3f63b72 allow smaller images in buckets and bucket them Jaret Burkett 2023-09-10 03:43:02 -06:00
• 626ed2939a bug fixes Jaret Burkett 2023-09-09 15:04:44 -06:00
• 2128ac1e08 fixed issue with embed name, save whole config to dir instead of just process so it can be easily shared. Only make one config, no timesteps Jaret Burkett 2023-09-09 12:24:08 -06:00
• be804c9cf5 Save embeddings as their trigger to match auto and comfy style loading. Also, FINALLY found why gradients were wonkey and fixed it. The root problem is dropping out of network state before backward pass. Jaret Burkett 2023-09-09 12:02:07 -06:00
• 408c50ead1 actually got gradient checkpointing working, again, again, maybe Jaret Burkett 2023-09-09 11:27:42 -06:00
• 4ed03a8d92 Fixed issue with buckets scaling. again Jaret Burkett 2023-09-08 16:32:14 -06:00
• b01ab5d375 FINALLY fixed gradient checkpointing issue. Big batches baby. Jaret Burkett 2023-09-08 15:21:46 -06:00
• cb91b0d6da Changed model download from HF to fp16 Jaret Burkett 2023-09-08 07:57:19 -06:00
• ce4f9fe02a Bug fixes and improvements to token injection Jaret Burkett 2023-09-08 06:10:59 -06:00
• 92a086d5a5 Fixed issue with token replacements Jaret Burkett 2023-09-07 13:42:39 -06:00
• 3feb663a51 Switched to new bucket system that matched sdxl trained buckets. Fixed requirements. Updated embeddings to work with sdxl. Added method to train lora with an embedding at the trigger. Still testing but works amazingly well from what I can see Jaret Burkett 2023-09-07 13:06:18 -06:00
• 436bf0c6a3 Added experimental concept replacer, replicate converter, bucket maker, and other goodies Jaret Burkett 2023-09-06 18:50:32 -06:00
• f84500159c Fixed issue with lora layer check Jaret Burkett 2023-09-04 14:27:37 -06:00
• 64a5441832 Fully tested and now supporting locon on sdxl. If you have the ram Jaret Burkett 2023-09-04 14:05:10 -06:00
• a4c3507a62 Added LoCON from LyCORIS Jaret Burkett 2023-09-04 08:48:07 -06:00
• fa8fc32c0a Corrected key saving and loading to better match kohya Jaret Burkett 2023-09-04 00:22:34 -06:00
• 22ed539321 Allow special args for schedulers Jaret Burkett 2023-09-03 20:38:44 -06:00
• 7cd6945082 Added my annotator/preprocessor and improved network jitter on reference trainer Jaret Burkett 2023-09-03 16:43:51 -06:00
• 2a40937b4f reworked samplers. Trying to find what is wrong with diffusers sampling is sdxl Jaret Burkett 2023-09-03 07:56:09 -06:00
• 4ca819a05e Fixes for dataloader Jaret Burkett 2023-08-31 04:54:10 -06:00
• addf024630 Fixed issue with omitting square pictures Jaret Burkett 2023-08-30 15:00:22 -06:00
• 33267e117c Reworked bucket loader to scale buckets to pixels amounts not just minimum size. Makes the network more consistant Jaret Burkett 2023-08-30 14:52:12 -06:00
• d401348c2e Make data loader resiliant to bad headers in meta Jaret Burkett 2023-08-29 18:56:06 -06:00
• 836fee47a6 Fixed some mismatched weights by adjusting tolerance. The mismatch ironically made the models better lol Jaret Burkett 2023-08-29 15:20:03 -06:00
• 14ff51ceb4 fixed issues with converting and saving models. Cleaned keys. Improved testing for cycle load saving. Jaret Burkett 2023-08-29 12:31:19 -06:00
• 714854ee86 Hude rework to move the batch to a DTO to make it far more modular to the future ui Jaret Burkett 2023-08-29 10:22:19 -06:00
• bd758ff203 Cleanup and small bug fixes Jaret Burkett 2023-08-29 05:45:49 -06:00
• a008d9e63b Fixed issue with loadin models after resume function added. Added additional flush if not training text encoder to clear out vram before grad accum Jaret Burkett 2023-08-28 17:56:30 -06:00
• b79ced3e10 Merge branch 'main' into development Jaret Burkett 2023-08-28 16:21:51 -06:00
• bee0b6a235 Added converters for all stable diffusion models to convert back to ldm format from diffusers. Jaret Burkett 2023-08-28 16:12:32 -06:00
• 2ecb5cf024 Merge branch 'main' into development Jaret Burkett 2023-08-28 14:01:44 -06:00
• fab7c2b04a Fixed issue with key mapping from diffusers back to ldm Jaret Burkett 2023-08-28 14:01:26 -06:00
• e866c75638 Built base interfaces for a DTO to handle batch infomation transports for the dataloader Jaret Burkett 2023-08-28 12:43:31 -06:00
• 71da78c8af improved normalization for a network with varrying batch network weights Jaret Burkett 2023-08-28 12:42:57 -06:00
• c446f768ea Huge memory optimizations, many big fixes Jaret Burkett 2023-08-27 17:48:02 -06:00
• cc49786ee9 Dataloader bug fixes Jaret Burkett 2023-08-27 14:36:38 -06:00
• 9b164a8688 Fixed issue with bucket dataloader corpping in too much. Added normalization capabilities to LoRA modules. Testing effects, but should prevent them from burning and also make them more compatable with stacking many LoRAs Jaret Burkett 2023-08-27 09:40:01 -06:00
• 6bd3851058 Fixed issue with prompt token replace adding more than one replacement Jaret Burkett 2023-08-26 18:52:23 -06:00