Jaret Burkett
e6739f7eb2
Convert wan lora weights on save to be something comfy can handle
2025-03-08 12:55:11 -07:00
Jaret Burkett
391cf80fea
Added training for Wan2.1. Not finalized, wait.
2025-03-07 13:53:44 -07:00
Jaret Burkett
6f6fb90812
Added cogview4. Loss still needs work.
2025-03-04 18:43:52 -07:00
Jaret Burkett
acc79956aa
WIP create new class to add new models more easily
2025-03-01 13:49:02 -07:00
Jaret Burkett
3e49337a58
Set step to the last step saved at when exiting
2025-02-23 13:21:22 -07:00
Jaret Burkett
60f848a877
Send more data when loading the model to the ui
2025-02-23 12:49:54 -07:00
Jaret Burkett
ad87f72384
Start, stop, monitor jobs from ui working.
2025-02-21 09:49:28 -07:00
Jaret Burkett
adcf884c0f
Built out the ui trainer plugin with db comminication
2025-02-21 05:53:35 -07:00
Jaret Burkett
8450aca10e
Fixed missed merge conflice and locked diffusers version
2025-02-12 09:40:02 -07:00
Jaret Burkett
0b8a32def7
merged in lumina2 branch
2025-02-12 09:33:03 -07:00
Jaret Burkett
787bb37e76
Small fixed for DFE, polar guidance, and other things
2025-02-12 09:27:44 -07:00
Jaret Burkett
10aa7e9d5e
Fixed some breaking changes with diffusers gradient checkpointing.
2025-02-10 09:35:31 -07:00
Jaret Burkett
d138f07365
Imitial lumina3 support
2025-02-08 10:59:53 -07:00
Jaret Burkett
c6d8eedb94
Added ability to use consistent noise for each image in a dataset by hashing the path and using that as a seed.
2025-02-08 07:13:48 -07:00
Jaret Burkett
216ab164ce
Experimental features and bug fixes
2025-02-04 13:36:34 -07:00
Jaret Burkett
e6180d1e1d
Bug fixes
2025-01-31 13:23:01 -07:00
Jaret Burkett
15a57bc89f
Add new version of DFE. Kitchen sink
2025-01-31 11:42:27 -07:00
Jaret Burkett
34a1c6947a
Added flux_shift as timestep type
2025-01-27 07:35:00 -07:00
Jaret Burkett
5e663746b8
Working multi gpu training. Still need a lot of tweaks and testing.
2025-01-25 16:46:20 -07:00
Jaret Burkett
89dd041b97
Added ability to pair samples with a closer noise with optimal_noise_pairing_samples
2025-01-21 18:30:10 -07:00
Jaret Burkett
4723f23c0d
Added ability to split up flux across gpus (experimental). Changed the way timestep scheduling works to prep for more specific schedules.
2024-12-31 07:06:55 -07:00
Jaret Burkett
8ef07a9c36
Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications
2024-12-15 08:59:27 -07:00
Jaret Burkett
f213996aa5
Fixed saving and displaying for automagic
2024-11-29 08:00:22 -07:00
Jaret Burkett
67c2e44edb
Added support for training flux redux adapters
2024-11-21 20:01:52 -07:00
Jaret Burkett
96d418bb95
Added support for full finetuning flux with randomized param activation. Examples coming soon
2024-11-21 13:05:32 -07:00
Jaret Burkett
58f9d01c2b
Added adafactor implementation that handles stochastic rounding of update and accumulation
2024-10-30 05:25:57 -06:00
Jaret Burkett
4747716867
Fixed issue with adapters not providing gradients with new grad activator
2024-10-29 14:22:10 -06:00
Jaret Burkett
22cd40d7b9
Improvements for full tuning flux. Added debugging launch config for vscode
2024-10-29 04:54:08 -06:00
Jaret Burkett
3400882a80
Added preliminary support for SD3.5-large lora training
2024-10-22 12:21:36 -06:00
Jaret Burkett
9f94c7b61e
Added experimental param multiplier to the ema module
2024-10-22 09:25:52 -06:00
Jaret Burkett
ab22674980
Allow for a default caption file in the folder. Minor bug fixes.
2024-10-10 07:31:33 -06:00
Jaret Burkett
04424fe2d6
Added config setting to set the timestep type
2024-09-24 06:53:59 -06:00
Jaret Burkett
2776221497
Added option to cache empty prompt or trigger and unload text encoders while training
2024-09-21 20:54:09 -06:00
apolinário
bc693488eb
fix diffusers codebase ( #183 )
2024-09-21 11:50:29 -06:00
Plat
79b4e04b80
Feat: Wandb logging ( #95 )
...
* wandb logging
* fix: start logging before train loop
* chore: add wandb dir to gitignore
* fix: wrap wandb functions
* fix: forget to send last samples
* chore: use valid type
* chore: use None when not type-checking
* chore: resolved complicated logic
* fix: follow log_every
---------
Co-authored-by: Plat <github@p1at.dev >
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com >
2024-09-19 20:01:01 -06:00
Jaret Burkett
121a760c19
Added proper grad accumulation
2024-09-03 07:24:18 -06:00
Jaret Burkett
d44d4eb61a
Added a new experimental linear weighing technique
2024-09-02 09:22:13 -06:00
apolinário
562405923f
Update README.md for push_to_hub ( #143 )
...
Add diffusers examples and clarify how to use the model locally
2024-08-30 16:34:28 -06:00
apolinário
4d35a29c97
Add push_to_hub to the trainer ( #109 )
...
* add push_to_hub
* fix indentation
* indent again
* model_config
* allow samples to not exist
* repo creation fix
* dont show empty [] if widget doesnt exist
* dont submit the config and optimizer
* Unsafe to have tokens saved in the yaml file
* make sure to catch only the latest samples
* change name to slug
* formatting
* formatting
---------
Co-authored-by: multimodalart <joaopaulo.passos+multimodal@gmail.com >
2024-08-22 21:18:56 -06:00
Jaret Burkett
af108bb964
Bug fix with dataloader. Added a flag to completly disable sampling
2024-08-12 09:19:40 -06:00
Jaret Burkett
a6aa4b2c7d
Added ability to set timesteps to linear for flowmatching schedule
2024-08-11 13:06:08 -06:00
Jaret Burkett
e69a520616
Reworked timestep distribution on flowmatch sampler when training.
2024-08-08 06:01:45 -06:00
Jaret Burkett
acafe9984f
Adjustments to loading of flux. Added a feedback to ema
2024-08-07 13:17:26 -06:00
Jaret Burkett
c2424087d6
8 bit training working on flux
2024-08-06 11:53:27 -06:00
Jaret Burkett
edb7e827ee
Adjusted flow matching so target noise multiplier works properly with it.
2024-08-05 11:40:05 -06:00
Jaret Burkett
f321de7bdb
Setup to retrain guidance embedding for flux. Use defualt timestep distribution for flux
2024-08-04 10:37:23 -06:00
Jaret Burkett
9beea1c268
Flux training should work now... maybe
2024-08-03 09:17:34 -06:00
Jaret Burkett
87ba867fdc
Added flux training. Still a WIP. Wont train right without rectified flow working right
2024-08-02 15:00:30 -06:00
Jaret Burkett
03613c523f
Bugfixes and cleanup
2024-08-01 11:45:12 -06:00
Jaret Burkett
47744373f2
Change img multiplier math
2024-07-30 11:33:41 -06:00
