diff --git a/README.md b/README.md index 1e3b727b..cf142480 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ -I am transitioning to working on my open source AI projects full time. If you find my work useful, please consider supporting me on [Patreon](https://www.patreon.com/ostris). I will be able to work on more projects and provide better support with your help. +I work on open source full time, which means I 100% rely on donations to make a living. If you find this project helpful, or use it in for commercial purposes, please consider donating to support my work on [Patreon](https://www.patreon.com/ostris) or [Gihub Sponsors](https://github.com/sponsors/ostris). ## Installation @@ -18,7 +18,6 @@ Requirements: - git - Linux: ```bash git clone https://github.com/ostris/ai-toolkit.git @@ -43,6 +42,43 @@ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121 pip install -r requirements.txt ``` + +# AI Toolkit UI + +AI Toolkit UI + +The AI Toolkit UI is a web interface for the AI Toolkit. It allows you to easily start, stop, and monitor jobs. It also allows you to easily train models with a few clicks. It is still in early beta and will likely have bugs and frequent breaking changes. It is currently only tested on linux for now. + + +WARNING: The UI is not secure and should not be exposed to the internet. It is only meant to be run locally or on a server that does not have ports exposed. Adding additional security is on the roadmap. + +## Installing the UI + +Requirements: +- Node.js > 18 + +You will need to do this with every update as well. + +```bash +cd ui +npm install +npm run build +npm run update_db +``` + +## Running the UI + +Make sure you built it as shown above. The UI does not need to be kept running for the jobs to run. It is only needed to start/stop/monitor jobs. + +```bash +cd ui +npm run start +``` + +You can now access the UI at `http://localhost:8675` or `http://:8675` if you are running it on a server. + + + ## FLUX.1 Training ### Tutorial @@ -275,186 +311,3 @@ You can also exclude layers by their names by using `ignore_if_contains` network `ignore_if_contains` takes priority over `only_if_contains`. So if a weight is covered by both, if will be ignored. - ---- - -## EVERYTHING BELOW THIS LINE IS OUTDATED - -It may still work like that, but I have not tested it in a while. - ---- - -### Batch Image Generation - -A image generator that can take frompts from a config file or form a txt file and generate them to a -folder. I mainly needed this for an SDXL test I am doing but added some polish to it so it can be used -for generat batch image generation. -It all runs off a config file, which you can find an example of in `config/examples/generate.example.yaml`. -Mere info is in the comments in the example - ---- - -### LoRA (lierla), LoCON (LyCORIS) extractor - -It is based on the extractor in the [LyCORIS](https://github.com/KohakuBlueleaf/LyCORIS) tool, but adding some QOL features -and LoRA (lierla) support. It can do multiple types of extractions in one run. -It all runs off a config file, which you can find an example of in `config/examples/extract.example.yml`. -Just copy that file, into the `config` folder, and rename it to `whatever_you_want.yml`. -Then you can edit the file to your liking. and call it like so: - -```bash -python3 run.py config/whatever_you_want.yml -``` - -You can also put a full path to a config file, if you want to keep it somewhere else. - -```bash -python3 run.py "/home/user/whatever_you_want.yml" -``` - -More notes on how it works are available in the example config file itself. LoRA and LoCON both support -extractions of 'fixed', 'threshold', 'ratio', 'quantile'. I'll update what these do and mean later. -Most people used fixed, which is traditional fixed dimension extraction. - -`process` is an array of different processes to run. You can add a few and mix and match. One LoRA, one LyCON, etc. - ---- - -### LoRA Rescale - -Change `` to `` or whatever you want with the same effect. -A tool for rescaling a LoRA's weights. Should would with LoCON as well, but I have not tested it. -It all runs off a config file, which you can find an example of in `config/examples/mod_lora_scale.yml`. -Just copy that file, into the `config` folder, and rename it to `whatever_you_want.yml`. -Then you can edit the file to your liking. and call it like so: - -```bash -python3 run.py config/whatever_you_want.yml -``` - -You can also put a full path to a config file, if you want to keep it somewhere else. - -```bash -python3 run.py "/home/user/whatever_you_want.yml" -``` - -More notes on how it works are available in the example config file itself. This is useful when making -all LoRAs, as the ideal weight is rarely 1.0, but now you can fix that. For sliders, they can have weird scales form -2 to 2 -or even -15 to 15. This will allow you to dile it in so they all have your desired scale - ---- - -### LoRA Slider Trainer - - - Open In Colab - - -This is how I train most of the recent sliders I have on Civitai, you can check them out in my [Civitai profile](https://civitai.com/user/Ostris/models). -It is based off the work by [p1atdev/LECO](https://github.com/p1atdev/LECO) and [rohitgandikota/erasing](https://github.com/rohitgandikota/erasing) -But has been heavily modified to create sliders rather than erasing concepts. I have a lot more plans on this, but it is -very functional as is. It is also very easy to use. Just copy the example config file in `config/examples/train_slider.example.yml` -to the `config` folder and rename it to `whatever_you_want.yml`. Then you can edit the file to your liking. and call it like so: - -```bash -python3 run.py config/whatever_you_want.yml -``` - -There is a lot more information in that example file. You can even run the example as is without any modifications to see -how it works. It will create a slider that turns all animals into dogs(neg) or cats(pos). Just run it like so: - -```bash -python3 run.py config/examples/train_slider.example.yml -``` - -And you will be able to see how it works without configuring anything. No datasets are required for this method. -I will post an better tutorial soon. - ---- - -## Extensions!! - -You can now make and share custom extensions. That run within this framework and have all the inbuilt tools -available to them. I will probably use this as the primary development method going -forward so I dont keep adding and adding more and more features to this base repo. I will likely migrate a lot -of the existing functionality as well to make everything modular. There is an example extension in the `extensions` -folder that shows how to make a model merger extension. All of the code is heavily documented which is hopefully -enough to get you started. To make an extension, just copy that example and replace all the things you need to. - - -### Model Merger - Example Extension -It is located in the `extensions` folder. It is a fully finctional model merger that can merge as many models together -as you want. It is a good example of how to make an extension, but is also a pretty useful feature as well since most -mergers can only do one model at a time and this one will take as many as you want to feed it. There is an -example config file in there, just copy that to your `config` folder and rename it to `whatever_you_want.yml`. -and use it like any other config file. - -## WIP Tools - - -### VAE (Variational Auto Encoder) Trainer - -This works, but is not ready for others to use and therefore does not have an example config. -I am still working on it. I will update this when it is ready. -I am adding a lot of features for criteria that I have used in my image enlargement work. A Critic (discriminator), -content loss, style loss, and a few more. If you don't know, the VAE -for stable diffusion (yes even the MSE one, and SDXL), are horrible at smaller faces and it holds SD back. I will fix this. -I'll post more about this later with better examples later, but here is a quick test of a run through with various VAEs. -Just went in and out. It is much worse on smaller faces than shown here. - - - ---- - -## TODO -- [X] Add proper regs on sliders -- [X] Add SDXL support (base model only for now) -- [ ] Add plain erasing -- [ ] Make Textual inversion network trainer (network that spits out TI embeddings) - ---- - -## Change Log - -#### 2023-08-05 - - Huge memory rework and slider rework. Slider training is better thant ever with no more -ram spikes. I also made it so all 4 parts of the slider algorythm run in one batch so they share gradient -accumulation. This makes it much faster and more stable. - - Updated the example config to be something more practical and more updated to current methods. It is now -a detail slide and shows how to train one without a subject. 512x512 slider training for 1.5 should work on -6GB gpu now. Will test soon to verify. - - -#### 2021-10-20 - - Windows support bug fixes - - Extensions! Added functionality to make and share custom extensions for training, merging, whatever. -check out the example in the `extensions` folder. Read more about that above. - - Model Merging, provided via the example extension. - -#### 2023-08-03 -Another big refactor to make SD more modular. - -Made batch image generation script - -#### 2023-08-01 -Major changes and update. New LoRA rescale tool, look above for details. Added better metadata so -Automatic1111 knows what the base model is. Added some experiments and a ton of updates. This thing is still unstable -at the moment, so hopefully there are not breaking changes. - -Unfortunately, I am too lazy to write a proper changelog with all the changes. - -I added SDXL training to sliders... but.. it does not work properly. -The slider training relies on a model's ability to understand that an unconditional (negative prompt) -means you do not want that concept in the output. SDXL does not understand this for whatever reason, -which makes separating out -concepts within the model hard. I am sure the community will find a way to fix this -over time, but for now, it is not -going to work properly. And if any of you are thinking "Could we maybe fix it by adding 1 or 2 more text -encoders to the model as well as a few more entirely separate diffusion networks?" No. God no. It just needs a little -training without every experimental new paper added to it. The KISS principal. - - -#### 2023-07-30 -Added "anchors" to the slider trainer. This allows you to set a prompt that will be used as a -regularizer. You can set the network multiplier to force spread consistency at high weights - diff --git a/ui/next.config.ts b/ui/next.config.ts index 4ae5a673..8655fe00 100644 --- a/ui/next.config.ts +++ b/ui/next.config.ts @@ -1,7 +1,10 @@ import type { NextConfig } from 'next'; const nextConfig: NextConfig = { - /* config options here */ + typescript: { + // Remove this. Build fails because of route types + ignoreBuildErrors: true, + }, experimental: { serverActions: { bodySizeLimit: '100mb', diff --git a/ui/package.json b/ui/package.json index 88a938bc..2a461afc 100644 --- a/ui/package.json +++ b/ui/package.json @@ -5,9 +5,9 @@ "scripts": { "dev": "next dev --turbopack", "build": "next build", - "start": "next start", + "start": "next start --port 8675", "lint": "next lint", - "update_db": "npx prisma generate && npx prisma db push", + "update_db": "npx prisma generate ; npx prisma db push", "format": "prettier --write \"**/*.{js,jsx,ts,tsx,css,scss}\"" }, "dependencies": { diff --git a/ui/src/app/api/caption/[...imagePath]/route.ts b/ui/src/app/api/caption/[...imagePath]/route.ts index 7aa0e07f..6919aa93 100644 --- a/ui/src/app/api/caption/[...imagePath]/route.ts +++ b/ui/src/app/api/caption/[...imagePath]/route.ts @@ -1,4 +1,4 @@ -// src/app/api/img/[imagePath]/route.ts +/* eslint-disable */ import { NextRequest, NextResponse } from 'next/server'; import fs from 'fs'; import path from 'path'; diff --git a/ui/src/app/api/files/[...filePath]/route.ts b/ui/src/app/api/files/[...filePath]/route.ts index 9c9b9398..44076e40 100644 --- a/ui/src/app/api/files/[...filePath]/route.ts +++ b/ui/src/app/api/files/[...filePath]/route.ts @@ -1,3 +1,4 @@ +/* eslint-disable */ import { NextRequest, NextResponse } from 'next/server'; import fs from 'fs'; import path from 'path'; diff --git a/ui/src/app/api/img/[...imagePath]/route.ts b/ui/src/app/api/img/[...imagePath]/route.ts index 45586c44..8c28275e 100644 --- a/ui/src/app/api/img/[...imagePath]/route.ts +++ b/ui/src/app/api/img/[...imagePath]/route.ts @@ -1,4 +1,4 @@ -// src/app/api/img/[imagePath]/route.ts +/* eslint-disable */ import { NextRequest, NextResponse } from 'next/server'; import fs from 'fs'; import path from 'path'; diff --git a/ui/src/app/layout.tsx b/ui/src/app/layout.tsx index 6e379e58..292c78c4 100644 --- a/ui/src/app/layout.tsx +++ b/ui/src/app/layout.tsx @@ -5,6 +5,7 @@ import Sidebar from '@/components/Sidebar'; import { ThemeProvider } from '@/components/ThemeProvider'; import ConfirmModal from '@/components/ConfirmModal'; import SampleImageModal from '@/components/SampleImageModal'; +import { Suspense } from 'react'; const inter = Inter({ subsets: ['latin'] }); @@ -23,7 +24,10 @@ export default function RootLayout({ children }: { children: React.ReactNode })
-
{children}
+ +
+ {children} +
diff --git a/ui/src/components/FilesWidget.tsx b/ui/src/components/FilesWidget.tsx index 7bfe755e..9c4754f8 100644 --- a/ui/src/components/FilesWidget.tsx +++ b/ui/src/components/FilesWidget.tsx @@ -23,7 +23,7 @@ export default function FilesWidget({ jobID }: { jobID: string }) {
-

Model Checkpoints

+

Checkpoints

{files.length}