Bing-su/adetailer
1
0
Fork 0
mirror of https://github.com/Bing-su/adetailer.git synced 2026-01-26 19:29:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev' into main

Browse Source
This commit is contained in:
Bingsu
2023-05-30 09:09:38 +09:00
parent 46a8ef2ac6 b6262c4884
commit 5ab196778b
14 changed files with 233 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
.github/ISSUE_TEMPLATE/bug_report.yaml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: Bug report
description: Create a report
title: "[Bug]: "
body:
- type: textarea
attributes:
label: Describe the bug
description: A clear and concise description of what the bug is.
- type: textarea
attributes:
label: Full console logs
description: |
The full console log of your terminal.
From `Python 3.10.*, Version: v1.*, Commit hash: *` to the end.
render: Shell
validations:
required: true
- type: textarea
attributes:
label: List of installed extensions

20
.github/ISSUE_TEMPLATE/feature_request.md vendored Normal file
View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

13
.github/workflows/stale.yml vendored Normal file
View File

@@ -0,0 +1,13 @@
name: 'Close stale issues and PRs'
on:
schedule:
- cron: '30 1 * * *'
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v8
with:
days-before-stale: 30
days-before-close: 5

3
.pre-commit-config.yaml
View File

@@ -5,6 +5,7 @@ repos:
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- id: end-of-file-fixer
- id: mixed-line-ending
- repo: https://github.com/pycqa/isort
rev: 5.12.0
@@ -12,7 +13,7 @@ repos:
- id: isort
- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: "v0.0.267"
rev: "v0.0.270"
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]

15
CHANGELOG.md
View File

@@ -1,5 +1,20 @@
# Changelog
## 2023-05-30
- v23.6.0
- 스크립트의 이름을 `After Detailer`에서 `ADetailer`로 변경
- API 사용자는 변경 필요함
- 몇몇 설정 변경
- `ad_conf` → `ad_confidence`. 0~100 사이의 int → 0.0~1.0 사이의 float
- `ad_inpaint_full_res` → `ad_inpaint_only_masked`
- `ad_inpaint_full_res_padding` → `ad_inpaint_only_masked_padding`
- mediapipe face mesh 모델 추가
- mediapipe 최소 버전 `0.10.0`
- rich traceback 제거함
- huggingface 다운로드 실패할 때 에러가 나지 않게 하고 해당 모델을 제거함
## 2023-05-26
- v23.5.19

9
README.md
View File

@@ -40,9 +40,11 @@ If you want to exclude objects in the background, try setting the min ratio to a
| Mask erosion (-) / dilation (+) | Enlarge or reduce the detected mask. | [opencv example](https://docs.opencv.org/4.7.0/db/df6/tutorial_erosion_dilatation.html) |
| Mask merge mode | `None`: Inpaint each mask<br/>`Merge`: Merge all masks and inpaint<br/>`Merge and Invert`: Merge all masks and Invert, then inpaint | |
Applied in this order: x, y offset → erosion/dilation → merge/invert.
#### Inpainting
![image](https://i.imgur.com/KbAeWar.png)
![image](https://i.imgur.com/wyWlT1n.png)
Each option corresponds to a corresponding option on the inpaint tab.
@@ -58,11 +60,12 @@ On the ControlNet tab, select a ControlNet inpaint model and set the model weigh
| --------------------- | --------------------- | ----------------------------- | ----------------------------- |
| face_yolov8n.pt | 2D / realistic face | 0.660 | 0.366 |
| face_yolov8s.pt | 2D / realistic face | 0.713 | 0.404 |
| mediapipe_face_full | realistic face | - | - |
| mediapipe_face_short | realistic face | - | - |
| hand_yolov8n.pt | 2D / realistic hand | 0.767 | 0.505 |
| person_yolov8n-seg.pt | 2D / realistic person | 0.782 (bbox)<br/>0.761 (mask) | 0.555 (bbox)<br/>0.460 (mask) |
| person_yolov8s-seg.pt | 2D / realistic person | 0.824 (bbox)<br/>0.809 (mask) | 0.605 (bbox)<br/>0.508 (mask) |
| mediapipe_face_full | realistic face | - | - |
| mediapipe_face_short | realistic face | - | - |
| mediapipe_face_mesh | realistic face | - | - |
The yolo models can be found on huggingface [Bingsu/adetailer](https://huggingface.co/Bingsu/adetailer).

2
adetailer/__init__.py
View File

@@ -4,7 +4,7 @@ from .common import PredictOutput, get_models
from .mediapipe import mediapipe_predict
from .ultralytics import ultralytics_predict
AFTER_DETAILER = "After Detailer"
AFTER_DETAILER = "ADetailer"
__all__ = [
"__version__",

2
adetailer/__version__.py
View File

@@ -1 +1 @@
__version__ = "23.5.19"
__version__ = "23.6.0"

27
adetailer/args.py
View File

@@ -13,7 +13,6 @@ from pydantic import (
PositiveInt,
confloat,
constr,
validator,
)
@@ -36,7 +35,7 @@ class ADetailerArgs(BaseModel, extra=Extra.forbid):
ad_model: str = "None"
ad_prompt: str = ""
ad_negative_prompt: str = ""
ad_conf: confloat(ge=0.0, le=1.0) = 0.3
ad_confidence: confloat(ge=0.0, le=1.0) = 0.3
ad_mask_min_ratio: confloat(ge=0.0, le=1.0) = 0.0
ad_mask_max_ratio: confloat(ge=0.0, le=1.0) = 1.0
ad_dilate_erode: int = 32
@@ -45,8 +44,8 @@ class ADetailerArgs(BaseModel, extra=Extra.forbid):
ad_mask_merge_invert: Literal["None", "Merge", "Merge and Invert"] = "None"
ad_mask_blur: NonNegativeInt = 4
ad_denoising_strength: confloat(ge=0.0, le=1.0) = 0.4
ad_inpaint_full_res: bool = True
ad_inpaint_full_res_padding: NonNegativeInt = 0
ad_inpaint_only_masked: bool = True
ad_inpaint_only_masked_padding: NonNegativeInt = 0
ad_use_inpaint_width_height: bool = False
ad_inpaint_width: PositiveInt = 512
ad_inpaint_height: PositiveInt = 512
@@ -58,17 +57,6 @@ class ADetailerArgs(BaseModel, extra=Extra.forbid):
ad_controlnet_model: constr(regex=r".*inpaint.*|^None$") = "None"
ad_controlnet_weight: confloat(ge=0.0, le=1.0) = 1.0
@validator("ad_conf", pre=True)
def check_ad_conf(cls, v: Any): # noqa: N805
if not isinstance(v, (int, float)):
try:
v = int(v)
except ValueError:
v = float(v)
if isinstance(v, int):
v /= 100.0
return v
@staticmethod
def ppop(
p: dict[str, Any],
@@ -90,7 +78,6 @@ class ADetailerArgs(BaseModel, extra=Extra.forbid):
return {}
p = {name: getattr(self, attr) for attr, name in ALL_ARGS}
p["ADetailer conf"] = int(p["ADetailer conf"] * 100)
ppop = partial(self.ppop, p)
ppop("ADetailer prompt")
@@ -100,7 +87,7 @@ class ADetailerArgs(BaseModel, extra=Extra.forbid):
ppop("ADetailer x offset", cond=0)
ppop("ADetailer y offset", cond=0)
ppop("ADetailer mask merge/invert", cond="None")
ppop("ADetailer inpaint full", ["ADetailer inpaint padding"])
ppop("ADetailer inpaint only masked", ["ADetailer inpaint padding"])
ppop(
"ADetailer use inpaint width/height",
[
@@ -148,7 +135,7 @@ _all_args = [
("ad_model", "ADetailer model"),
("ad_prompt", "ADetailer prompt"),
("ad_negative_prompt", "ADetailer negative prompt"),
("ad_conf", "ADetailer conf"),
("ad_confidence", "ADetailer confidence"),
("ad_mask_min_ratio", "ADetailer mask min ratio"),
("ad_mask_max_ratio", "ADetailer mask max ratio"),
("ad_x_offset", "ADetailer x offset"),
@@ -157,8 +144,8 @@ _all_args = [
("ad_mask_merge_invert", "ADetailer mask merge/invert"),
("ad_mask_blur", "ADetailer mask blur"),
("ad_denoising_strength", "ADetailer denoising strength"),
("ad_inpaint_full_res", "ADetailer inpaint full"),
("ad_inpaint_full_res_padding", "ADetailer inpaint padding"),
("ad_inpaint_only_masked", "ADetailer inpaint only masked"),
("ad_inpaint_only_masked_padding", "ADetailer inpaint padding"),
("ad_use_inpaint_width_height", "ADetailer use inpaint width/height"),
("ad_inpaint_width", "ADetailer inpaint width"),
("ad_inpaint_height", "ADetailer inpaint height"),

73
adetailer/common.py
View File

@@ -13,11 +13,19 @@ repo_id = "Bingsu/adetailer"
@dataclass
class PredictOutput:
bboxes: list[list[float]] = field(default_factory=list)
bboxes: list[list[int | float]] = field(default_factory=list)
masks: list[Image.Image] = field(default_factory=list)
preview: Optional[Image.Image] = None
def hf_download(file: str):
try:
path = hf_hub_download(repo_id, file)
except Exception:
path = "INVALID"
return path
def get_models(
model_dir: Union[str, Path], huggingface: bool = True
) -> OrderedDict[str, Optional[str]]:
@@ -31,29 +39,28 @@ def get_models(
else:
model_paths = []
models = OrderedDict()
if huggingface:
models = OrderedDict(
models.update(
{
"face_yolov8n.pt": hf_hub_download(repo_id, "face_yolov8n.pt"),
"face_yolov8s.pt": hf_hub_download(repo_id, "face_yolov8s.pt"),
"mediapipe_face_full": None,
"mediapipe_face_short": None,
"hand_yolov8n.pt": hf_hub_download(repo_id, "hand_yolov8n.pt"),
"person_yolov8n-seg.pt": hf_hub_download(
repo_id, "person_yolov8n-seg.pt"
),
"person_yolov8s-seg.pt": hf_hub_download(
repo_id, "person_yolov8s-seg.pt"
),
}
)
else:
models = OrderedDict(
{
"mediapipe_face_full": None,
"mediapipe_face_short": None,
"face_yolov8n.pt": hf_download("face_yolov8n.pt"),
"face_yolov8s.pt": hf_download("face_yolov8s.pt"),
"hand_yolov8n.pt": hf_download("hand_yolov8n.pt"),
"person_yolov8n-seg.pt": hf_download("person_yolov8n-seg.pt"),
"person_yolov8s-seg.pt": hf_download("person_yolov8s-seg.pt"),
}
)
models.update(
{
"mediapipe_face_full": None,
"mediapipe_face_short": None,
"mediapipe_face_mesh": None,
}
)
invalid_keys = [k for k, v in models.items() if v == "INVALID"]
for key in invalid_keys:
models.pop(key)
for path in model_paths:
if path.name in models:
@@ -88,3 +95,29 @@ def create_mask_from_bbox(
mask_draw.rectangle(bbox, fill=255)
masks.append(mask)
return masks
def create_bbox_from_mask(
masks: list[Image.Image], shape: tuple[int, int]
) -> list[list[int]]:
"""
Parameters
----------
masks: list[Image.Image]
A list of masks
shape: tuple[int, int]
shape of the image (width, height)
Returns
-------
bboxes: list[list[float]]
A list of bounding boxes
"""
bboxes = []
for mask in masks:
mask = mask.resize(shape)
bbox = mask.getbbox()
if bbox is not None:
bboxes.append(list(bbox))
return bboxes

77
adetailer/mediapipe.py
View File

@@ -1,19 +1,33 @@
from __future__ import annotations
from functools import partial
import numpy as np
from PIL import Image
from PIL import Image, ImageDraw
from adetailer import PredictOutput
from adetailer.common import create_mask_from_bbox
from adetailer.common import create_bbox_from_mask, create_mask_from_bbox
def mediapipe_predict(
model_type: int | str, image: Image.Image, confidence: float = 0.3
model_type: str, image: Image.Image, confidence: float = 0.3
) -> PredictOutput:
mapping = {
"mediapipe_face_short": partial(mediapipe_face_detection, model_type=0),
"mediapipe_face_full": partial(mediapipe_face_detection, model_type=1),
"mediapipe_face_mesh": mediapipe_face_mesh,
}
if model_type in mapping:
func = mapping[model_type]
return func(image, confidence)
raise RuntimeError(f"[-] ADetailer: Invalid mediapipe model type: {model_type}")
def mediapipe_face_detection(
model_type: int, image: Image.Image, confidence: float = 0.3
) -> PredictOutput:
import mediapipe as mp
if isinstance(model_type, str):
model_type = mediapipe_model_name_to_type(model_type)
img_width, img_height = image.size
mp_face_detection = mp.solutions.face_detection
@@ -51,12 +65,47 @@ def mediapipe_predict(
return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)
def mediapipe_model_name_to_type(name: str) -> int:
name = name.lower()
mapping = {
"mediapipe_face_short": 0,
"mediapipe_face_full": 1,
}
if name not in mapping:
raise ValueError(f"[-] ADetailer: Invalid model name: {name}")
return mapping[name]
def mediapipe_face_mesh(image: Image.Image, confidence: float = 0.3) -> PredictOutput:
import mediapipe as mp
from scipy.spatial import ConvexHull
mp_face_mesh = mp.solutions.face_mesh
draw_util = mp.solutions.drawing_utils
drawing_styles = mp.solutions.drawing_styles
w, h = image.size
with mp_face_mesh.FaceMesh(
static_image_mode=True, max_num_faces=20, min_detection_confidence=confidence
) as face_mesh:
arr = np.array(image)
pred = face_mesh.process(arr)
if pred.multi_face_landmarks is None:
return PredictOutput()
preview = arr.copy()
masks = []
for landmarks in pred.multi_face_landmarks:
draw_util.draw_landmarks(
image=preview,
landmark_list=landmarks,
connections=mp_face_mesh.FACEMESH_TESSELATION,
landmark_drawing_spec=None,
connection_drawing_spec=drawing_styles.get_default_face_mesh_tesselation_style(),
)
points = np.array([(land.x * w, land.y * h) for land in landmarks.landmark])
hull = ConvexHull(points)
vertices = hull.vertices
outline = list(zip(points[vertices, 0], points[vertices, 1]))
mask = Image.new("L", image.size, "black")
draw = ImageDraw.Draw(mask)
draw.polygon(outline, fill="white")
masks.append(mask)
bboxes = create_bbox_from_mask(masks, image.size)
preview = Image.fromarray(preview)
return PredictOutput(bboxes=bboxes, masks=masks, preview=preview)

35
adetailer/ui.py
View File

@@ -1,6 +1,7 @@
from __future__ import annotations
from functools import partial
from types import SimpleNamespace
from typing import Any
import gradio as gr
@@ -10,7 +11,7 @@ from adetailer.args import AD_ENABLE, ALL_ARGS, MASK_MERGE_INVERT
from controlnet_ext import controlnet_exists, get_cn_inpaint_models
class Widgets:
class Widgets(SimpleNamespace):
def tolist(self):
return [getattr(self, attr) for attr in ALL_ARGS.attrs]
@@ -200,14 +201,14 @@ def detection(w: Widgets, n: int, is_img2img: bool):
with gr.Row():
with gr.Column():
w.ad_conf = gr.Slider(
label="Detection model confidence threshold %" + suffix(n),
minimum=0,
maximum=100,
step=1,
value=30,
w.ad_confidence = gr.Slider(
label="Detection model confidence threshold" + suffix(n),
minimum=0.0,
maximum=1.0,
step=0.01,
value=0.3,
visible=True,
elem_id=eid("ad_conf"),
elem_id=eid("ad_confidence"),
)
with gr.Column(variant="compact"):
@@ -262,7 +263,7 @@ def mask_preprocessing(w: Widgets, n: int, is_img2img: bool):
minimum=-128,
maximum=128,
step=4,
value=32,
value=4,
visible=True,
elem_id=eid("ad_dilate_erode"),
)
@@ -303,26 +304,26 @@ def inpainting(w: Widgets, n: int, is_img2img: bool):
with gr.Row():
with gr.Column(variant="compact"):
w.ad_inpaint_full_res = gr.Checkbox(
label="Inpaint at full resolution " + suffix(n),
w.ad_inpaint_only_masked = gr.Checkbox(
label="Inpaint only masked" + suffix(n),
value=True,
visible=True,
elem_id=eid("ad_inpaint_full_res"),
)
w.ad_inpaint_full_res_padding = gr.Slider(
label="Inpaint at full resolution padding, pixels " + suffix(n),
w.ad_inpaint_only_masked_padding = gr.Slider(
label="Inpaint only masked padding, pixels" + suffix(n),
minimum=0,
maximum=256,
step=4,
value=0,
value=32,
visible=True,
elem_id=eid("ad_inpaint_full_res_padding"),
)
w.ad_inpaint_full_res.change(
w.ad_inpaint_only_masked.change(
gr_interactive,
inputs=w.ad_inpaint_full_res,
outputs=w.ad_inpaint_full_res_padding,
inputs=w.ad_inpaint_only_masked,
outputs=w.ad_inpaint_only_masked_padding,
queue=False,
)

7
install.py
View File

@@ -1,14 +1,15 @@
from __future__ import annotations
import importlib.util
import subprocess
import sys
from importlib.metadata import version # python >= 3.8
from typing import Optional
from packaging.version import parse
def is_installed(
package: str, min_version: Optional[str] = None, max_version: Optional[str] = None
package: str, min_version: str | None = None, max_version: str | None = None
):
try:
spec = importlib.util.find_spec(package)
@@ -44,7 +45,7 @@ def install():
deps = [
# requirements
("ultralytics", "8.0.97", None),
("mediapipe", "0.9.3.0", None),
("mediapipe", "0.10.0", None),
("huggingface_hub", None, None),
("pydantic", None, None),
# mediapipe

18
scripts/!adetailer.py
View File

@@ -5,7 +5,7 @@ import platform
import re
import sys
import traceback
from contextlib import contextmanager
from contextlib import contextmanager, suppress
from copy import copy, deepcopy
from pathlib import Path
from textwrap import dedent
@@ -41,13 +41,9 @@ from sd_webui.processing import (
)
from sd_webui.shared import cmd_opts, opts, state
try:
with suppress(ImportError):
from rich import print
from rich.traceback import install
install(show_locals=True)
except Exception:
pass
no_huggingface = getattr(cmd_opts, "ad_no_huggingface", False)
adetailer_dir = Path(models_path, "adetailer")
@@ -347,8 +343,8 @@ class AfterDetailerScript(scripts.Script):
mask=None,
mask_blur=args.ad_mask_blur,
inpainting_fill=1,
inpaint_full_res=args.ad_inpaint_full_res,
inpaint_full_res_padding=args.ad_inpaint_full_res_padding,
inpaint_full_res=args.ad_inpaint_only_masked,
inpaint_full_res_padding=args.ad_inpaint_only_masked_padding,
inpainting_mask_invert=0,
sd_model=p.sd_model,
outpath_samples=p.outpath_samples,
@@ -429,7 +425,7 @@ class AfterDetailerScript(scripts.Script):
def i2i_prompts_replace(
self, i2i, prompts: list[str], negative_prompts: list[str], j: int
):
) -> None:
i1 = min(j, len(prompts) - 1)
i2 = min(j, len(negative_prompts) - 1)
prompt = prompts[i1]
@@ -437,7 +433,7 @@ class AfterDetailerScript(scripts.Script):
i2i.prompt = prompt
i2i.negative_prompt = negative_prompt
def is_need_call_process(self, p):
def is_need_call_process(self, p) -> bool:
i = p._idx
n_iter = p.iteration
bs = p.batch_size
@@ -483,7 +479,7 @@ class AfterDetailerScript(scripts.Script):
kwargs["device"] = self.ultralytics_device
with change_torch_load():
pred = predictor(ad_model, pp.image, args.ad_conf, **kwargs)
pred = predictor(ad_model, pp.image, args.ad_confidence, **kwargs)
masks = self.pred_preprocessing(pred, args)