mirror of
https://github.com/pharmapsychotic/clip-interrogator-ext.git
synced 2026-11-05 01:27:20 +00:00
[GH-ISSUE #1] Error loading script #3
Labels
No labels
enhancement
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
pharmapsychotic/clip-interrogator-ext#3
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Woisek on GitHub (Feb 17, 2023).
Original GitHub issue: https://github.com/pharmapsychotic/clip-interrogator-ext/issues/1
This extension is still not working, please fix it.
Restarting UI...
Error loading script: clip_interrogator_ext.py
Traceback (most recent call last):
File "I:\Super SD 2.0\stable-diffusion-webui\modules\scripts.py", line 229, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "I:\Super SD 2.0\stable-diffusion-webui\modules\script_loading.py", line 11, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "I:\Super SD 2.0\stable-diffusion-webui\extensions\clip-interrogator-ext\scripts\clip_interrogator_ext.py", line 3, in
import clip_interrogator
ModuleNotFoundError: No module named 'clip_interrogator'
@pharmapsychotic commented on GitHub (Feb 17, 2023):
Sorry have been really busy at work so didn't get much time to investigate install issues more. Would love if anyone else is able to sort out why the dependency clip_interrogator library doesn't get installed correctly on some machines when the extension is installed.
You can try activating the auto1111 venv and manually installing with
pip install clip-interrogatorand see if that works or also has issues@vcsab05 commented on GitHub (Feb 17, 2023):
I simply paste this line in windows cmd after I install the extension, and it works for me:
pip install clip-interrogator==0.4.4
(I don't know what I'm doing but hope it helps :) )
@Woisek commented on GitHub (Feb 17, 2023):
@pharmapsychotic
For that, I get additional errors:
Building wheels for collected packages: pycocotools
Building wheel for pycocotools (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for pycocotools (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [16 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-310
creating build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools\coco.py -> build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools\cocoeval.py -> build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools\mask.py -> build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools_init_.py -> build\lib.win-amd64-cpython-310\pycocotools
running build_ext
cythoning pycocotools/_mask.pyx to pycocotools_mask.c
T:\systemp\pip-build-env-s_6ftc_m\overlay\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: T:\systemp\pip-install-wkr5_0r8\pycocotools_1048f4241e9f4fd1bfdc6a3951b02b98\pycocotools_mask.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
building 'pycocotools._mask' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pycocotools
Failed to build pycocotools
ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects
[notice] A new release of pip available: 22.3.1 -> 23.0
[notice] To update, run: python.exe -m pip install --upgrade pip
I:\Super SD 2.0\stable-diffusion-webui\extensions\clip-interrogator-ext>
What do I have to do now?
@pharmapsychotic commented on GitHub (Feb 17, 2023):
ohhh thank you that explains why my machines have no problems (I have VC++ installed on them). I think this pycocotools came up in some other issue on CLIP Interrogator. I'll hunt down what's picking up that library and see if it's really necessary.
thanks for posting this error!
@Woisek commented on GitHub (Feb 17, 2023):
OK, thanks for your effort to solve this. :)
@pharmapsychotic commented on GitHub (Feb 18, 2023):
@Woisek can you try installing the extension again now? (I updated the dependencies)
@Woisek commented on GitHub (Feb 18, 2023):
@pharmapsychotic
Thanks for the update.
The extension loads now, but unfortunately a new error occurs:
To create a public link, set
share=Trueinlaunch().Loading CLIP Interrogator 0.4.4...
Loading BLIP model...
load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_large_caption.pth
Loading CLIP model...
Loaded CLIP model and data in 5.91 seconds.
100%|█████████████████████████████████████████████████████████████████████████████████| 55/55 [00:00<00:00, 223.24it/s]
Flavor chain: 0%| | 0/32 [00:00<?, ?it/s]
Traceback (most recent call last):
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 337, in run_predict
output = await app.get_blocks().process_api(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1015, in process_api
result = await self.call_function(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 833, in call_function
prediction = await anyio.to_thread.run_sync(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "I:\Super SD 2.0\stable-diffusion-webui\extensions\clip-interrogator-ext\scripts\clip_interrogator_ext.py", line 59, in image_to_prompt
return ci.interrogate(image)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\clip_interrogator\clip_interrogator.py", line 275, in interrogate
best_prompt = self.chain(image_features, flaves, best_prompt, best_sim, min_count=min_flavors, max_count=max_flavors, desc="Flavor chain")
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\clip_interrogator\clip_interrogator.py", line 190, in chain
best = self.rank_top(image_features, [f"{curr_prompt}, {f}" for f in phrases], reverse=reverse)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\clip_interrogator\clip_interrogator.py", line 285, in rank_top
text_features = self.clip_model.encode_text(text_tokens)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\model.py", line 192, in encode_text
x = self.transformer(x, attn_mask=self.attn_mask)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\transformer.py", line 230, in forward
x = r(x, attn_mask=attn_mask)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\transformer.py", line 154, in forward
x = x + self.ls_1(self.attention(self.ln_1(x), attn_mask=attn_mask))
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\transformer.py", line 151, in attention
return self.attn(x, x, x, need_weights=False, attn_mask=attn_mask)[0]
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\activation.py", line 1153, in forward
attn_output, attn_output_weights = F.multi_head_attention_forward(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 5066, in multi_head_attention_forward
q, k, v = _in_projection_packed(query, key, value, in_proj_weight, in_proj_bias)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 4745, in _in_projection_packed
return linear(q, w, b).chunk(3, dim=-1)
RuntimeError: CUDA out of memory. Tried to allocate 690.00 MiB (GPU 0; 8.00 GiB total capacity; 6.41 GiB already allocated; 0 bytes free; 6.72 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
This is after a complete re-start of WebUI and done nothing else but useing the Interrogater. Very strange, this "CUDA out of memory" ...
@pharmapsychotic commented on GitHub (Feb 18, 2023):
Oooh that's great it installs now 🎉
How much VRAM does your GPU have? I will have to do some optimizations for lower VRAM GPUs.
@Woisek commented on GitHub (Feb 18, 2023):
Unfortunately only 8GB. :/
But til now, I could do very much with it, except Dreambooth training. But all other things are working (Embedding, LoRA, rendering images higher than 512x512). :)
@pharmapsychotic commented on GitHub (Feb 18, 2023):
Ok! My brother has a 8GB card so I'll test with that
@Woisek commented on GitHub (Feb 18, 2023):
Wonderful! Looking foreward to use it. :)
@pharmapsychotic commented on GitHub (Feb 19, 2023):
@Woisek okay hopefully works on 8GB cards now! is working on a GTX 1070 8GB with ViT-L at least
on about tab after you update and fully restart web ui you should see
CLIP Interrogator version: 0.5.1
Extension version: 0.0.3
@Woisek commented on GitHub (Feb 19, 2023):
@pharmapsychotic
Hey, thanks for this quick work.
It seems it works now, but here is a little "addon error" for you to check (Error loading cached table ...). :)
Loading CLIP Interrogator 0.5.1...
detected < 12GB VRAM, using low VRAM mode
Loading BLIP model...
load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_caption_capfilt_large.pth
Loading CLIP model...
ViT-L-14_openai_artists.safetensors: 100%|████████████████████████████████████████| 16.2M/16.2M [00:05<00:00, 2.94MB/s]
ViT-L-14_openai_flavors.safetensors: 100%|██████████████████████████████████████████| 155M/155M [00:46<00:00, 3.36MB/s]
ViT-L-14_openai_mediums.safetensors: 100%|███████████████████████████████████████████| 146k/146k [00:00<00:00, 692kB/s]
ViT-L-14_openai_movements.safetensors: 100%|████████████████████████████████████████| 307k/307k [00:00<00:00, 1.13MB/s]
ViT-L-14_openai_negative.safetensors: 100%|████████████████████████████████████████| 63.2k/63.2k [00:00<00:00, 438kB/s]
ViT-L-14_openai_trendings.safetensors: 100%|█████████████████████████████████████████| 111k/111k [00:00<00:00, 650kB/s]
Error loading cached table artists: invalid load key, 'E'.
Error loading cached table flavors: invalid load key, 'E'.
Error loading cached table mediums: invalid load key, 'E'.
Error loading cached table movements: invalid load key, 'E'.
Error loading cached table trendings: invalid load key, 'E'.
Loaded CLIP model and data in 64.62 seconds.
100%|███████████████████████████████████████████████████████████████████████████████| 110/110 [00:00<00:00, 416.20it/s]
Flavor chain: 34%|███████████████████████▍ | 11/32 [00:12<00:24, 1.15s/it]
100%|███████████████████████████████████████████████████████████████████████████████| 110/110 [00:00<00:00, 410.02it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 380.33it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 99/99 [00:00<00:00, 431.53it/s]
Thanks again!
@pharmapsychotic commented on GitHub (Feb 20, 2023):
Ok I dropped support for the .pkl format which I believe is where that error was coming from and did more VRAM improvements. Inside the SD Web UI you should be able to go to extension tab, check for updates, apply and restart UI and stop and restart the web ui and should be all updated and see
on the About tab
@Woisek commented on GitHub (Feb 20, 2023):
Yes, wonderful! It looks like it's now working. Thanks! :)
The only thing I'm missing, is a "copy to txt2img/img2img", please ...
Oh, and is it possible to get a brief descriptions of the different clip models, what they do, the differences betweeen them and when we should use what model? Would be awesome. :D
Thanks again!
@dtlnor commented on GitHub (Feb 22, 2023):
same here (shows bclip-ci can't build)
@tsound97 commented on GitHub (Feb 23, 2023):
@dtlnor Same error, I tried to download the blip-ci-0.0.2.tar.gz and changed the Unicode in it, which successfully completed the installation of blip-ci, but there are still problems when running the script.
PS：催更本地化
@dtlnor commented on GitHub (Feb 23, 2023):
try this pr
https://github.com/pharmapsychotic/BLIP/pull/6
it fix
@mudashi33 commented on GitHub (Feb 23, 2023):
Solve the problem of failing to install the clip-interrogator-ext plug-in, prompting 'gbk' codec can't decode byte...... (or prompting No module named 'clip_interrogator' when running sd webui):
long_description=open('README.md', encoding='utf-8').read(),pip install blip-ci-0.0.2.tar.gzcommand to install the blip-ci modulepip install clip-interrogator==0.5.4command to install the clip-interrogator module@andyleeyuan commented on GitHub (Feb 23, 2023):
I installed blip-ci-0.0.2.tar.gz and clip-interrogator 0.4.4
when start webui got this error
I use conda at windows for venv
@pharmapsychotic commented on GitHub (Mar 4, 2023):
I believe all the install issues are sorted out by now.