[GH-ISSUE #108] Embedding suggestions for embeddings loaded in other extensions #79

New issue

Closed

opened 2026-10-30 21:18:45 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-30 21:18:45 +00:00

Owner

Originally created by @ctwrs on GitHub (Jan 8, 2023).
Original GitHub issue: https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/issues/108

This should already work with how searching for embeddings is implemented (and after the fix for #100) but it doesn't really work yet for Umi. Either I'm a dumdum and can add the embedding dir later or we can get another callback for reacting to this.

Auto thread with more details: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/6539

Writing here to let you know. We don't yet ship any embeddings but we plan to in a week or two so no pressure :).

Originally created by @ctwrs on GitHub (Jan 8, 2023). Original GitHub issue: https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/issues/108 This should already work with how searching for embeddings is implemented (and after the fix for #100) but it doesn't really work yet for Umi. Either I'm a dumdum and can add the embedding dir later or we can get another callback for reacting to this. Auto thread with more details: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/6539 Writing here to let you know. We don't yet ship any embeddings but we plan to in a week or two so no pressure :).

bsakaguchi closed this issue

2026-10-30 21:18:45 +00:00

bsakaguchi commented

2026-10-30 21:18:46 +00:00

Author

Owner

@DominikDoom commented on GitHub (Jan 20, 2023):

Sorry for the late answer, I had a look at it and believe it might be fixable on your end. But I'm not sure since I'm unfamiliar with the exact way add_embedding_dir behaves for extensions (or what its intended use looks like) and didn't have time to actually try it out.

Like you said, it should already work with the callback method, since the model loads after the extension scripts are first run.
I think the issue might be that you create a new instance of EmbeddingDatabase() in the UMI script:
embedding_db = modules.textual_inversion.textual_inversion.EmbeddingDatabase()
Maybe it would work if you instead added the dir to the internal instance used by the webui (sd_hijack.model_hijack.embedding_db)?

To be honest I'm not sure if your method even loads the embeddings at all or just prints in the console that they were validated, since the actual embedding db run through txt2img has no reference to your script-owned instance:

embedding, embedding_length_in_tokens = self.hijack.embedding_db.find_embedding_at_position(tokens, position)

    def find_embedding_at_position(self, tokens, offset):
        token = tokens[offset]
        possible_matches = self.ids_lookup.get(token, None)

As you can see in this line from sd_hijack_clip.py and textual_inversion.py, only the model_hijack embedding db actually ever gets its ID lookup table of registered embeddings referenced. And since you don't further use or wire your own instance in your process function, I don't think that info ever makes its way to the sd hijack / processing (feel free to correct me if I'm wrong).

But I don't know if there's an easy way to check when the internal embedding db is first created. In my first experiments, before the model loaded callback was moved after the embedding load, I used a while loop and checked if the sd_hijack.model_hijack.embedding_db variable was still None. That worked for waiting for the instance to be created at least. Depending on when the def __init__(self): of your script runs it could also be late enough that it already exists, but I don't think that's reliable on every system. Don't know if there's a cleaner way or callback for that.

If you can add your folder to the internal db before script_callbacks.on_model_loaded() is triggered in sd_models.py, it should work out of the box.

@DominikDoom commented on GitHub (Jan 20, 2023): Sorry for the late answer, I had a look at it and believe it might be fixable on your end. But I'm not sure since I'm unfamiliar with the exact way add_embedding_dir behaves for extensions (or what its intended use looks like) and didn't have time to actually try it out. Like you said, it should already work with the callback method, since the model loads after the extension scripts are first run. I think the issue might be that you create a new instance of EmbeddingDatabase() in the UMI script: `embedding_db = modules.textual_inversion.textual_inversion.EmbeddingDatabase()` Maybe it would work if you instead added the dir to the internal instance used by the webui (`sd_hijack.model_hijack.embedding_db`)? To be honest I'm not sure if your method even loads the embeddings at all or just prints in the console that they were validated, since the actual embedding db run through txt2img has no reference to your script-owned instance: ```python embedding, embedding_length_in_tokens = self.hijack.embedding_db.find_embedding_at_position(tokens, position) ``` ```python def find_embedding_at_position(self, tokens, offset): token = tokens[offset] possible_matches = self.ids_lookup.get(token, None) ``` As you can see in this line from sd_hijack_clip.py and textual_inversion.py, only the model_hijack embedding db actually ever gets its ID lookup table of registered embeddings referenced. And since you don't further use or wire your own instance in your process function, I don't think that info ever makes its way to the sd hijack / processing (feel free to correct me if I'm wrong). But I don't know if there's an easy way to check when the internal embedding db is first created. In my first experiments, before the model loaded callback was moved after the embedding load, I used a while loop and checked if the sd_hijack.model_hijack.embedding_db variable was still None. That worked for waiting for the instance to be created at least. Depending on when the `def __init__(self):` of your script runs it could also be late enough that it already exists, but I don't think that's reliable on every system. Don't know if there's a cleaner way or callback for that. If you can add your folder to the internal db before `script_callbacks.on_model_loaded()` is triggered in sd_models.py, it should work out of the box.

bsakaguchi commented

2026-10-30 21:18:46 +00:00

Author

Owner

@ctwrs commented on GitHub (Jan 20, 2023):

No problemo. We got it fixed a couple days ago by adding the dir outside of the init function. Maleant to close this issue but forgot 😅.

-------- Original Message --------
On Jan 20, 2023, 3:01 PM, DominikDoom wrote:

Sorry for the late answer, I had a look at it and believe it might be fixable on your end. But I'm not sure since I'm unfamiliar with the exact way add_embedding_dir behaves for extensions (or what its intended use is) and didn't have time to actually try it out.

Like you said, it should already work with the callback method, since the model loads after the extension scripts are first run.
I think the issue might be that you create a new instance of EmbeddingDatabase() in the UMI script:
embedding_db = modules.textual_inversion.textual_inversion.EmbeddingDatabase()
Maybe it would work if you instead added the dir to the internal instance used by the webui (sd_hijack.model_hijack.embedding_db)?

To be honest I'm not sure if your method even loads the embeddings at all or just prints in the console that they were validated, since the actual embedding db run through txt2img has no reference to your script-owned instance:

embedding

,

embedding_length_in_tokens

=

self

.

hijack

.

embedding_db

.

find_embedding_at_position

(

tokens

,

position

)

def

find_embedding_at_position

(

self

,

tokens

,

offset

):

token

=

tokens

[

offset

]

possible_matches

=

self

.

ids_lookup

.

get

(

token

,

None

)

As you can see in this line from sd_hijack_clip.py and textual_inversion.py, only the model_hijack embedding db actually ever gets its ID lookup table of registered embeddings referenced. And since you don't further use or wire your own instance in your process function, I don't think that info ever makes its way to the processing (feel free to correct me if I'm wrong).

But I don't know if there's an easy way to check when the internal embedding db is first created. In my first experiments, before the model loaded callback was moved after the embedding load, I used a while loop and checked if the sd_hijack.model_hijack.embedding_db variable was still None. That worked for waiting for the instance to be created at least. Depending on when the def init(self): of your script runs it could also be late enough that it already exists, but I don't think that's reliable on every system. Don't know if there's a cleaner way or callback for that.

If you can add your folder to the internal db before script_callbacks.on_model_loaded() is triggered in sd_models.py, it should work out of the box.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

@ctwrs commented on GitHub (Jan 20, 2023): No problemo. We got it fixed a couple days ago by adding the dir outside of the init function. Maleant to close this issue but forgot 😅. -------- Original Message -------- On Jan 20, 2023, 3:01 PM, DominikDoom wrote: > Sorry for the late answer, I had a look at it and believe it might be fixable on your end. But I'm not sure since I'm unfamiliar with the exact way add_embedding_dir behaves for extensions (or what its intended use is) and didn't have time to actually try it out. > > Like you said, it should already work with the callback method, since the model loads after the extension scripts are first run. > I think the issue might be that you create a new instance of EmbeddingDatabase() in the UMI script: > embedding_db = modules.textual_inversion.textual_inversion.EmbeddingDatabase() > Maybe it would work if you instead added the dir to the internal instance used by the webui (sd_hijack.model_hijack.embedding_db)? > > To be honest I'm not sure if your method even loads the embeddings at all or just prints in the console that they were validated, since the actual embedding db run through txt2img has no reference to your script-owned instance: > > embedding > > , > > embedding_length_in_tokens > > = > > self > > . > > hijack > > . > > embedding_db > > . > > find_embedding_at_position > > ( > > tokens > > , > > position > > ) > > def > > find_embedding_at_position > > ( > > self > > , > > tokens > > , > > offset > > ): > > token > > = > > tokens > > [ > > offset > > ] > > possible_matches > > = > > self > > . > > ids_lookup > > . > > get > > ( > > token > > , > > None > > ) > > As you can see in this line from sd_hijack_clip.py and textual_inversion.py, only the model_hijack embedding db actually ever gets its ID lookup table of registered embeddings referenced. And since you don't further use or wire your own instance in your process function, I don't think that info ever makes its way to the processing (feel free to correct me if I'm wrong). > > But I don't know if there's an easy way to check when the internal embedding db is first created. In my first experiments, before the model loaded callback was moved after the embedding load, I used a while loop and checked if the sd_hijack.model_hijack.embedding_db variable was still None. That worked for waiting for the instance to be created at least. Depending on when the def __init__(self): of your script runs it could also be late enough that it already exists, but I don't think that's reliable on every system. Don't know if there's a cleaner way or callback for that. > > If you can add your folder to the internal db before script_callbacks.on_model_loaded() is triggered in sd_models.py, it should work out of the box. > > — > Reply to this email directly, [view it on GitHub](https://github.com/DominikDoom/a1111-sd-webui-tagcomplete/issues/108#issuecomment-1398436430), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/A332QNKQMKHLEE5HG2MD2P3WTKLE7ANCNFSM6AAAAAATU3EUZQ). > You are receiving this because you authored the thread.Message ID: ***@***.***>