mirror of
https://github.com/openai/CLIP.git
synced 2026-11-05 01:37:26 +00:00
[GH-ISSUE #112] CLIP behaviour #102
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
openai/CLIP#102
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Garrettlynch on GitHub (Jun 7, 2021).
Original GitHub issue: https://github.com/openai/CLIP/issues/112
Using CLIP to VQGAN I'm noticing a behaviour that I think is originating in CLIP. Sometimes words in the parsed text are rendered as typography instead of being interpreted and rendered as a object/thing etc. e.g. input someone's name who is not well known and their name is rendered in the image as typography. Is this a bug/limitation/expected behaviour and does it have anything to do with the reported typographic attacks?
@jongwook commented on GitHub (Jun 30, 2021):
This behavior was analyzed as "multimodal neurons" in an OpenAI blog post and a distill.pub article. We don't have a method to prevent this out-of-the-box, but I can imagine doing a classifier guidance with prompt engineering to detect letters and dissuade the latent updates from that direction.
@Garrettlynch commented on GitHub (Jun 30, 2021):
I'm not looking to prevent the behaviour, just to understand what it occurring. From reading the articles you linked to am I correct that the Clip neuron I am triggering through text (the third trigger in the first figure of both articles e.g. "Responds to the text 'spider' and others") is effectively failing, not linking the text to a concept, and so rendering again as text i.e. text in - text out?
@jongwook commented on GitHub (Jul 1, 2021):
Depending on the application it could be an unintended behavior, like in the typographic attacks shown in the articles, but calling it a 'failure' might not be the best description. Human neurons have a multimodal behavior after all.
Sometimes, we want the model to have the "optical character recognition" capability, and it's how we performed well on multimodal datasets like Hateful Memes. Dall-E also has a similar feature where it can choose to render words in the text prompt:
I'd say that the ability to recognize textual information is a net-positive thing to have in a multimodal vision model.
@Garrettlynch commented on GitHub (Jul 1, 2021):
Yes I see your point when the input query has the suggestion of writing/words e.g. "a store front that has the word 'openai' written on it". My specific input has nothing like this, it's actually an affirmation similar to to what somebody would say in the first person e.g. "I am ...". Many thanks for pointing me to some details that have helped me understand what might be occurring and why - very helpful!