mirror of
https://github.com/openai/CLIP.git
synced 2026-11-05 01:37:26 +00:00
[GH-ISSUE #242] How does CLIP think without the prompts? #208
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
openai/CLIP#208
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @tanveer-sayyed on GitHub (May 4, 2022).
Original GitHub issue: https://github.com/openai/CLIP/issues/242
Hi,
Thank you for this amazing contribution!
The central question that I want to raise is - without entering any prompts, what all words/phrases is CLIP thinking.
So, I want to know if its possible if this model is able to give out all (text) outputs for a given image,
and not just the ones queried by prompts. That is for an image the set of "all" words/phrases that the
model has found (in decreasing order).
If its possible, how do I begin? Need some help here.
@ProGamerGov commented on GitHub (May 7, 2022):
It sounds like you're attempting to maximize the image embeddings by searching text embeddings.
Researchers did that here: https://distill.pub/2021/multimodal-neurons/
But people were running into issues trying to figure out how to reproduce it: https://github.com/openai/CLIP-featurevis/issues/2