mirror of
https://github.com/openai/CLIP.git
synced 2026-11-05 01:37:26 +00:00
[GH-ISSUE #389] Clip's capablity of detecting scene or background information #334
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
openai/CLIP#334
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Seeker98 on GitHub (Sep 21, 2023).
Original GitHub issue: https://github.com/openai/CLIP/issues/389
how is clip performing on global information detection? For example, finding whether an image is noisy-corrupted, downsample-d or hazy, and furthermore, choosing the right corruption parameters like noise std? I tried images with different types of noises like gaussian poisson or gamma, and other corruptions like downsampling or hazy, and tokens like [gaussian noise with std=25, gaussian noise with std=50], [noisy, hazy], but the inference result is not well. Am i missing any key parts on my way of testing?
@mgupta70 commented on GitHub (Jan 31, 2024):
CLIP is not for text generation.
CLIP needs text from the user as input to create its embeddings, which matches with the image and you can get a cosine similarity score.
CLIP is not good for fine-grained classification (as mentioned in the paper)
CLIP is trained on internet data, which may focus more on everyday objects. For example, if an image of a car on the internet is noisy/hazy, a high chance that the text description over the internet still mentions 'car' and not about 'noise'.