mirror of
https://github.com/openai/CLIP.git
synced 2026-11-05 01:37:26 +00:00
[GH-ISSUE #146] Why the results of cosine similarities between the corresponding image and text features may appear negative numbers and NaN? #128
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
openai/CLIP#128
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @lr19960813 on GitHub (Aug 30, 2021).
Original GitHub issue: https://github.com/openai/CLIP/issues/146
I input a image and some texts in one list, then use model.(image,text) to calculate their values. Follow the instructions, the values are cosine similarities between the corresponding image and text features, times 100.
They should all be positive numbers, but when I fine tuned clip and use the fine tuned clip model for calculations, some results are negative numbers and even all results are NaN. I don‘t know why this happened?
@jongwook commented on GitHub (Sep 24, 2021):
While we usually observe positive cosine similarities for typical text and image inputs, they can have negative values, because it's a cosine function.
See #40, #63, #81, and #144 for tips for avoiding
NaNs during fine-tuning. In short, converting everything to FP32 will mostly solve the problem.