[GH-ISSUE #301] why using np.arange(n) rather than one-hot as labels? #260

New issue

Closed

opened 2026-10-31 21:13:05 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 21:13:05 +00:00

Owner

Originally created by @leviome on GitHub (Nov 10, 2022).
Original GitHub issue: https://github.com/openai/CLIP/issues/301

Thanks for sharing the nice work. But I didn't fully understand the labels usage:

labels = np.arange(n) # (0, 1, 2, 3, 4, ...)
loss_i = cross_entropy_loss(logits, labels, axis=0)
loss_t = cross_entropy_loss(logits, labels, axis=1)
loss = (loss_i + loss_t)/2

the logits is the cosine similarity matrix for n image-text pairs. But cross entropy loss will make the rows/cols close to [0, 1, 2, 3, ...] (np.arange(n)), leading to logits[0][0]=0, equally the similarity of image[0] and text[0] is getting to 0.
But our goal is to enlarge the cosine similarity of image[0] and text[0].

So, could some awesome guys help me understand why using np.arange(n) rather than using one-hot as labels?

Originally created by @leviome on GitHub (Nov 10, 2022). Original GitHub issue: https://github.com/openai/CLIP/issues/301 Thanks for sharing the nice work. But I didn't fully understand the labels usage: ```python labels = np.arange(n) # (0, 1, 2, 3, 4, ...) loss_i = cross_entropy_loss(logits, labels, axis=0) loss_t = cross_entropy_loss(logits, labels, axis=1) loss = (loss_i + loss_t)/2 ``` the `logits` is the cosine similarity matrix for `n` image-text pairs. But cross entropy loss will make the rows/cols close to [0, 1, 2, 3, ...] (`np.arange(n)`), leading to `logits[0][0]=0`, equally the similarity of image[0] and text[0] is getting to 0. But our goal is to enlarge the cosine similarity of image[0] and text[0]. So, could some awesome guys help me understand why using np.arange(n) rather than using one-hot as labels?

bsakaguchi closed this issue

2026-10-31 21:13:05 +00:00

bsakaguchi commented

2026-10-31 21:13:06 +00:00

Author

Owner

@leviome commented on GitHub (Dec 5, 2022):

Ok! I worked it out

@leviome commented on GitHub (Dec 5, 2022): Ok! I worked it out

bsakaguchi commented

2026-10-31 21:13:06 +00:00

Author

Owner

@yeknafar commented on GitHub (Feb 18, 2023):

can you expalin this ?

@yeknafar commented on GitHub (Feb 18, 2023): can you expalin this ?

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

openai/CLIP#260

No description provided.

Rows
Columns