openai/CLIP

Fork 0

mirror of https://github.com/openai/CLIP.git synced 2026-11-05 01:37:26 +00:00

[GH-ISSUE #371] Question / Confirmation on arange in encode_text #323

New issue

Closed

opened 2026-10-31 21:13:51 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 21:13:51 +00:00

Owner

Originally created by @AwePhD on GitHub (Jun 29, 2023).
Original GitHub issue: https://github.com/openai/CLIP/issues/371

Hello,

This is a minor question about the code, I want to be sure I do not let slip any subtleties.

In L354 of model.py, there is the final step to extract the text

# x.shape = [batch_size, n_ctx, transformer.width]
x = x[torch.arange(x.shape[0]), text.argmax(dim=-1)] @ self.text_projection

self.text_projection is the last projection to have the text features
text.argmax(dim=-1) picks the features of the EOT token.

Why there is a torch.arange(x.shape[0])? It could be x[:, text.argmax(dim=-1)], right?

Thanks for the work, code and model.

Originally created by @AwePhD on GitHub (Jun 29, 2023). Original GitHub issue: https://github.com/openai/CLIP/issues/371 Hello, This is a minor question about the code, I want to be sure I do not let slip any subtleties. In L354 of `model.py`, there is the final step to extract the text ```python # x.shape = [batch_size, n_ctx, transformer.width] x = x[torch.arange(x.shape[0]), text.argmax(dim=-1)] @ self.text_projection ``` - `self.text_projection` is the last projection to have the text features - `text.argmax(dim=-1)` picks the features of the EOT token. Why there is a `torch.arange(x.shape[0])`? It could be `x[:, text.argmax(dim=-1)]`, right? Thanks for the work, code and model.

bsakaguchi closed this issue

2026-10-31 21:13:51 +00:00

bsakaguchi commented

2026-10-31 21:13:53 +00:00

Author

Owner

@jongwook commented on GitHub (Jun 29, 2023):

We need to extract the features (of shape transformer.width) from specific different locations of the EOT token across the batch, so the shape becomes [batch_size, transformer.width].

If we did x[:, text.argmax(dim=-1)], it simply selects the features at any of the argmax indices across the batch, so the shape would become [batch_size, batch_size, transformer.width].

It could be simpler to use index_select instead and still achieve the same effect, but I found myself fewer mistakes by explicitly indexing with torch.arange().

@jongwook commented on GitHub (Jun 29, 2023): We need to extract the features (of shape `transformer.width`) from specific different locations of the EOT token across the batch, so the shape becomes `[batch_size, transformer.width]`. If we did `x[:, text.argmax(dim=-1)]`, it simply selects the features at any of the argmax indices across the batch, so the shape would become `[batch_size, batch_size, transformer.width]`. It could be simpler to use `index_select` instead and still achieve the same effect, but I found myself fewer mistakes by explicitly indexing with `torch.arange()`.

bsakaguchi commented

2026-10-31 21:13:53 +00:00

Author

Owner

@AwePhD commented on GitHub (Jul 11, 2023):

Hello,

You are absolutely right, thanks for your time to clear my confusion!

@AwePhD commented on GitHub (Jul 11, 2023): Hello, You are absolutely right, thanks for your time to clear my confusion!

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

openai/CLIP#323

No description provided.

Rows
Columns