openai/CLIP
1
0
Fork 0
mirror of https://github.com/openai/CLIP.git synced 2026-11-05 01:37:26 +00:00
Code Issues 274 Projects Packages Wiki Activity

[GH-ISSUE #189] RuntimeError: The size of tensor a (2) must match the size of tensor b (50) at non-singleton dimension 1 #164

New issue
Open
opened 2026-10-31 21:11:53 +00:00 by bsakaguchi · 6 comments
bsakaguchi commented 2026-10-31 21:11:53 +00:00
Owner
Copy link

Originally created by @RylanSchaeffer on GitHub (Nov 28, 2021).
Original GitHub issue: https://github.com/openai/CLIP/issues/189

When I load the ViT-B/32 model, like so,

clip_image_and_text_model, preprocess = clip.load("ViT-B/32", device=device)

And then construct a CIFAR100 dataloader using preprocess as the dataloader, I have input tensors of shape (batch size, 3, 32, 32). But when I try calling

clip_image_and_text_model.encode_image(input_tensor)

I get RuntimeError: The size of tensor a (2) must match the size of tensor b (50) at non-singleton dimension 1. The error is caused by line 224 in model.py:

x = x + self.positional_embedding.to(x.dtype)

At this line, x has shape (batch size, 2, 768) whereas self.positional_embedding has shape (batch size, 768).

I don't know what the correct behavior is to know whether x should not have dimension 1 or self.positional_embedding should have dimension 1 or something else entirely.

Could someone please clarify?

Originally created by @RylanSchaeffer on GitHub (Nov 28, 2021). Original GitHub issue: https://github.com/openai/CLIP/issues/189 When I load the ViT-B/32 model, like so, ``` clip_image_and_text_model, preprocess = clip.load("ViT-B/32", device=device) ``` And then construct a CIFAR100 dataloader using preprocess as the dataloader, I have input tensors of shape `(batch size, 3, 32, 32)`. But when I try calling ``` clip_image_and_text_model.encode_image(input_tensor) ``` I get `RuntimeError: The size of tensor a (2) must match the size of tensor b (50) at non-singleton dimension 1`. The error is caused by line 224 in model.py: ``` x = x + self.positional_embedding.to(x.dtype) ``` At this line, `x` has shape `(batch size, 2, 768)` whereas `self.positional_embedding` has shape `(batch size, 768)`. I don't know what the correct behavior is to know whether `x` should not have dimension 1 or `self.positional_embedding` should have dimension 1 or something else entirely. Could someone please clarify?
bsakaguchi commented 2026-10-31 21:11:54 +00:00
Author
Owner
Copy link

@RylanSchaeffer commented on GitHub (Nov 28, 2021):

The comment on line 223 suggests that x should indeed be 3 dimensional, which suggests to me that the positional embedding is wrong. Is this correct?

<!-- gh-comment-id:981153737 --> @RylanSchaeffer commented on GitHub (Nov 28, 2021): The comment on line 223 suggests that `x` should indeed be 3 dimensional, which suggests to me that the positional embedding is wrong. Is this correct?
bsakaguchi commented 2026-10-31 21:11:54 +00:00
Author
Owner
Copy link

@Evm7 commented on GitHub (Jan 13, 2022):

Hey @RylanSchaeffer , I am finding a simillar problem when changing the input size of the model. Did you finally manage to find the proper solution?

<!-- gh-comment-id:1012314288 --> @Evm7 commented on GitHub (Jan 13, 2022): Hey @RylanSchaeffer , I am finding a simillar problem when changing the input size of the model. Did you finally manage to find the proper solution?
bsakaguchi commented 2026-10-31 21:11:54 +00:00
Author
Owner
Copy link

@RylanSchaeffer commented on GitHub (Jan 13, 2022):

@Evm7 unfortunately I can't remember :(

<!-- gh-comment-id:1012423690 --> @RylanSchaeffer commented on GitHub (Jan 13, 2022): @Evm7 unfortunately I can't remember :(
bsakaguchi commented 2026-10-31 21:11:54 +00:00
Author
Owner
Copy link

@jogisuda commented on GitHub (Jan 28, 2022):

Facing exactly the same issue here

<!-- gh-comment-id:1024362369 --> @jogisuda commented on GitHub (Jan 28, 2022): Facing exactly the same issue here
bsakaguchi commented 2026-10-31 21:11:54 +00:00
Author
Owner
Copy link

@jogisuda commented on GitHub (Jan 28, 2022):

Found the solution. My problem was the size of the images: I had batches of dimension (16, 3, 32, 32) (16 images per batch, 3 channels, 32 height/width). Got it working when changed the Transforms to resize to dimension 224. So, final shape became (16, 3, 224, 224). Hope it helps!

<!-- gh-comment-id:1024427816 --> @jogisuda commented on GitHub (Jan 28, 2022): Found the solution. My problem was the size of the images: I had batches of dimension (16, 3, 32, 32) (16 images per batch, 3 channels, 32 height/width). Got it working when changed the Transforms to resize to dimension 224. So, final shape became (16, 3, 224, 224). Hope it helps!
bsakaguchi commented 2026-10-31 21:11:54 +00:00
Author
Owner
Copy link

@Dinosaurcubs commented on GitHub (Dec 25, 2023):

Found the solution. My problem was the size of the images: I had batches of dimension (16, 3, 32, 32) (16 images per batch, 3 channels, 32 height/width). Got it working when changed the Transforms to resize to dimension 224. So, final shape became (16, 3, 224, 224). Hope it helps!

in other words, the input shape must be (B, 3 , 224, 224) ?

<!-- gh-comment-id:1868831569 --> @Dinosaurcubs commented on GitHub (Dec 25, 2023): > Found the solution. My problem was the size of the images: I had batches of dimension (16, 3, 32, 32) (16 images per batch, 3 channels, 32 height/width). Got it working when changed the Transforms to resize to dimension 224. So, final shape became (16, 3, 224, 224). Hope it helps! in other words, the input shape must be (B, 3 , 224, 224) ?
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
codex/pin-github-workflow-refs-20260325-132838
jongwook/issue-396
jongwook-patch-1
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
openai/CLIP#164
No description provided.