openai/CLIP

Fork 0

mirror of https://github.com/openai/CLIP.git synced 2026-11-05 01:37:26 +00:00

[GH-ISSUE #92] Half precision error on forward pass of RN50 and RN101 models #84

New issue

Closed

opened 2026-10-31 21:10:55 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-31 21:10:55 +00:00

Owner

Originally created by @spetryk on GitHub (Apr 10, 2021).
Original GitHub issue: https://github.com/openai/CLIP/issues/92

Hi, thanks for putting together a great repo. It has worked smoothly, except for the following:

When running the line logits_per_image, logits_per_text = model(image, text) with the RN50 or RN101 models, I run into the following error:

RuntimeError: default_program(57): error: identifier "prim_constantchunk_flat__1" is undefined

default_program(58): error: no operator "=" matches these operands
            operand types are: half = float

2 errors detected in the compilation of "default_program".

nvrtc compilation failed:

And then there's a bunch of C-looking code from the error message that I can post if needed.

A sample of the code I ran, for reference (though it's pretty much straight from the example):

with torch.no_grad():
    text = clip.tokenize(text_list).to(device)
    for i in tqdm(range(len(images))):
        image = preprocess(Image.open(images[i])).unsqueeze(0).to(device)    
        logits_per_image, logits_per_text = model(image, text)

It seems sort of indeterministic - in a for loop, sometimes it runs for one pass and breaks on the second. Sometimes it breaks on the first.

I'm using one GPU. The text is dtype torch.int64, and image is dtype torch.float32.

However, the forward pass works with the RN50x4 and ViT-B/32 models.

There are a couple other issues at the moment that address half-precision:

#57, on mix-precision training
#30, on the half-precision for a GPU and full for a CPU

Yet, these don't mention problem I'm running into.

Any help would be really appreciated!

Originally created by @spetryk on GitHub (Apr 10, 2021). Original GitHub issue: https://github.com/openai/CLIP/issues/92 Hi, thanks for putting together a great repo. It has worked smoothly, except for the following: When running the line `logits_per_image, logits_per_text = model(image, text)` with the RN50 or RN101 models, I run into the following error: ``` RuntimeError: default_program(57): error: identifier "prim_constantchunk_flat__1" is undefined default_program(58): error: no operator "=" matches these operands operand types are: half = float 2 errors detected in the compilation of "default_program". nvrtc compilation failed: ``` And then there's a bunch of C-looking code from the error message that I can post if needed. A sample of the code I ran, for reference (though it's pretty much straight from the example): ``` with torch.no_grad(): text = clip.tokenize(text_list).to(device) for i in tqdm(range(len(images))): image = preprocess(Image.open(images[i])).unsqueeze(0).to(device) logits_per_image, logits_per_text = model(image, text) ``` It seems sort of indeterministic - in a for loop, sometimes it runs for one pass and breaks on the second. Sometimes it breaks on the first. I'm using one GPU. The `text` is dtype `torch.int64`, and `image` is dtype `torch.float32`. However, the forward pass works with the RN50x4 and ViT-B/32 models. There are a couple other issues at the moment that address half-precision: - #57, on mix-precision training - #30, on the half-precision for a GPU and full for a CPU Yet, these don't mention problem I'm running into. Any help would be really appreciated!

bsakaguchi closed this issue

2026-10-31 21:10:55 +00:00

bsakaguchi commented

2026-10-31 21:10:56 +00:00

Author

Owner

@jongwook commented on GitHub (Apr 11, 2021):

Can you try supplying jit=False while loading the model? I'm thinking of making jit=False the default since it's less compatible across PyTorch versions. (cf. #79)

@jongwook commented on GitHub (Apr 11, 2021): Can you try supplying `jit=False` while loading the model? I'm thinking of making jit=False the default since it's less compatible across PyTorch versions. (cf. #79)

bsakaguchi commented

2026-10-31 21:10:56 +00:00

Author

Owner

@spetryk commented on GitHub (Apr 12, 2021):

Great, that seems to have been the problem! It's working now. Thanks! 🎉

@spetryk commented on GitHub (Apr 12, 2021): Great, that seems to have been the problem! It's working now. Thanks! :tada:

bsakaguchi commented

2026-10-31 21:10:56 +00:00

Author

Owner

@ooza commented on GitHub (May 17, 2021):

@spetryk
Hi, as a new in the domain of image captioning, I'm wondering if the text_list was prepared by yourself or it's a default setting?
What should I prepare if I want to test the available clip models on my own images ?
Thanks

@ooza commented on GitHub (May 17, 2021): @spetryk Hi, as a new in the domain of image captioning, I'm wondering if the text_list was prepared by yourself or it's a default setting? What should I prepare if I want to test the available clip models on my own images ? Thanks

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

openai/CLIP#84

No description provided.

Rows
Columns