openai/CLIP
1
0
Fork 0
mirror of https://github.com/openai/CLIP.git synced 2026-11-05 01:37:26 +00:00
Code Issues 274 Projects Packages Wiki Activity

[GH-ISSUE #352] Improve the Speed in Batch Processing #306

New issue
Open
opened 2026-10-31 21:13:40 +00:00 by bsakaguchi · 3 comments
bsakaguchi commented 2026-10-31 21:13:40 +00:00
Owner
Copy link

Originally created by @ahmadmustafaanis on GitHub (Apr 19, 2023).
Original GitHub issue: https://github.com/openai/CLIP/issues/352

How can I improve the Speed of my CLIP model in Batch Processing. Minimum Code:

tokenized_text = clip.tokenize(classes_descriptions).to(clip_device)

pre_processed_imgs =  [preprocess(Image.fromarray(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))) for im in batch]

logits_per_image, logits_per_text = model(
        torch.stack(pre_processed_imgs).to(device), tokenized_text
    )
probs = logits_per_image.softmax(dim=-1).detach().cpu().numpy()

classes = [classes_descriptions[np.argmax(prob)] if prob.max() > 0.1 else None for prob in probs]

When the batch size is big, even for small images (100x100) it takes a lot of time i.e a 0.4, 0.5 seconds. I want to process it faster. What are some of the techniques/tricks I can use.

Originally created by @ahmadmustafaanis on GitHub (Apr 19, 2023). Original GitHub issue: https://github.com/openai/CLIP/issues/352 How can I improve the Speed of my CLIP model in Batch Processing. Minimum Code: ```python3 tokenized_text = clip.tokenize(classes_descriptions).to(clip_device) pre_processed_imgs = [preprocess(Image.fromarray(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))) for im in batch] logits_per_image, logits_per_text = model( torch.stack(pre_processed_imgs).to(device), tokenized_text ) probs = logits_per_image.softmax(dim=-1).detach().cpu().numpy() classes = [classes_descriptions[np.argmax(prob)] if prob.max() > 0.1 else None for prob in probs] ``` When the batch size is big, even for small images (100x100) it takes a lot of time i.e a 0.4, 0.5 seconds. I want to process it faster. What are some of the techniques/tricks I can use.
bsakaguchi commented 2026-10-31 21:13:40 +00:00
Author
Owner
Copy link

@adtygan commented on GitHub (May 4, 2023):

Line 2: pre_processed_imgs = [preprocess(Image.fromarray(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))) for im in batch]
Line 5: classes = [classes_descriptions[np.argmax(prob)] if prob.max() > 0.1 else None for prob in probs]

These 2 lines are using list comprehensions. Could you try using vectorization and see if performance improves?

<!-- gh-comment-id:1534705783 --> @adtygan commented on GitHub (May 4, 2023): Line 2: `pre_processed_imgs = [preprocess(Image.fromarray(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))) for im in batch]` Line 5: `classes = [classes_descriptions[np.argmax(prob)] if prob.max() > 0.1 else None for prob in probs]` These 2 lines are using list comprehensions. Could you try using vectorization and see if performance improves?
bsakaguchi commented 2026-10-31 21:13:40 +00:00
Author
Owner
Copy link

@ahmadmustafaanis commented on GitHub (May 5, 2023):

preprocess works using PIL and hence works on a single image not a batc.

<!-- gh-comment-id:1536655769 --> @ahmadmustafaanis commented on GitHub (May 5, 2023): preprocess works using PIL and hence works on a single image not a batc.
bsakaguchi commented 2026-10-31 21:13:40 +00:00
Author
Owner
Copy link

@mohdomama commented on GitHub (Dec 30, 2023):

Any solution for this yet?

<!-- gh-comment-id:1872608455 --> @mohdomama commented on GitHub (Dec 30, 2023): Any solution for this yet?
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
codex/pin-github-workflow-refs-20260325-132838
jongwook/issue-396
jongwook-patch-1
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
openai/CLIP#306
No description provided.