[GH-ISSUE #336] Any Google Colab Notebook for calculating image similarity of given 2 images? #289

New issue

Open

opened 2026-10-31 21:13:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 21:13:29 +00:00

Owner

Originally created by @FurkanGozukara on GitHub (Mar 11, 2023).
Original GitHub issue: https://github.com/openai/CLIP/issues/336

Hello. I am looking for a complete code piece to calculate image similarity of given 2 images.

These 2 images can be any dimension or any image format so it should be able to handle all cases.

I want to calculate similarity of given 2 cloth images. Such as how similar t shirt A and B

Any help is very appreciated. I believe currently CLIP is the best AI model for this task?

So it is pretty simple application actually.

It will take image A and B and then will give similarity score as a number.

Thank you so much.

where is the error in this code?

import torch
import torchvision.transforms as transforms
import urllib.request
from transformers import CLIPProcessor, CLIPModel, CLIPTokenizer
from PIL import Image

# Load the CLIP model
device = "cuda" if torch.cuda.is_available() else "cpu"
model_ID = "openai/clip-vit-base-patch32"
model = CLIPModel.from_pretrained(model_ID).to(device)

preprocess = CLIPProcessor.from_pretrained(model_ID)


# Define a function to load an image and preprocess it for CLIP
def load_and_preprocess_image(image_path):
    # Load the image from the specified path
    image = Image.open(image_path)

    # Apply the CLIP preprocessing to the image
    image = preprocess(image).unsqueeze(0).to(device)

    # Return the preprocessed image
    return image

# Load the two images and preprocess them for CLIP
image_a = load_and_preprocess_image('/content/a.png')
image_b = load_and_preprocess_image('/content/b.png')

# Calculate the embeddings for the images using the CLIP model
with torch.no_grad():
    embedding_a = model.encode_image(image_a)
    embedding_b = model.encode_image(image_b)

# Calculate the cosine similarity between the embeddings
similarity_score = torch.nn.functional.cosine_similarity(embedding_a, embedding_b)

# Print the similarity score
print('Similarity score:', similarity_score.item())

this is the error i am getting

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-24-e95a926e1bc8>](https://localhost:8080/#) in <module>
     25 
     26 # Load the two images and preprocess them for CLIP
---> 27 image_a = load_and_preprocess_image('/content/a.png')
     28 image_b = load_and_preprocess_image('/content/b.png')
     29 

3 frames
[/usr/local/lib/python3.9/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in _call_one(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   2579 
   2580         if not _is_valid_text_input(text):
-> 2581             raise ValueError(
   2582                 "text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) "
   2583                 "or `List[List[str]]` (batch of pretokenized examples)."

ValueError: text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples)

Originally created by @FurkanGozukara on GitHub (Mar 11, 2023). Original GitHub issue: https://github.com/openai/CLIP/issues/336 Hello. I am looking for a complete code piece to calculate image similarity of given 2 images. These 2 images can be any dimension or any image format so it should be able to handle all cases. I want to calculate similarity of given 2 cloth images. Such as how similar t shirt A and B Any help is very appreciated. I believe currently CLIP is the best AI model for this task? So it is pretty simple application actually. It will take image A and B and then will give similarity score as a number. Thank you so much. where is the error in this code? ``` import torch import torchvision.transforms as transforms import urllib.request from transformers import CLIPProcessor, CLIPModel, CLIPTokenizer from PIL import Image # Load the CLIP model device = "cuda" if torch.cuda.is_available() else "cpu" model_ID = "openai/clip-vit-base-patch32" model = CLIPModel.from_pretrained(model_ID).to(device) preprocess = CLIPProcessor.from_pretrained(model_ID) # Define a function to load an image and preprocess it for CLIP def load_and_preprocess_image(image_path): # Load the image from the specified path image = Image.open(image_path) # Apply the CLIP preprocessing to the image image = preprocess(image).unsqueeze(0).to(device) # Return the preprocessed image return image # Load the two images and preprocess them for CLIP image_a = load_and_preprocess_image('/content/a.png') image_b = load_and_preprocess_image('/content/b.png') # Calculate the embeddings for the images using the CLIP model with torch.no_grad(): embedding_a = model.encode_image(image_a) embedding_b = model.encode_image(image_b) # Calculate the cosine similarity between the embeddings similarity_score = torch.nn.functional.cosine_similarity(embedding_a, embedding_b) # Print the similarity score print('Similarity score:', similarity_score.item()) ``` this is the error i am getting ``` --------------------------------------------------------------------------- ValueError Traceback (most recent call last) [<ipython-input-24-e95a926e1bc8>](https://localhost:8080/#) in <module> 25 26 # Load the two images and preprocess them for CLIP ---> 27 image_a = load_and_preprocess_image('/content/a.png') 28 image_b = load_and_preprocess_image('/content/b.png') 29 3 frames [/usr/local/lib/python3.9/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in _call_one(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs) 2579 2580 if not _is_valid_text_input(text): -> 2581 raise ValueError( 2582 "text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) " 2583 "or `List[List[str]]` (batch of pretokenized examples)." ValueError: text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples) ```

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

openai/CLIP#289

No description provided.

Rows
Columns