[GH-ISSUE #220] How can I get accuracy metrics when training? #192

New issue

Open

opened 2026-10-31 21:12:13 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 21:12:13 +00:00

Owner

Originally created by @dongyun-kim-arch on GitHub (Feb 20, 2022).
Original GitHub issue: https://github.com/openai/CLIP/issues/220

Hello!
Thank you for sharing this awesome model!

I am trying to fine-tune CLIP model to my own dataset. It seems many people talked about the accuracy when they evaluated their model performance, but I could only get loss to evaluate my model performance.
Is there any reference code I can refer to for getting accuracy?

Thanks!

Originally created by @dongyun-kim-arch on GitHub (Feb 20, 2022). Original GitHub issue: https://github.com/openai/CLIP/issues/220 Hello! Thank you for sharing this awesome model! I am trying to fine-tune CLIP model to my own dataset. It seems many people talked about the accuracy when they evaluated their model performance, but I could only get loss to evaluate my model performance. Is there any reference code I can refer to for getting accuracy? Thanks!

bsakaguchi commented

2026-10-31 21:12:14 +00:00

Author

Owner

@jongwook commented on GitHub (Apr 11, 2022):

"accuracy" during training probably meant the proportion of the training examples that had correctly predicted the contrastive label, e.g.:

contrastive_label = torch.arange(batch_size)

image_loss = cross_entropy(image_logits, contrastive_label)
text_loss = cross_entropy(text_logits, contrastive_label)

image_acc = (image_logits.argmax(dim=-1) == contrastive_label).float()
text_acc = (text_logits.argmax(dim=-1) == contrastive_label).float()

@jongwook commented on GitHub (Apr 11, 2022): "accuracy" during training probably meant the proportion of the training examples that had correctly predicted the contrastive label, e.g.: contrastive_label = torch.arange(batch_size) image_loss = cross_entropy(image_logits, contrastive_label) text_loss = cross_entropy(text_logits, contrastive_label) image_acc = (image_logits.argmax(dim=-1) == contrastive_label).float() text_acc = (text_logits.argmax(dim=-1) == contrastive_label).float()

bsakaguchi commented

2026-10-31 21:12:14 +00:00

Author

Owner

@newbietuan commented on GitHub (Apr 18, 2022):

"accuracy" during training probably meant the proportion of the training examples that had correctly predicted the contrastive label, e.g.:
contrastive_label = torch.arange(batch_size)

image_loss = cross_entropy(image_logits, contrastive_label)
text_loss = cross_entropy(text_logits, contrastive_label)

image_acc = (image_logits.argmax(dim=-1) == contrastive_label).float()
text_acc = (text_logits.argmax(dim=-1) == contrastive_label).float()

@jongwook hello! during fine-tune the model on my own dataset with the format of image-caption, if i should use 'image_acc = (image_logits.argmax(dim=-1) == contrastive_label).float()' directly?

@newbietuan commented on GitHub (Apr 18, 2022): > "accuracy" during training probably meant the proportion of the training examples that had correctly predicted the contrastive label, e.g.: > > ``` > contrastive_label = torch.arange(batch_size) > > image_loss = cross_entropy(image_logits, contrastive_label) > text_loss = cross_entropy(text_logits, contrastive_label) > > image_acc = (image_logits.argmax(dim=-1) == contrastive_label).float() > text_acc = (text_logits.argmax(dim=-1) == contrastive_label).float() > ``` @jongwook hello! during fine-tune the model on my own dataset with the format of image-caption, if i should use 'image_acc = (image_logits.argmax(dim=-1) == contrastive_label).float()' directly?

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

openai/CLIP#192

No description provided.

Rows
Columns