[GH-ISSUE #269] How to train CLIP from scratch? #235

Open
opened 2026-10-31 21:12:48 +00:00 by bsakaguchi · 1 comment
Owner

Originally created by @newbietuan on GitHub (Jul 20, 2022).
Original GitHub issue: https://github.com/openai/CLIP/issues/269

Hi, Thanks for the great work.
Due to the needs of specific tasks, i want to train CLIP from scratch without using BPE coding and the length limit of 77, how should i do?
about length_limit, it seems that is ok not to use simple_tokenizer.py, so the mainly problem is how to train CLIP from scratch?
Very thanks.

Originally created by @newbietuan on GitHub (Jul 20, 2022). Original GitHub issue: https://github.com/openai/CLIP/issues/269 Hi, Thanks for the great work. Due to the needs of specific tasks, i want to train CLIP from scratch without using BPE coding and the length limit of 77, how should i do? about length_limit, it seems that is ok not to use simple_tokenizer.py, so the mainly problem is how to train CLIP from scratch? Very thanks.
Author
Owner

@jinghaoliu commented on GitHub (Jul 25, 2022):

It seems that OpenAi doesn't provide a code that can train CLIP from scratch. You could refer to the codebase.https://github.com/mlfoundations/open_clip Hope that is helpful for you.

<!-- gh-comment-id:1193625698 --> @jinghaoliu commented on GitHub (Jul 25, 2022): It seems that OpenAi doesn't provide a code that can train CLIP from scratch. You could refer to the codebase.[https://github.com/mlfoundations/open_clip](url) Hope that is helpful for you.
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
openai/CLIP#235
No description provided.