mirror of
https://github.com/openai/CLIP.git
synced 2026-11-05 01:37:26 +00:00
[GH-ISSUE #212] RuntimeError: Input is too long for context length 77 #185
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
openai/CLIP#185
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @ancordovag on GitHub (Feb 5, 2022).
Original GitHub issue: https://github.com/openai/CLIP/issues/212
This happens when trying to tokenize ( clip.tokenize(train_sentences).to(device) ) sentences that have less than 77 tokens (for example 44), but some of them are unknown.
I have tried to operate the default argument context_length of the tokenize function (for example context_length =100), but then the encode function ( clip_model.encode_text ) would complain.
I have tried replacing all the atypical expressions and special characters, but still is an issue. Even with some simple sentence of 44 tokens like:
"But the Commission cannot accept Amendments Nos 1 4 9 11 13 15 19 23 39 40 41 49 51 59 61 64 66 70 71 72 74 77 83 87 90 91 92 95 97 98 81 rev 100 101 103 and 107".
I'm getting the data of the European Parliament which sure was not created to describe images, but still. I wanted to know if there is a possible solution, even like skipping the impossible sentences.
Thanks.
@dazcona commented on GitHub (Feb 10, 2022):
I am experiencing the same issue. There is no way to provide a keyword argument to
clip.tokenizeliketruncatehere (github.com/openai/CLIP@40f5484c1c/clip/clip.py (L195)) so it does not raise an exception@hannahlyon commented on GitHub (Sep 12, 2023):
Not sure if this was updated, but you can pass truncate as a kwarg:
self.x = clip.tokenize(text, context_length=77, truncate=True)
@arnodjiang commented on GitHub (Oct 19, 2023):
so why design for " def tokenize(texts: Union[str, List[str]], context_length: int = 77, truncate: bool = False) -> torch.LongTensor: ", what should I change context_length?