mirror of
https://github.com/openai/CLIP.git
synced 2026-11-05 01:37:26 +00:00
[GH-ISSUE #195] Question about the paper. #171
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
openai/CLIP#171
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @zhhao1 on GitHub (Dec 15, 2021).
Original GitHub issue: https://github.com/openai/CLIP/issues/195
The paper gave me a lot of inspiration, but I have some questions about the structure.
First, why the text encoder use the mask self attention? The paper have written " Masked self-attention was used in the text encoder to preserve the ability to initialize with a pre-trained language model or add language modeling as an auxiliary objective, though exploration of this is left as future work." And the is considered as the whole sentence representation. Will it be better if using self-attention and considering the mean of the last transformer encoder output?
Second, why attention pooling mechanism is used instead of global average pooling? The paper have written "We also replace
the global average pooling layer with an attention pooling mechanism." But i can't find and explanation for this.
I am very confused about this, and hope you take the time to answer my question.