deepseek-ai/DeepSeek-V3
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-V3.git synced 2026-11-05 00:47:48 +00:00
Code Issues 25 Projects Packages Wiki Activity

[GH-ISSUE #592] [Paper BUG] Conflict between Figure 3, formula 21 and formula 22 #319

New issue
Closed
opened 2026-10-26 21:19:32 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 21:19:32 +00:00
Owner
Copy link

Originally created by @junjzhang on GitHub (Feb 6, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/592

The conflict is that Figure 3 and formula 22 indicate the input of TRM_k is T-K token (k:T-k), while formula 22 indicates the input of TRM_k is T-k token (1:T-k).
Clearly, we can see this from Figure 3
Image
Also, according to formula 21, since the word embedding is shifted by k, we can conclude that the token in i'th position of $$TRM_k$$'s input should be (i+k)'th token in the whole sentence.
Image
However, we can see the subscript of formula 22 is 1:T-k
Image

Originally created by @junjzhang on GitHub (Feb 6, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/592 The conflict is that Figure 3 and formula 22 indicate the input of $$TRM_k$$ is T-K token (k:T-k), while formula 22 indicates the input of $$TRM_k$$ is T-k token (1:T-k). Clearly, we can see this from Figure 3 ![Image](https://github.com/user-attachments/assets/fc1b96c3-9ea8-4de3-b819-c35222582cf0) Also, according to formula 21, since the word embedding is shifted by k, we can conclude that the token in i'th position of $$TRM_k$$'s input should be (i+k)'th token in the whole sentence. ![Image](https://github.com/user-attachments/assets/89562bcc-e8d4-47ab-93b5-f538b5ee9bec) However, we can see the subscript of formula 22 is 1:T-k ![Image](https://github.com/user-attachments/assets/1bbb55f8-5c3d-402a-9afe-59b18f40802f)
bsakaguchi 2026-10-26 21:19:32 +00:00
bsakaguchi commented 2026-10-26 21:19:33 +00:00
Author
Owner
Copy link

@github-actions[bot] commented on GitHub (Mar 11, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

<!-- gh-comment-id:2712136356 --> @github-actions[bot] commented on GitHub (Mar 11, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
bsakaguchi commented 2026-10-26 21:19:33 +00:00
Author
Owner
Copy link

@github-actions[bot] commented on GitHub (Mar 30, 2025):

false

<!-- gh-comment-id:2764304929 --> @github-actions[bot] commented on GitHub (Mar 30, 2025): false
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v1.0.0
Labels
Clear labels   
closed-as-stale

   
enhancement

   
question

   
spam

   
stale
No labels
closed-as-stale
enhancement
question
spam
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-V3#319
No description provided.