mirror of
https://github.com/deepseek-ai/DeepSeek-V2.git
synced 2026-11-05 00:47:23 +00:00
[GH-ISSUE #92] Inquiry about Key/Value Storage and Matrix Merging in DeepSeekerV2 Inference Code #82
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-V2#82
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @xlim1997 on GitHub (Sep 26, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/92
Dear DeepSeekerV2 team,
First of all, I would like to thank you for your incredible work on DeepSeekerV2. I am very interested in the model and have been exploring it in detail. However, I have a couple of questions related to the implementation of your inference process.
In the paper, you mentioned that during inference, the compressed latent vectors for keys and values (ct^kv) are stored. However, when I checked the HuggingFace code implementation, I noticed that Key_states and Value_states are still being saved separately during inference. Could you clarify how this aligns with the approach mentioned in the paper?
Additionally, the paper discusses merging W^UV into WO and W^UK into WQ for efficiency. However, I couldn't locate this merging process in the code either. Could you provide some insights or point me in the right direction on how this is implemented?
Thank you again for your fantastic work, and I really look forward to your guidance on these points.
Best regards,
lucas