deepseek-ai/DeepSeek-V2
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-V2.git synced 2026-11-05 00:47:23 +00:00
Code Issues 90 Projects Packages Wiki Activity

[GH-ISSUE #101] [Bug]: MLA实现没有带来任何收益 #93

New issue
Open
opened 2026-10-26 21:07:48 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:07:48 +00:00
Owner
Copy link

Originally created by @foamliu on GitHub (Jan 1, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/101

MLA（multi head latent attention）的实现本来是为着提升推理速度，但由于存入缓存的数据比基线（Llama）更大，因此不但未带来任何收益，而且与基线（Llama）相比，占用显存更多，推理更慢。

下面是 DeepSeekV3 HF官网的MLA实现，可见存入KVCache的数据量，比基线（Llama）还大：
cba6bdda9920aacfab1acc96e21652a

下面是推理测速的结果：
image

Originally created by @foamliu on GitHub (Jan 1, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/101 MLA（multi head latent attention）的实现本来是为着提升推理速度，但由于存入缓存的数据比基线（Llama）更大，因此不但未带来任何收益，而且与基线（Llama）相比，占用显存更多，推理更慢。 下面是 DeepSeekV3 HF官网的MLA实现，可见存入KVCache的数据量，比基线（Llama）还大： ![cba6bdda9920aacfab1acc96e21652a](https://github.com/user-attachments/assets/7dbad84d-ff6d-4aff-b5c2-574054bc4fba) 下面是推理测速的结果： ![image](https://github.com/user-attachments/assets/701dd5e2-92b4-4c12-bc67-9dac03149108)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-V2#93
No description provided.