mirror of
https://github.com/deepseek-ai/DeepGEMM.git
synced 2026-11-05 00:47:26 +00:00
[GH-ISSUE #279] [v3.2] deep_gemm::fp8_mqa_logits performs poorly on the H200 #142
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepGEMM#142
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @thqq479 on GitHub (Jan 16, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/279
@thqq479 commented on GitHub (Jan 16, 2026):
env : python3.10 cuda12.4 deep_gemm=2.1.1.post3
server : python -m sglang.launch_server --disable-overlap-schedule --model-path "$MODEL_PATH" --port "$PORT" --schedule-conservativeness 0.2 --tp 8 --chunked-prefill-size 8192 --mem-fraction-static 0 .83 --tool-call-parser deepseekv32 --reasoning-parser deepseek-v3 --decode-log-interval 1 --disable-radix-cache
client : python3 -m sglang.bench_serving --backend sglang --dataset-name random-ids --num-prompts 1 --random-input 64000 --random-output 100 --random-range-ratio 0.9 --model /DeepSeek-V3.2 --max-concurrency 1 --host 127.0.0.1 --port 30000 --request-rate 1 --profier
@thqq479 commented on GitHub (Jan 16, 2026):
The performance of sglang running v3.2 is significantly worse than v3.1. Profiler analysis shows that the fp8_mqa_logits() function accounts for half of the total attention time, which is not what we expected, right? @LyricZhao
@thqq479 commented on GitHub (Jan 16, 2026):