deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[GH-ISSUE #195] Perf regression since sha f85ec6 #109

New issue
Open
opened 2026-10-26 21:12:49 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 21:12:49 +00:00
Owner
Copy link

Originally created by @hj-mistral on GitHub (Sep 19, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/195

Hi team, I am investigating a regression in performance for m_grouped_gemm_fp8_fp8_bf16_nt_masked. Benchmarking script to repro: https://gist.github.com/hj-mistral/d38801ce8e35860a7faba1e1688546cc.

Env

GPU: H200
CUDA: 12.9

Script output

On sha 79f48ee15a (current head)

Average time per iteration: 26.55 us
Bandwidth: 1030.01 GB/s

On sha ea9c5d9270

Average time per iteration: 26.70 us
Bandwidth: 1024.26 GB/s

On sha 3254b758e2

Average time per iteration: 20.16 us
Bandwidth: 1356.20 GB/s

Can you confirm?

Originally created by @hj-mistral on GitHub (Sep 19, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/195 Hi team, I am investigating a regression in performance for `m_grouped_gemm_fp8_fp8_bf16_nt_masked`. Benchmarking script to repro: https://gist.github.com/hj-mistral/d38801ce8e35860a7faba1e1688546cc. ## Env GPU: H200 CUDA: 12.9 ## Script output ### On sha 79f48ee15a82dd5fad5cd9beaa393c1f755e6b55 (current head) ``` Average time per iteration: 26.55 us Bandwidth: 1030.01 GB/s ``` ### On sha ea9c5d9270226c5dd7a577c212e9ea385f6ef048 ``` Average time per iteration: 26.70 us Bandwidth: 1024.26 GB/s ``` ### On sha 3254b758e27a5b2f2ae68279314acb1adcb6c1bc ``` Average time per iteration: 20.16 us Bandwidth: 1356.20 GB/s ``` Can you confirm?
bsakaguchi commented 2026-10-26 21:12:50 +00:00
Author
Owner
Copy link

@zheanxu commented on GitHub (Sep 25, 2025):

Compared to commit 3254b75, the latest commit has optimized the shape configuration used by the actual inference service (tested in test_fp8.py). However, in some other scenarios, performance degradation is possible, as the current heuristic is difficult to cover all cases.
You can set the DG_PRINT_CONFIGS=1 environment variable to print out the configuration differences, and choose the optimal configuration based on your use case.

<!-- gh-comment-id:3333150410 --> @zheanxu commented on GitHub (Sep 25, 2025): Compared to commit 3254b75, the latest commit has optimized the shape configuration used by the actual inference service (tested in `test_fp8.py`). However, in some other scenarios, performance degradation is possible, as the current heuristic is difficult to cover all cases. You can set the `DG_PRINT_CONFIGS=1` environment variable to print out the configuration differences, and choose the optimal configuration based on your use case.
bsakaguchi commented 2026-10-26 21:12:51 +00:00
Author
Owner
Copy link

@hj-mistral commented on GitHub (Oct 20, 2025):

Thanks! Is there a way to override the configurations without maintaining a fork? With some env variables or flags?

<!-- gh-comment-id:3422000877 --> @hj-mistral commented on GitHub (Oct 20, 2025): Thanks! Is there a way to override the configurations without maintaining a fork? With some env variables or flags?
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#109
No description provided.