deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[PR #44] [MERGED] Performance: Configuration algorithms tuned to minimize the impact of tail effects, now up to 1402 TFLOPS #217

New issue
Closed
opened 2026-10-26 21:14:58 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:14:58 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/44
Author: @sazczmh
Created: 3/6/2025
Status: ✅ Merged
Merged: 3/10/2025
Merged by: @LyricZhao

Base: main ← Head: main

📝 Commits (3)

  • 50cf26c Performance: Configuration algorithms tuned to minimize the impact of tail effects, now up to 1402 TFLOPS
  • ed278ed formats: Optimize get_best_configs implementation
  • bed67b2 Minor fix

📊 Changes

2 files changed (+11 additions, -4 deletions)

View changed files

📝 deep_gemm/jit_kernels/gemm.py (+9 -2)
📝 deep_gemm/jit_kernels/m_grouped_gemm.py (+2 -2)

📄 Description

Configuration algorithms tuned to minimize the impact of tail effects, for 4096x7168x16384, approximately 1.6% performance gain.

M N K Base_BlockS Computation Tail_BlockS Computation Speedup
64 2112 7168 132 195 TFLOPS 132 194 TFLOPS -0.51%
64 24576 1536 132 281 TFLOPS 128 283 TFLOPS 0.71%
64 32768 512 132 217 TFLOPS 128 218 TFLOPS 0.46%
64 7168 16384 132 329 TFLOPS 132 328 TFLOPS -0.30%
64 4096 7168 132 269 TFLOPS 132 268 TFLOPS -0.37%
64 7168 2048 132 272 TFLOPS 132 269 TFLOPS -1.10%
128 2112 7168 132 341 TFLOPS 132 340 TFLOPS -0.29%
128 24576 1536 132 521 TFLOPS 128 520 TFLOPS -0.19%
128 32768 512 132 358 TFLOPS 128 365 TFLOPS 1.96%
128 7168 16384 132 631 TFLOPS 132 633 TFLOPS 0.32%
128 4096 7168 132 505 TFLOPS 132 504 TFLOPS -0.20%
128 7168 2048 132 486 TFLOPS 132 485 TFLOPS -0.21%
4096 2112 7168 132 1054 TFLOPS 124 1071 TFLOPS 1.61%
4096 24576 1536 132 992 TFLOPS 132 988 TFLOPS -0.40%
4096 32768 512 132 595 TFLOPS 132 591 TFLOPS -0.67%
4096 7168 16384 132 1348 TFLOPS 128 1369 TFLOPS 1.56%
4096 4096 7168 132 1325 TFLOPS 128 1333 TFLOPS 0.60%
4096 7168 2048 132 1026 TFLOPS 128 1022 TFLOPS -0.39%

Together with the optimization of FFMA interleaving, for 4096x7168x16384, approximately 4% performance gain，up to 1402TFLOPS

M N K Base_BlockS Computation Tail+FFMA_BlockS Computation Speedup
64 2112 7168 132 195 TFLOPS 132 194 TFLOPS -0.51%
64 24576 1536 132 281 TFLOPS 128 282 TFLOPS 0.36%
64 32768 512 132 217 TFLOPS 128 219 TFLOPS 0.92%
64 7168 16384 132 329 TFLOPS 132 328 TFLOPS -0.30%
64 4096 7168 132 269 TFLOPS 132 270 TFLOPS 0.37%
64 7168 2048 132 272 TFLOPS 132 271 TFLOPS -0.37%
128 2112 7168 132 341 TFLOPS 132 339 TFLOPS -0.59%
128 24576 1536 132 521 TFLOPS 128 523 TFLOPS 0.38%
128 32768 512 132 358 TFLOPS 128 355 TFLOPS -0.84%
128 7168 16384 132 631 TFLOPS 132 633 TFLOPS 0.32%
128 4096 7168 132 505 TFLOPS 132 511 TFLOPS 1.19%
128 7168 2048 132 486 TFLOPS 132 485 TFLOPS -0.21%
4096 2112 7168 132 1054 TFLOPS 124 1065 TFLOPS 1.04%
4096 24576 1536 132 992 TFLOPS 132 1009 TFLOPS 1.71%
4096 32768 512 132 595 TFLOPS 132 592 TFLOPS -0.50%
4096 7168 16384 132 1348 TFLOPS 128 1402 TFLOPS 4.01%
4096 4096 7168 132 1325 TFLOPS 128 1355 TFLOPS 2.26%
4096 7168 2048 132 1026 TFLOPS 128 1042 TFLOPS 1.56%

Test on H100-SXM && CUDA 12.8.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/44 **Author:** [@sazczmh](https://github.com/sazczmh) **Created:** 3/6/2025 **Status:** ✅ Merged **Merged:** 3/10/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (3) - [`50cf26c`](https://github.com/deepseek-ai/DeepGEMM/commit/50cf26cc7cfbf2d6b243bfce995cef976969a880) Performance: Configuration algorithms tuned to minimize the impact of tail effects, now up to 1402 TFLOPS - [`ed278ed`](https://github.com/deepseek-ai/DeepGEMM/commit/ed278eddd34c724a6bbfda5cbd5253d7ba2ee603) formats: Optimize get_best_configs implementation - [`bed67b2`](https://github.com/deepseek-ai/DeepGEMM/commit/bed67b234cf8341015dc6c5b99759bc4479a7f3e) Minor fix ### 📊 Changes **2 files changed** (+11 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `deep_gemm/jit_kernels/gemm.py` (+9 -2) 📝 `deep_gemm/jit_kernels/m_grouped_gemm.py` (+2 -2) </details> ### 📄 Description Configuration algorithms tuned to minimize the impact of tail effects, for 4096x7168x16384, approximately 1.6% performance gain. | M | N | K | Base_BlockS | Computation | Tail_BlockS | Computation | Speedup | |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 64 | 2112 | 7168 | 132 | 195 TFLOPS | 132 | 194 TFLOPS | -0.51% | | 64 | 24576 | 1536 | 132 | 281 TFLOPS | **128** | 283 TFLOPS | 0.71% | | 64 | 32768 | 512 | 132 | 217 TFLOPS | **128** | 218 TFLOPS | 0.46% | | 64 | 7168 | 16384 | 132 | 329 TFLOPS | 132 | 328 TFLOPS | -0.30% | | 64 | 4096 | 7168 | 132 | 269 TFLOPS | 132 | 268 TFLOPS | -0.37% | | 64 | 7168 | 2048 | 132 | 272 TFLOPS | 132 | 269 TFLOPS | -1.10% | | 128 | 2112 | 7168 | 132 | 341 TFLOPS | 132 | 340 TFLOPS | -0.29% | | 128 | 24576 | 1536 | 132 | 521 TFLOPS | **128** | 520 TFLOPS | -0.19% | | 128 | 32768 | 512 | 132 | 358 TFLOPS | **128** | 365 TFLOPS | 1.96% | | 128 | 7168 | 16384 | 132 | 631 TFLOPS | 132 | 633 TFLOPS | 0.32% | | 128 | 4096 | 7168 | 132 | 505 TFLOPS | 132 | 504 TFLOPS | -0.20% | | 128 | 7168 | 2048 | 132 | 486 TFLOPS | 132 | 485 TFLOPS | -0.21% | | 4096 | 2112 | 7168 | 132 | 1054 TFLOPS | **124** | 1071 TFLOPS | 1.61% | | 4096 | 24576 | 1536 | 132 | 992 TFLOPS | 132 | 988 TFLOPS | -0.40% | | 4096 | 32768 | 512 | 132 | 595 TFLOPS | 132 | 591 TFLOPS | -0.67% | | 4096 | 7168 | 16384 | 132 | 1348 TFLOPS | **128** | 1369 TFLOPS | 1.56% | | 4096 | 4096 | 7168 | 132 | 1325 TFLOPS | **128** | 1333 TFLOPS | 0.60% | | 4096 | 7168 | 2048 | 132 | 1026 TFLOPS | **128** | 1022 TFLOPS | -0.39% | Together with the optimization of FFMA interleaving, for 4096x7168x16384, approximately 4% performance gain，up to 1402TFLOPS | M | N | K | Base_BlockS | Computation | Tail+FFMA_BlockS | Computation | Speedup | |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 64 | 2112 | 7168 | 132 | 195 TFLOPS | 132 | 194 TFLOPS | -0.51% | | 64 | 24576 | 1536 | 132 | 281 TFLOPS | **128** | 282 TFLOPS | 0.36% | | 64 | 32768 | 512 | 132 | 217 TFLOPS | **128** | 219 TFLOPS | 0.92% | | 64 | 7168 | 16384 | 132 | 329 TFLOPS | 132 | 328 TFLOPS | -0.30% | | 64 | 4096 | 7168 | 132 | 269 TFLOPS | 132 | 270 TFLOPS | 0.37% | | 64 | 7168 | 2048 | 132 | 272 TFLOPS | 132 | 271 TFLOPS | -0.37% | | 128 | 2112 | 7168 | 132 | 341 TFLOPS | 132 | 339 TFLOPS | -0.59% | | 128 | 24576 | 1536 | 132 | 521 TFLOPS | **128** | 523 TFLOPS | 0.38% | | 128 | 32768 | 512 | 132 | 358 TFLOPS | **128** | 355 TFLOPS | -0.84% | | 128 | 7168 | 16384 | 132 | 631 TFLOPS | 132 | 633 TFLOPS | 0.32% | | 128 | 4096 | 7168 | 132 | 505 TFLOPS | 132 | 511 TFLOPS | 1.19% | | 128 | 7168 | 2048 | 132 | 486 TFLOPS | 132 | 485 TFLOPS | -0.21% | | 4096 | 2112 | 7168 | 132 | 1054 TFLOPS | **124** | 1065 TFLOPS | 1.04% | | 4096 | 24576 | 1536 | 132 | 992 TFLOPS | 132 | 1009 TFLOPS | 1.71% | | 4096 | 32768 | 512 | 132 | 595 TFLOPS | 132 | 592 TFLOPS | -0.50% | | 4096 | 7168 | 16384 | 132 | 1348 TFLOPS | **128** | 1402 TFLOPS | 4.01% | | 4096 | 4096 | 7168 | 132 | 1325 TFLOPS | **128** | 1355 TFLOPS | 2.26% | | 4096 | 7168 | 2048 | 132 | 1026 TFLOPS | **128** | 1042 TFLOPS | 1.56% | Test on H100-SXM && CUDA 12.8. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-26 21:14:58 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#217
No description provided.