deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[PR #81] [MERGED] Performance: BlockTile 256x128 optimizations enable 1500+ TF FP8 #237

New issue
Closed
opened 2026-10-26 21:15:11 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:15:11 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/81
Author: @sazczmh
Created: 4/8/2025
Status: ✅ Merged
Merged: 4/9/2025
Merged by: @LyricZhao

Base: main ← Head: blocktile-256x128

📝 Commits (9)

  • 97575bf Performance: BlockTile 256x128 optimizations enable 1500+ TFLOPS FP8 performance on the H800-SXM platform
  • ce65d5e Remove unused x256 WGMMA
  • 48a5f07 Clean up config heuristics
  • a6524d4 Larger block N candidates
  • 4c0cc29 Refactor M repetition with loops
  • bdca8b0 Fix indent
  • 5a80e4b Fix indent x2
  • a9967bc Update README
  • 989c9e3 Update README

📊 Changes

3 files changed (+97 additions, -64 deletions)

View changed files

📝 README.md (+11 -5)
📝 deep_gemm/include/deep_gemm/fp8_gemm.cuh (+75 -55)
📝 deep_gemm/jit_kernels/gemm.py (+11 -4)

📄 Description

By resuing the Accumulator registers of Tensor Cores to implement a 256x128 BlockTile structure, this approach significantly increases data reuse, reduces the demand for L2 Cache and HBM memory accesses, and enhances the SM's computational frequency, ultimately achieving FP8 performance exceeding 1,500+ TFLOPS.

M N K Base BMxBN Computation Opti BMxBN Computation Speedup
4096 24576 1536 128x160 1162 TF 256x128 1204 TF 3.61%
4096 32768 512 128x160 801 TF 256x128 777 TF -3.00%
4096 7168 16384 128x160 1451 TF 256x128 1500 TF 3.38%
4096 4096 7168 128x160 1304 TF 256x128 1377 TF 5.60%
4096 7168 2048 128x160 1185 TF 256x128 1159 TF -2.19%

Test on “H800”-SXM && CUDA 12.8.1

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/81 **Author:** [@sazczmh](https://github.com/sazczmh) **Created:** 4/8/2025 **Status:** ✅ Merged **Merged:** 4/9/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `blocktile-256x128` --- ### 📝 Commits (9) - [`97575bf`](https://github.com/deepseek-ai/DeepGEMM/commit/97575bf1c6009f7e8d1ae6388aeafcd8f57cf73a) Performance: BlockTile 256x128 optimizations enable 1500+ TFLOPS FP8 performance on the H800-SXM platform - [`ce65d5e`](https://github.com/deepseek-ai/DeepGEMM/commit/ce65d5e33c82185c6d13c27ab55b4b61b9c5c72c) Remove unused x256 WGMMA - [`48a5f07`](https://github.com/deepseek-ai/DeepGEMM/commit/48a5f071beca106a636e82f502ce7cd4d9201220) Clean up config heuristics - [`a6524d4`](https://github.com/deepseek-ai/DeepGEMM/commit/a6524d411a0eb6a45b715d7b562eab565ec2d304) Larger block N candidates - [`4c0cc29`](https://github.com/deepseek-ai/DeepGEMM/commit/4c0cc290c7114725bceaac9d2fd1518cde5324ed) Refactor M repetition with loops - [`bdca8b0`](https://github.com/deepseek-ai/DeepGEMM/commit/bdca8b06242c19278d4844301a997602397958fe) Fix indent - [`5a80e4b`](https://github.com/deepseek-ai/DeepGEMM/commit/5a80e4bb96167b62bbb6527157e353df07a87caa) Fix indent x2 - [`a9967bc`](https://github.com/deepseek-ai/DeepGEMM/commit/a9967bc27cdcead54e8d11e1736644eb51d49b33) Update README - [`989c9e3`](https://github.com/deepseek-ai/DeepGEMM/commit/989c9e3694638dd02aa36b4d8cb42339305caa85) Update README ### 📊 Changes **3 files changed** (+97 additions, -64 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+11 -5) 📝 `deep_gemm/include/deep_gemm/fp8_gemm.cuh` (+75 -55) 📝 `deep_gemm/jit_kernels/gemm.py` (+11 -4) </details> ### 📄 Description By resuing the Accumulator registers of Tensor Cores to implement a 256x128 BlockTile structure, this approach significantly increases data reuse, reduces the demand for L2 Cache and HBM memory accesses, and enhances the SM's computational frequency, ultimately achieving FP8 performance exceeding 1,500+ TFLOPS. | M | N | K | Base BMxBN | Computation | Opti BMxBN | Computation | Speedup | |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 4096 | 24576 | 1536 | 128x160 | 1162 TF | 256x128 | 1204 TF | 3.61% | | 4096 | 32768 | 512 | 128x160 | 801 TF | 256x128 | 777 TF | -3.00% | | 4096 | 7168 | 16384 | 128x160 | 1451 TF | 256x128 | 1500 TF | 3.38% | | 4096 | 4096 | 7168 | 128x160 | 1304 TF | 256x128 | 1377 TF | 5.60% | | 4096 | 7168 | 2048 | 128x160 | 1185 TF | 256x128 | 1159 TF | -2.19% | Test on “H800”-SXM && CUDA 12.8.1 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-26 21:15:11 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#237
No description provided.