[PR #74] [MERGED] Performance: Larger BlockTile optimizations enable 1470+ TF FP8 on the "H800"-SXM #231

New issue

Closed

opened 2026-10-26 21:15:07 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:15:07 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/74
Author: @sazczmh
Created: 3/25/2025
Status: ✅ Merged
Merged: 3/25/2025
Merged by: @LyricZhao

Base: main ← Head: larger-block

📝 Commits (10+)

46eb0d0 Performance: Larger BlockTile optimizations enable 1470+ TFLOPS FP8 performance on the H800-SXM platform
b922e64 Support block size 160
742fb1c Compilation-time GCD
7ffb118 Support multicasting on B
3497428 Minor fix
7768319 Remove unaligned predicates
046fab6 Fix grouped GEMM cases
612dd57 Simplify code
9c4f6f5 Optimize compilation speed
ddccb23 Fix NVCC branch divergence

📊 Changes

9 files changed (+270 additions, -71 deletions)

View changed files

📝 README.md (+10 -10)
📝 deep_gemm/include/deep_gemm/fp8_gemm.cuh (+53 -28)
📝 deep_gemm/include/deep_gemm/mma_utils.cuh (+131 -0)
📝 deep_gemm/include/deep_gemm/scheduler.cuh (+23 -10)
📝 deep_gemm/include/deep_gemm/utils.cuh (+5 -0)
📝 deep_gemm/jit/compiler.py (+1 -1)
📝 deep_gemm/jit/template.py (+4 -1)
📝 deep_gemm/jit_kernels/gemm.py (+31 -15)
📝 deep_gemm/jit_kernels/m_grouped_gemm.py (+12 -6)

📄 Description

By leveraging Large BlockTile optimization to alleviate L2 cache pressure and maximize data reuse, the H800-SXM achieves peak FP8 compute performance of 1470+ TFLOPS.
@LyricZhao

Normal GEMMs for dense models

M	N	K	Base BMxBN	Computation	Opti BMxBN	Computation	Speedup
4096	24576	1536	128x128	999 TF	128x160	1166 TF	16.72%
4096	32768	512	128x128	591 TF	128x160	748 TF	26.57%
4096	7168	16384	128x128	1404 TF	128x160	1470 TF	4.70%
4096	7168	2048	128x128	1031 TF	128x160	1204 TF	16.78%

Grouped GEMMs for MoE models (contiguous layout)

Groups	M	N	K	Base BMxBN	Computation	Opti BMxBN	Computation	Speedup
4	8192	4096	7168	128x128	1317 TF	128x160	1381 TF	4.86%
4	8192	7168	2048	128x128	1114 TF	128x160	1262 TF	13.29%
8	4096	4096	7168	128x128	1317 TF	128x160	1383 TF	5.01%
8	4096	7168	2048	128x128	1107 TF	128x160	1259 TF	13.73%

Test on “H800”-SXM && CUDA 12.8.1

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/74 **Author:** [@sazczmh](https://github.com/sazczmh) **Created:** 3/25/2025 **Status:** ✅ Merged **Merged:** 3/25/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `larger-block` --- ### 📝 Commits (10+) - [`46eb0d0`](https://github.com/deepseek-ai/DeepGEMM/commit/46eb0d08fb41d86e2c0c2dae4d896ad078fc82d8) Performance: Larger BlockTile optimizations enable 1470+ TFLOPS FP8 performance on the H800-SXM platform - [`b922e64`](https://github.com/deepseek-ai/DeepGEMM/commit/b922e64cb284bf258e71ae2cd86a6fb6ba00759e) Support block size 160 - [`742fb1c`](https://github.com/deepseek-ai/DeepGEMM/commit/742fb1c8a589e8f5ceffdcb3e71080dba380f516) Compilation-time GCD - [`7ffb118`](https://github.com/deepseek-ai/DeepGEMM/commit/7ffb118e5453898ef8daf6a9acf37be575db5156) Support multicasting on B - [`3497428`](https://github.com/deepseek-ai/DeepGEMM/commit/3497428a5e509644e476f19c52f836b426861aa1) Minor fix - [`7768319`](https://github.com/deepseek-ai/DeepGEMM/commit/7768319ffe68eaad45c0b507703f6a1a5a9607a4) Remove unaligned predicates - [`046fab6`](https://github.com/deepseek-ai/DeepGEMM/commit/046fab64b775966cd027f9aca774484c2241ba99) Fix grouped GEMM cases - [`612dd57`](https://github.com/deepseek-ai/DeepGEMM/commit/612dd57001878793ad8818797c6f84383f967e6c) Simplify code - [`9c4f6f5`](https://github.com/deepseek-ai/DeepGEMM/commit/9c4f6f53f57d9c00f6a32784bff719edca5039ab) Optimize compilation speed - [`ddccb23`](https://github.com/deepseek-ai/DeepGEMM/commit/ddccb230ca773179d3899abab54ff0fe54b36c26) Fix NVCC branch divergence ### 📊 Changes **9 files changed** (+270 additions, -71 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+10 -10) 📝 `deep_gemm/include/deep_gemm/fp8_gemm.cuh` (+53 -28) 📝 `deep_gemm/include/deep_gemm/mma_utils.cuh` (+131 -0) 📝 `deep_gemm/include/deep_gemm/scheduler.cuh` (+23 -10) 📝 `deep_gemm/include/deep_gemm/utils.cuh` (+5 -0) 📝 `deep_gemm/jit/compiler.py` (+1 -1) 📝 `deep_gemm/jit/template.py` (+4 -1) 📝 `deep_gemm/jit_kernels/gemm.py` (+31 -15) 📝 `deep_gemm/jit_kernels/m_grouped_gemm.py` (+12 -6) </details> ### 📄 Description By leveraging Large BlockTile optimization to alleviate **L2 cache pressure** and **maximize data reuse**, the H800-SXM achieves peak FP8 compute performance of 1470+ TFLOPS. @LyricZhao ### Normal GEMMs for dense models | M | N | K | Base BMxBN | Computation | Opti BMxBN | Computation | Speedup | |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 4096 | 24576 | 1536 | 128x128 | 999 TF | 128x160 | 1166 TF | 16.72% | | 4096 | 32768 | 512 | 128x128 | 591 TF | 128x160 | 748 TF | **26.57**% | | 4096 | 7168 | 16384 | 128x128 | 1404 TF | 128x160 | **1470** TF | 4.70% | | 4096 | 7168 | 2048 | 128x128 | 1031 TF | 128x160 | 1204 TF | 16.78% | ### Grouped GEMMs for MoE models (contiguous layout) | Groups | M | N | K | Base BMxBN | Computation | Opti BMxBN | Computation | Speedup | |:----: |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 4 | 8192 | 4096 | 7168 | 128x128 | 1317 TF | 128x160 | 1381 TF | 4.86% | | 4 | 8192 | 7168 | 2048 | 128x128 | 1114 TF | 128x160 | 1262 TF | 13.29% | | 8 | 4096 | 4096 | 7168 | 128x128 | 1317 TF | 128x160 | 1383 TF | 5.01% | | 8 | 4096 | 7168 | 2048 | 128x128 | 1107 TF | 128x160 | 1259 TF | 13.73% | Test on “H800”-SXM && CUDA 12.8.1 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:15:07 +00:00