[PR #73] [CLOSED] Performance: Larger BlockTile optimizations enable 1470+ TF FP8 on the "H800"-SXM #230

New issue

Closed

opened 2026-10-26 21:15:06 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:15:06 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/73
Author: @sazczmh
Created: 3/25/2025
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (1)

46eb0d0 Performance: Larger BlockTile optimizations enable 1470+ TFLOPS FP8 performance on the H800-SXM platform

📊 Changes

5 files changed (+1009 additions, -185 deletions)

View changed files

📝 deep_gemm/include/deep_gemm/fp8_gemm.cuh (+728 -169)
📝 deep_gemm/include/deep_gemm/mma_utils.cuh (+131 -0)
📝 deep_gemm/include/deep_gemm/scheduler.cuh (+92 -0)
📝 deep_gemm/jit_kernels/gemm.py (+50 -12)
📝 deep_gemm/jit_kernels/m_grouped_gemm.py (+8 -4)

📄 Description

By leveraging Large BlockTile optimization to alleviate L2 cache pressure and maximize data reuse, the H800-SXM achieves peak FP8 compute performance of 1470+ TFLOPS.
@LyricZhao

Normal GEMMs for dense models

M	N	K	Base BMxBN	Computation	Opti BMxBN	Computation	Speedup
4096	24576	1536	128x128	999 TF	128x160	1166 TF	16.72%
4096	32768	512	128x128	591 TF	128x160	748 TF	26.57%
4096	7168	16384	128x128	1404 TF	128x160	1470 TF	4.70%
4096	7168	2048	128x128	1031 TF	128x160	1204 TF	16.78%

Grouped GEMMs for MoE models (contiguous layout)

Groups	M	N	K	Base BMxBN	Computation	Opti BMxBN	Computation	Speedup
4	8192	4096	7168	128x128	1317 TF	128x160	1381 TF	4.86%
4	8192	7168	2048	128x128	1114 TF	128x160	1262 TF	13.29%
8	4096	4096	7168	128x128	1317 TF	128x160	1383 TF	5.01%
8	4096	7168	2048	128x128	1107 TF	128x160	1259 TF	13.73%

Test on “H800”-SXM && CUDA 12.8.1

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/73 **Author:** [@sazczmh](https://github.com/sazczmh) **Created:** 3/25/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`46eb0d0`](https://github.com/deepseek-ai/DeepGEMM/commit/46eb0d08fb41d86e2c0c2dae4d896ad078fc82d8) Performance: Larger BlockTile optimizations enable 1470+ TFLOPS FP8 performance on the H800-SXM platform ### 📊 Changes **5 files changed** (+1009 additions, -185 deletions) <details> <summary>View changed files</summary> 📝 `deep_gemm/include/deep_gemm/fp8_gemm.cuh` (+728 -169) 📝 `deep_gemm/include/deep_gemm/mma_utils.cuh` (+131 -0) 📝 `deep_gemm/include/deep_gemm/scheduler.cuh` (+92 -0) 📝 `deep_gemm/jit_kernels/gemm.py` (+50 -12) 📝 `deep_gemm/jit_kernels/m_grouped_gemm.py` (+8 -4) </details> ### 📄 Description By leveraging Large BlockTile optimization to alleviate **L2 cache pressure** and **maximize data reuse**, the H800-SXM achieves peak FP8 compute performance of 1470+ TFLOPS. @LyricZhao ### Normal GEMMs for dense models | M | N | K | Base BMxBN | Computation | Opti BMxBN | Computation | Speedup | |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 4096 | 24576 | 1536 | 128x128 | 999 TF | 128x160 | 1166 TF | 16.72% | | 4096 | 32768 | 512 | 128x128 | 591 TF | 128x160 | 748 TF | **26.57**% | | 4096 | 7168 | 16384 | 128x128 | 1404 TF | 128x160 | **1470** TF | 4.70% | | 4096 | 7168 | 2048 | 128x128 | 1031 TF | 128x160 | 1204 TF | 16.78% | ### Grouped GEMMs for MoE models (contiguous layout) | Groups | M | N | K | Base BMxBN | Computation | Opti BMxBN | Computation | Speedup | |:----: |:----:|:-----:|:-----:|:-----: |:-----------: |:-----: |:-----------: |:-------:| | 4 | 8192 | 4096 | 7168 | 128x128 | 1317 TF | 128x160 | 1381 TF | 4.86% | | 4 | 8192 | 7168 | 2048 | 128x128 | 1114 TF | 128x160 | 1262 TF | 13.29% | | 8 | 4096 | 4096 | 7168 | 128x128 | 1317 TF | 128x160 | 1383 TF | 5.01% | | 8 | 4096 | 7168 | 2048 | 128x128 | 1107 TF | 128x160 | 1259 TF | 13.73% | Test on “H800”-SXM && CUDA 12.8.1 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:15:06 +00:00