[PR #386] Optimize sm100 version with token-aware BM240 tiling, native UE8M0, and direct combine #406

New issue

Open

opened 2026-10-26 21:17:20 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:17:20 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/386
Author: @qinqinwo
Created: 7/17/2026
Status: 🔄 Open

Base: main ← Head: perf/sm100-megamoe-b200

📝 Commits (1)

df4c356 perf: optimize SM100 MegaMoE on B200

📊 Changes

7 files changed (+446 additions, -22 deletions)

View changed files

➕ SM100_MEGAMOE_B200.md (+103 -0)
📝 csrc/apis/mega.hpp (+18 -5)
📝 csrc/jit_kernels/heuristics/mega_moe.hpp (+65 -9)
📝 csrc/jit_kernels/impls/sm100_fp8_fp4_mega_moe.hpp (+98 -1)
📝 deep_gemm/include/deep_gemm/common/math.cuh (+19 -0)
📝 deep_gemm/include/deep_gemm/impls/sm100_fp8_fp4_mega_moe.cuh (+128 -1)
📝 deep_gemm/include/deep_gemm/layout/mega_moe.cuh (+15 -6)

📄 Description

For DeepSeek-V4-Pro:
The wide sweep from 128 to 32768 tokens per rank has a 1.0206x geometric-mean
speedup. The only measured regression is 4096 tokens per rank at -0.03%,
which is within run-to-run noise.

Tokens/rank	Speedup	Improvement
128	1.0065x	+0.65%
256	1.0033x	+0.33%
512	1.0172x	+1.72%
768	1.0363x	+3.63%
1024	1.0515x	+5.15%
2048	1.0446x	+4.46%
4096	0.9997x	-0.03%
8192	1.0077x	+0.77%
16384	1.0208x	+2.08%
32768	1.0196x	+1.96%

Across the union of 24 unique token counts, the geometric-mean speedup is
1.0382x (3.82%). Final-source digest checks at 832, 1792, 8192, and 32768
tokens per rank produced bitwise-identical outputs for all 32 rank pairs,
covering the small/mid, BM240, and direct-combine paths. The broader
masked/unmasked regression sweep across small, mid, and BM240 regimes was
also bitwise identical for all 96 tested rank pairs.

(coauthor with agent)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/386 **Author:** [@qinqinwo](https://github.com/qinqinwo) **Created:** 7/17/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `perf/sm100-megamoe-b200` --- ### 📝 Commits (1) - [`df4c356`](https://github.com/deepseek-ai/DeepGEMM/commit/df4c356132a69642067cc6a20f167f9d7bfdcc36) perf: optimize SM100 MegaMoE on B200 ### 📊 Changes **7 files changed** (+446 additions, -22 deletions) <details> <summary>View changed files</summary> ➕ `SM100_MEGAMOE_B200.md` (+103 -0) 📝 `csrc/apis/mega.hpp` (+18 -5) 📝 `csrc/jit_kernels/heuristics/mega_moe.hpp` (+65 -9) 📝 `csrc/jit_kernels/impls/sm100_fp8_fp4_mega_moe.hpp` (+98 -1) 📝 `deep_gemm/include/deep_gemm/common/math.cuh` (+19 -0) 📝 `deep_gemm/include/deep_gemm/impls/sm100_fp8_fp4_mega_moe.cuh` (+128 -1) 📝 `deep_gemm/include/deep_gemm/layout/mega_moe.cuh` (+15 -6) </details> ### 📄 Description For DeepSeek-V4-Pro: The wide sweep from 128 to 32768 tokens per rank has a 1.0206x geometric-mean speedup. The only measured regression is 4096 tokens per rank at -0.03%, which is within run-to-run noise. | Tokens/rank | Speedup | Improvement | |---:|---:|---:| | 128 | 1.0065x | +0.65% | | 256 | 1.0033x | +0.33% | | 512 | 1.0172x | +1.72% | | 768 | 1.0363x | +3.63% | | 1024 | 1.0515x | +5.15% | | 2048 | 1.0446x | +4.46% | | 4096 | 0.9997x | -0.03% | | 8192 | 1.0077x | +0.77% | | 16384 | 1.0208x | +2.08% | | 32768 | 1.0196x | +1.96% | Across the union of 24 unique token counts, the geometric-mean speedup is 1.0382x (3.82%). Final-source digest checks at 832, 1792, 8192, and 32768 tokens per rank produced bitwise-identical outputs for all 32 rank pairs, covering the small/mid, BM240, and direct-combine paths. The broader masked/unmasked regression sweep across small, mid, and BM240 regimes was also bitwise identical for all 96 tested rank pairs. (coauthor with agent) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 21:17:20 +00:00

bsakaguchi referenced this issue

2026-10-26 21:17:37 +00:00

[PR #406] Add SM120 heuristic calibration harness #423