deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[PR #383] [MERGED] [SM90] Add FP8 MegaMoE support #403

New issue
Closed
opened 2026-10-26 21:17:20 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:17:20 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/383
Author: @AichenF
Created: 7/16/2026
Status: ✅ Merged
Merged: 9/8/2026
Merged by: @RayWang96

Base: nv_dev ← Head: megamoe_sm90_fp8_dev

📝 Commits (1)

  • 788d5d9 Add SM90 FP8 MegaMoE support

📊 Changes

14 files changed (+5185 additions, -10 deletions)

View changed files

📝 README.md (+30 -0)
➕ csrc/apis/sm90_mega.hpp (+267 -0)
➕ csrc/jit_kernels/heuristics/sm90_mega_moe.hpp (+912 -0)
➕ csrc/jit_kernels/impls/sm90_fp8_mega_moe.hpp (+303 -0)
📝 csrc/python_api.cpp (+2 -0)
📝 deep_gemm/__init__.py (+4 -0)
📝 deep_gemm/include/deep_gemm/comm/barrier.cuh (+19 -10)
➕ deep_gemm/include/deep_gemm/comm/barrier_with_timeout_policy.cuh (+40 -0)
➕ deep_gemm/include/deep_gemm/impls/sm90_fp8_mega_moe.cuh (+2351 -0)
➕ deep_gemm/include/deep_gemm/layout/sm90_mega_moe.cuh (+75 -0)
➕ deep_gemm/include/deep_gemm/scheduler/sm90_mega_moe.cuh (+170 -0)
📝 deep_gemm/mega/__init__.py (+146 -0)
➕ tests/bench_mega_moe_sm90.py (+320 -0)
➕ tests/test_mega_moe_sm90.py (+546 -0)

📄 Description

co-author： @jychen21

This PR adds an SM90 FP8 MegaMoE kernel for Hopper GPUs, executing the expert MLP as L1 + SwiGLU followed by L2.

Benchmark Results

Speedup is calculated as DeepEP / MegaMoE - 1.

H200 High Throughput

Model M MegaMoE (us) DeepEP HT (us) Speedup
Flash 8 225.0 348.0 +54.7%
Flash 16 245.9 395.4 +60.8%
Flash 32 253.1 394.4 +55.8%
Flash 64 265.8 402.6 +51.5%
Flash 128 265.1 414.0 +56.2%
Flash 256 293.4 439.3 +49.7%
Flash 512 413.4 496.8 +20.2%
Flash 1024 637.3 705.0 +10.6%
Flash 2048 959.1 1208.0 +26.0%
Flash 4096 1739.0 2238.1 +28.7%
Flash 8192 3269.0 4317.6 +32.1%
Pro 8 636.6 801.8 +25.9%
Pro 16 778.0 946.4 +21.6%
Pro 32 822.9 1244.4 +51.2%
Pro 64 858.1 1022.8 +19.2%
Pro 128 871.6 1042.5 +19.6%
Pro 256 881.8 1085.2 +23.1%
Pro 512 1004.5 1178.7 +17.3%
Pro 1024 1392.7 1617.5 +16.1%
Pro 2048 2405.7 2678.6 +11.3%
Pro 4096 4217.0 4832.9 +14.6%
Pro 8192 7939.0 9207.9 +16.0%

H200 Low Latency

Model M MegaMoE (us) DeepEP LL (us) Speedup
Flash 8 225.0 264.0 +17.3%
Flash 16 245.9 398.4 +62.0%
Flash 32 253.1 405.2 +60.1%
Flash 64 265.8 309.1 +16.3%
Flash 128 265.1 419.7 +58.3%
Pro 8 636.6 712.5 +11.9%
Pro 16 778.0 850.7 +9.3%
Pro 32 822.9 911.3 +10.7%
Pro 64 858.1 976.9 +13.8%
Pro 128 871.6 948.2 +8.8%

H20 High Throughput

Model M MegaMoE (us) DeepEP HT (us) Speedup
Flash 8 273.1 566.7 +107.5%
Flash 16 304.4 619.2 +103.4%
Flash 32 302.0 618.6 +104.8%
Flash 64 340.7 622.6 +82.7%
Flash 128 414.4 638.5 +54.1%
Flash 256 569.5 688.7 +20.9%
Flash 512 922.0 1057.3 +14.7%
Flash 1024 1516.6 1983.4 +30.8%
Flash 2048 2735.1 3419.5 +25.0%
Flash 4096 5116.0 6087.7 +19.0%
Flash 8192 9749.0 11779.2 +20.8%
Pro 8 768.0 1349.2 +75.7%
Pro 16 950.3 1585.8 +66.9%
Pro 32 1026.3 1770.2 +72.5%
Pro 64 1059.9 1787.6 +68.7%
Pro 128 1201.0 1803.9 +50.2%
Pro 256 1639.9 1857.7 +13.3%
Pro 512 2599.0 2898.8 +11.5%
Pro 1024 4036.0 5412.7 +34.1%
Pro 2048 6986.0 8067.9 +15.5%
Pro 4096 12932.0 14614.0 +13.0%
Pro 8192 24777.0 28184.2 +13.8%

H20 Low Latency

Model M MegaMoE (us) DeepEP LL (us) Speedup
Flash 8 273.1 479.7 +75.7%
Flash 16 304.4 528.4 +73.6%
Flash 32 302.0 531.8 +76.1%
Flash 64 340.7 552.2 +62.1%
Flash 128 414.4 552.8 +33.4%
Pro 8 768.0 1265.6 +64.8%
Pro 16 950.3 1493.3 +57.1%
Pro 32 1026.3 1667.2 +62.4%
Pro 64 1059.9 1683.8 +58.9%
Pro 128 1201.0 1714.1 +42.7%

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/383 **Author:** [@AichenF](https://github.com/AichenF) **Created:** 7/16/2026 **Status:** ✅ Merged **Merged:** 9/8/2026 **Merged by:** [@RayWang96](https://github.com/RayWang96) **Base:** `nv_dev` ← **Head:** `megamoe_sm90_fp8_dev` --- ### 📝 Commits (1) - [`788d5d9`](https://github.com/deepseek-ai/DeepGEMM/commit/788d5d9d7ed8d23236897b7f2e82575c92f1ac4f) Add SM90 FP8 MegaMoE support ### 📊 Changes **14 files changed** (+5185 additions, -10 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+30 -0) ➕ `csrc/apis/sm90_mega.hpp` (+267 -0) ➕ `csrc/jit_kernels/heuristics/sm90_mega_moe.hpp` (+912 -0) ➕ `csrc/jit_kernels/impls/sm90_fp8_mega_moe.hpp` (+303 -0) 📝 `csrc/python_api.cpp` (+2 -0) 📝 `deep_gemm/__init__.py` (+4 -0) 📝 `deep_gemm/include/deep_gemm/comm/barrier.cuh` (+19 -10) ➕ `deep_gemm/include/deep_gemm/comm/barrier_with_timeout_policy.cuh` (+40 -0) ➕ `deep_gemm/include/deep_gemm/impls/sm90_fp8_mega_moe.cuh` (+2351 -0) ➕ `deep_gemm/include/deep_gemm/layout/sm90_mega_moe.cuh` (+75 -0) ➕ `deep_gemm/include/deep_gemm/scheduler/sm90_mega_moe.cuh` (+170 -0) 📝 `deep_gemm/mega/__init__.py` (+146 -0) ➕ `tests/bench_mega_moe_sm90.py` (+320 -0) ➕ `tests/test_mega_moe_sm90.py` (+546 -0) </details> ### 📄 Description co-author： @jychen21 This PR adds an SM90 FP8 MegaMoE kernel for Hopper GPUs, executing the expert MLP as L1 + SwiGLU followed by L2. ## Benchmark Results Speedup is calculated as `DeepEP / MegaMoE - 1`. ### H200 High Throughput | Model | M | MegaMoE (us) | DeepEP HT (us) | Speedup | |---|---:|---:|---:|---:| | Flash | 8 | 225.0 | 348.0 | +54.7% | | Flash | 16 | 245.9 | 395.4 | +60.8% | | Flash | 32 | 253.1 | 394.4 | +55.8% | | Flash | 64 | 265.8 | 402.6 | +51.5% | | Flash | 128 | 265.1 | 414.0 | +56.2% | | Flash | 256 | 293.4 | 439.3 | +49.7% | | Flash | 512 | 413.4 | 496.8 | +20.2% | | Flash | 1024 | 637.3 | 705.0 | +10.6% | | Flash | 2048 | 959.1 | 1208.0 | +26.0% | | Flash | 4096 | 1739.0 | 2238.1 | +28.7% | | Flash | 8192 | 3269.0 | 4317.6 | +32.1% | | Pro | 8 | 636.6 | 801.8 | +25.9% | | Pro | 16 | 778.0 | 946.4 | +21.6% | | Pro | 32 | 822.9 | 1244.4 | +51.2% | | Pro | 64 | 858.1 | 1022.8 | +19.2% | | Pro | 128 | 871.6 | 1042.5 | +19.6% | | Pro | 256 | 881.8 | 1085.2 | +23.1% | | Pro | 512 | 1004.5 | 1178.7 | +17.3% | | Pro | 1024 | 1392.7 | 1617.5 | +16.1% | | Pro | 2048 | 2405.7 | 2678.6 | +11.3% | | Pro | 4096 | 4217.0 | 4832.9 | +14.6% | | Pro | 8192 | 7939.0 | 9207.9 | +16.0% | ### H200 Low Latency | Model | M | MegaMoE (us) | DeepEP LL (us) | Speedup | |---|---:|---:|---:|---:| | Flash | 8 | 225.0 | 264.0 | +17.3% | | Flash | 16 | 245.9 | 398.4 | +62.0% | | Flash | 32 | 253.1 | 405.2 | +60.1% | | Flash | 64 | 265.8 | 309.1 | +16.3% | | Flash | 128 | 265.1 | 419.7 | +58.3% | | Pro | 8 | 636.6 | 712.5 | +11.9% | | Pro | 16 | 778.0 | 850.7 | +9.3% | | Pro | 32 | 822.9 | 911.3 | +10.7% | | Pro | 64 | 858.1 | 976.9 | +13.8% | | Pro | 128 | 871.6 | 948.2 | +8.8% | ### H20 High Throughput | Model | M | MegaMoE (us) | DeepEP HT (us) | Speedup | |---|---:|---:|---:|---:| | Flash | 8 | 273.1 | 566.7 | +107.5% | | Flash | 16 | 304.4 | 619.2 | +103.4% | | Flash | 32 | 302.0 | 618.6 | +104.8% | | Flash | 64 | 340.7 | 622.6 | +82.7% | | Flash | 128 | 414.4 | 638.5 | +54.1% | | Flash | 256 | 569.5 | 688.7 | +20.9% | | Flash | 512 | 922.0 | 1057.3 | +14.7% | | Flash | 1024 | 1516.6 | 1983.4 | +30.8% | | Flash | 2048 | 2735.1 | 3419.5 | +25.0% | | Flash | 4096 | 5116.0 | 6087.7 | +19.0% | | Flash | 8192 | 9749.0 | 11779.2 | +20.8% | | Pro | 8 | 768.0 | 1349.2 | +75.7% | | Pro | 16 | 950.3 | 1585.8 | +66.9% | | Pro | 32 | 1026.3 | 1770.2 | +72.5% | | Pro | 64 | 1059.9 | 1787.6 | +68.7% | | Pro | 128 | 1201.0 | 1803.9 | +50.2% | | Pro | 256 | 1639.9 | 1857.7 | +13.3% | | Pro | 512 | 2599.0 | 2898.8 | +11.5% | | Pro | 1024 | 4036.0 | 5412.7 | +34.1% | | Pro | 2048 | 6986.0 | 8067.9 | +15.5% | | Pro | 4096 | 12932.0 | 14614.0 | +13.0% | | Pro | 8192 | 24777.0 | 28184.2 | +13.8% | ### H20 Low Latency | Model | M | MegaMoE (us) | DeepEP LL (us) | Speedup | |---|---:|---:|---:|---:| | Flash | 8 | 273.1 | 479.7 | +75.7% | | Flash | 16 | 304.4 | 528.4 | +73.6% | | Flash | 32 | 302.0 | 531.8 | +76.1% | | Flash | 64 | 340.7 | 552.2 | +62.1% | | Flash | 128 | 414.4 | 552.8 | +33.4% | | Pro | 8 | 768.0 | 1265.6 | +64.8% | | Pro | 16 | 950.3 | 1493.3 | +57.1% | | Pro | 32 | 1026.3 | 1667.2 | +62.4% | | Pro | 64 | 1059.9 | 1683.8 | +58.9% | | Pro | 128 | 1201.0 | 1714.1 | +42.7% | --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-26 21:17:20 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#403
No description provided.