[PR #432] [MERGED] Public Release 26/09 #442

New issue

Closed

opened 2026-10-26 21:17:55 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:17:55 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/432
Author: @zheanxu
Created: 9/10/2026
Status: ✅ Merged
Merged: 9/10/2026
Merged by: @zheanxu

Base: main ← Head: public-release-2609

📝 Commits (2)

ab69f76 Public release 26/09
39d8c4c Update News

📊 Changes

128 files changed (+12176 additions, -5113 deletions)

View changed files

📝 .gitmodules (+3 -3)
📝 CMakeLists.txt (+4 -2)
📝 README.md (+20 -22)
📝 csrc/apis/attention.hpp (+243 -45)
📝 csrc/apis/config.hpp (+17 -21)
📝 csrc/apis/einsum.hpp (+82 -37)
📝 csrc/apis/gemm.hpp (+236 -64)
📝 csrc/apis/hyperconnection.hpp (+1 -7)
📝 csrc/apis/layout.hpp (+6 -15)
➕ csrc/apis/mega_gate.hpp (+234 -0)
➕ csrc/apis/mega_mhc.hpp (+287 -0)
📝 csrc/apis/mega_moe.hpp (+11 -14)
📝 csrc/indexing/main.cu (+1 -1)
➖ csrc/jit/cache.hpp (+0 -31)
➖ csrc/jit/compiler.hpp (+0 -362)
➖ csrc/jit/device_runtime.hpp (+0 -138)
➖ csrc/jit/handle.hpp (+0 -222)
➖ csrc/jit/include_parser.hpp (+0 -80)
➖ csrc/jit/kernel_runtime.hpp (+0 -165)
📝 csrc/jit_kernels/heuristics/common.hpp (+6 -2)

...and 80 more files

📄 Description

This update introduces new kernels, features, and performance improvements:

New kernels

Sparse Indexer: hierarchical sparse indexing for V4.1, with throughput close to the Lightning Indexer.
Mega Gate: fuses the routing GEMM and multiple MoE routing operations into one kernel, with 30–75% speedups.
Mega mHC: fuses multiple Hyper-Connection operations and RMSNorm into one kernel, with 45–85% speedups.

New features

Integration with DeepJIT, a lightweight, header-only JIT runtime.
More epilogue fusion options, including runtime alpha scaling and fused output casting.
Broader use of cuBLASLt for simpler GEMM and batched operations.

Performance improvements

Faster MoE backward GEMMs, with ~20% gains.
Mega MoE optimizations for V4.1: ~15% faster at low latency, ~4% at high throughput.
Improved Lightning Indexer kernels, with a follow-up version (~30% faster) coming soon.

Breaking changes

Previously, all kernels were fully deterministic and batch-invariant.
This release adds a use_deterministic_algorithms() API, which defaults to False so some kernels can run faster.
Set it to True to restore deterministic, batch-invariant behavior for RL and similar workloads.

Contributors

DeepJIT Integration: @LyricZhao
Sparse Indexer, Mega mHC: @zheanxu
Mega Gate: @huahuaquq
GEMM and MoE Optimizations, Epilogue Fusion Options: @xyf007
Mega MoE optimizations for V4.1: @zheanxu
Lightning Indexer Improvements: @zheanxu @leavelet

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/432 **Author:** [@zheanxu](https://github.com/zheanxu) **Created:** 9/10/2026 **Status:** ✅ Merged **Merged:** 9/10/2026 **Merged by:** [@zheanxu](https://github.com/zheanxu) **Base:** `main` ← **Head:** `public-release-2609` --- ### 📝 Commits (2) - [`ab69f76`](https://github.com/deepseek-ai/DeepGEMM/commit/ab69f76be5bb9ea3499bc755002b1a876cb0b3d9) Public release 26/09 - [`39d8c4c`](https://github.com/deepseek-ai/DeepGEMM/commit/39d8c4cacc2c07c1fa9921c6c29c6a8a2da75359) Update News ### 📊 Changes **128 files changed** (+12176 additions, -5113 deletions) <details> <summary>View changed files</summary> 📝 `.gitmodules` (+3 -3) 📝 `CMakeLists.txt` (+4 -2) 📝 `README.md` (+20 -22) 📝 `csrc/apis/attention.hpp` (+243 -45) 📝 `csrc/apis/config.hpp` (+17 -21) 📝 `csrc/apis/einsum.hpp` (+82 -37) 📝 `csrc/apis/gemm.hpp` (+236 -64) 📝 `csrc/apis/hyperconnection.hpp` (+1 -7) 📝 `csrc/apis/layout.hpp` (+6 -15) ➕ `csrc/apis/mega_gate.hpp` (+234 -0) ➕ `csrc/apis/mega_mhc.hpp` (+287 -0) 📝 `csrc/apis/mega_moe.hpp` (+11 -14) 📝 `csrc/indexing/main.cu` (+1 -1) ➖ `csrc/jit/cache.hpp` (+0 -31) ➖ `csrc/jit/compiler.hpp` (+0 -362) ➖ `csrc/jit/device_runtime.hpp` (+0 -138) ➖ `csrc/jit/handle.hpp` (+0 -222) ➖ `csrc/jit/include_parser.hpp` (+0 -80) ➖ `csrc/jit/kernel_runtime.hpp` (+0 -165) 📝 `csrc/jit_kernels/heuristics/common.hpp` (+6 -2) _...and 80 more files_ </details> ### 📄 Description This update introduces new kernels, features, and performance improvements: ### New kernels - Sparse Indexer: hierarchical sparse indexing for V4.1, with throughput close to the Lightning Indexer. - Mega Gate: fuses the routing GEMM and multiple MoE routing operations into one kernel, with 30–75% speedups. - Mega mHC: fuses multiple Hyper-Connection operations and RMSNorm into one kernel, with 45–85% speedups. ### New features - Integration with [DeepJIT](https://github.com/deepseek-ai/DeepJIT), a lightweight, header-only JIT runtime. - More epilogue fusion options, including runtime alpha scaling and fused output casting. - Broader use of cuBLASLt for simpler GEMM and batched operations. ### Performance improvements - Faster MoE backward GEMMs, with ~20% gains. - Mega MoE optimizations for V4.1: ~15% faster at low latency, ~4% at high throughput. - Improved Lightning Indexer kernels, with a follow-up version (~30% faster) coming soon. ### Breaking changes - Previously, all kernels were fully deterministic and batch-invariant. - This release adds a `use_deterministic_algorithms()` API, which defaults to `False` so some kernels can run faster. - Set it to `True` to restore deterministic, batch-invariant behavior for RL and similar workloads. ### Contributors - DeepJIT Integration: @LyricZhao - Sparse Indexer, Mega mHC: @zheanxu - Mega Gate: @huahuaquq - GEMM and MoE Optimizations, Epilogue Fusion Options: @xyf007 - Mega MoE optimizations for V4.1: @zheanxu - Lightning Indexer Improvements: @zheanxu @leavelet --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:17:55 +00:00