[PR #504] [MERGED] Optimization of Hybrid-EP #601

New issue

Closed

opened 2026-10-27 21:04:27 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:04:27 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/504
Author: @Autumn1998
Created: 11/27/2025
Status: ✅ Merged
Merged: 12/3/2025
Merged by: @jershi425

Base: hybrid-ep ← Head: tongliu_dev

📝 Commits (10+)

151f27e us sm copy flags
4d4cc12 Use ~/.deepep/hybrid_ep/jit as the jit path
17313f8 [BIG CHANGE] refactor permute, group 2 syncs into 1
c594d90 Add topo detection
c77e40c [TO FIX] Update hybrid ep version
7541f65 fix bug in scaling factor
25da0ca Jit different kernel in inter-node/intra-node case
b3359fd [TO FIX] dynamic seq len for RDMA
3fd405d disable RDMA dynamic token temporary
4ea5679 move sync to the point before dispatch

📊 Changes

21 files changed (+3032 additions, -789 deletions)

View changed files

📝 Hybrid-EP_Implementation.md (+35 -8)
➕ csrc/hybrid_ep/backend/NCCL_LICENSE.txt (+38 -0)
📝 csrc/hybrid_ep/backend/hybrid_ep_backend.cuh (+392 -192)
➕ csrc/hybrid_ep/backend/ibvcore.h (+221 -0)
➕ csrc/hybrid_ep/backend/topo_detection.cuh (+1657 -0)
📝 csrc/hybrid_ep/backend/utils.cuh (+50 -1)
📝 csrc/hybrid_ep/config.cuh (+13 -2)
📝 csrc/hybrid_ep/executor/executor.cu (+111 -87)
📝 csrc/hybrid_ep/executor/executor.cuh (+11 -8)
📝 csrc/hybrid_ep/extension/permute.cu (+189 -278)
📝 csrc/hybrid_ep/extension/permute.cuh (+55 -34)
📝 csrc/hybrid_ep/hybrid_ep.cu (+20 -21)
📝 csrc/hybrid_ep/hybrid_ep.cuh (+2 -4)
📝 csrc/hybrid_ep/internode.cu (+64 -43)
📝 csrc/hybrid_ep/internode.cuh (+8 -5)
📝 csrc/hybrid_ep/jit/compiler.cu (+36 -19)
📝 csrc/hybrid_ep/jit/compiler.cuh (+10 -8)
📝 csrc/hybrid_ep/pybind_hybrid_ep.cu (+15 -7)
📝 deep_ep/hybrid_ep_buffer.py (+54 -53)
📝 setup.py (+16 -1)

...and 1 more files

📄 Description

Add topo-detection for RDMA
Optimize the perf in small hidden/large EP
Use ~/.deepep/hybrid_ep/jit as the jit path
Group 2 sync into 1, and move sync point before the dispatch
dynamic seq len for RDMA
fix some bugs

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/504 **Author:** [@Autumn1998](https://github.com/Autumn1998) **Created:** 11/27/2025 **Status:** ✅ Merged **Merged:** 12/3/2025 **Merged by:** [@jershi425](https://github.com/jershi425) **Base:** `hybrid-ep` ← **Head:** `tongliu_dev` --- ### 📝 Commits (10+) - [`151f27e`](https://github.com/deepseek-ai/DeepEP/commit/151f27ef2a5fe4146492b8fe4c9d69acf0d06cdd) us sm copy flags - [`4d4cc12`](https://github.com/deepseek-ai/DeepEP/commit/4d4cc12fecd2e933828de42394f039f48b7a6dca) Use ~/.deepep/hybrid_ep/jit as the jit path - [`17313f8`](https://github.com/deepseek-ai/DeepEP/commit/17313f8e8b4ab2abfac70cf48eae96abd36806fc) [BIG CHANGE] refactor permute, group 2 syncs into 1 - [`c594d90`](https://github.com/deepseek-ai/DeepEP/commit/c594d90dba68dbf3297ee27f0a3bc14b8f6357d7) Add topo detection - [`c77e40c`](https://github.com/deepseek-ai/DeepEP/commit/c77e40cb06db65e7da1da85d85a58b9c324ce7ed) [TO FIX] Update hybrid ep version - [`7541f65`](https://github.com/deepseek-ai/DeepEP/commit/7541f653e7f576d57022d0feb44d40b84fd85c57) fix bug in scaling factor - [`25da0ca`](https://github.com/deepseek-ai/DeepEP/commit/25da0ca4dee3c730d298f149d4e8242a13d89214) Jit different kernel in inter-node/intra-node case - [`b3359fd`](https://github.com/deepseek-ai/DeepEP/commit/b3359fdf59fc54c8b0a7b9e05cd9ef65f7b86ec4) [TO FIX] dynamic seq len for RDMA - [`3fd405d`](https://github.com/deepseek-ai/DeepEP/commit/3fd405d03568b85d6b919484935486e23e2aaacb) disable RDMA dynamic token temporary - [`4ea5679`](https://github.com/deepseek-ai/DeepEP/commit/4ea5679ffbd0c77a3f5781e74dfce23e240f2a75) move sync to the point before dispatch ### 📊 Changes **21 files changed** (+3032 additions, -789 deletions) <details> <summary>View changed files</summary> 📝 `Hybrid-EP_Implementation.md` (+35 -8) ➕ `csrc/hybrid_ep/backend/NCCL_LICENSE.txt` (+38 -0) 📝 `csrc/hybrid_ep/backend/hybrid_ep_backend.cuh` (+392 -192) ➕ `csrc/hybrid_ep/backend/ibvcore.h` (+221 -0) ➕ `csrc/hybrid_ep/backend/topo_detection.cuh` (+1657 -0) 📝 `csrc/hybrid_ep/backend/utils.cuh` (+50 -1) 📝 `csrc/hybrid_ep/config.cuh` (+13 -2) 📝 `csrc/hybrid_ep/executor/executor.cu` (+111 -87) 📝 `csrc/hybrid_ep/executor/executor.cuh` (+11 -8) 📝 `csrc/hybrid_ep/extension/permute.cu` (+189 -278) 📝 `csrc/hybrid_ep/extension/permute.cuh` (+55 -34) 📝 `csrc/hybrid_ep/hybrid_ep.cu` (+20 -21) 📝 `csrc/hybrid_ep/hybrid_ep.cuh` (+2 -4) 📝 `csrc/hybrid_ep/internode.cu` (+64 -43) 📝 `csrc/hybrid_ep/internode.cuh` (+8 -5) 📝 `csrc/hybrid_ep/jit/compiler.cu` (+36 -19) 📝 `csrc/hybrid_ep/jit/compiler.cuh` (+10 -8) 📝 `csrc/hybrid_ep/pybind_hybrid_ep.cu` (+15 -7) 📝 `deep_ep/hybrid_ep_buffer.py` (+54 -53) 📝 `setup.py` (+16 -1) _...and 1 more files_ </details> ### 📄 Description 1. Add topo-detection for RDMA 2. Optimize the perf in small hidden/large EP 3. Use ~/.deepep/hybrid_ep/jit as the jit path 4. Group 2 sync into 1, and move sync point before the dispatch 5. dynamic seq len for RDMA 6. fix some bugs --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:04:27 +00:00