[PR #673] [MERGED] [HybridEP] Add dense top-k routing scan path #696

New issue

Closed

opened 2026-10-27 21:05:22 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:05:22 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/673
Author: @harryzhou2000
Created: 6/29/2026
Status: ✅ Merged
Merged: 8/5/2026
Merged by: @jershi425

Base: hybrid-ep ← Head: hhanyu/hybrid-ep-dense-routing-scan-opt

📝 Commits (10+)

149ff76 Improve test_hybrid_ep.py: check_bitwise diagnostics, probs=False benchmarks, --only-bf16, permute/unpermute kernel profiling
62f7232 Optimize sparse prob communication
db9880b Fix sparse prob copy edge cases
7e65444 Add dense routing map for scan input
482a958 [HybridEP] Optimize dense scan local expert lookup
10358ab [HybridEP] Use rank bitsets in dense scan
88f1b6a [HybridEP] Standardize benchmark probability labels
18d401a [HybridEP] Test dense topk permute routing
a139d9e [HybridEP] Infer dense routing from topk indices
936e059 [HybridEP] Guard dense topk routing inference

📊 Changes

10 files changed (+801 additions, -363 deletions)

View changed files

📝 csrc/hybrid_ep/backend/hybrid_ep_backend.cuh (+355 -158)
📝 csrc/hybrid_ep/config.cuh (+18 -2)
📝 csrc/hybrid_ep/executor/executor.cu (+39 -9)
📝 csrc/hybrid_ep/extension/allgather.cu (+9 -4)
📝 csrc/hybrid_ep/extension/allgather.cuh (+2 -1)
📝 csrc/hybrid_ep/jit/compiler.cu (+5 -4)
📝 csrc/hybrid_ep/jit/compiler.cuh (+1 -1)
📝 csrc/hybrid_ep/pybind_hybrid_ep.cu (+1 -0)
📝 deep_ep/hybrid_ep_buffer.py (+136 -34)
📝 tests/test_hybrid_ep.py (+235 -150)

📄 Description

Summary

Add dense int16 top-k routing input for HybridEP metadata preprocessing.
Optimize dense scan with local-expert and rank bitsets.
Improve HybridEP test diagnostics and benchmark labels, including with/without-probs paths.

Performance

Primary impact is scan preprocessing: dense scan is significantly faster on the target dense top-k path. End-to-end dispatch/combine bandwidth is effectively unchanged versus upstream hybrid-ep in single-node B300 tests, with no observed regression.

Testing

Built on B300 with TORCH_CUDA_ARCH_LIST="10.0;10.3+PTX".
tests/test_hybrid_ep.py --num-processes 8 on B300 for default config.
Same test for H=512/E=8/K=8 and H=4096/E=8/K=8.
BF16 and FP8 correctness passed, including dense top-k scan and dense top-k scan+permute paths.
Re-ran updated test script against upstream hybrid-ep; dense-routing checks skip cleanly when unsupported.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/673 **Author:** [@harryzhou2000](https://github.com/harryzhou2000) **Created:** 6/29/2026 **Status:** ✅ Merged **Merged:** 8/5/2026 **Merged by:** [@jershi425](https://github.com/jershi425) **Base:** `hybrid-ep` ← **Head:** `hhanyu/hybrid-ep-dense-routing-scan-opt` --- ### 📝 Commits (10+) - [`149ff76`](https://github.com/deepseek-ai/DeepEP/commit/149ff76f7079d8c38c784e2e8361258476e4c32d) Improve test_hybrid_ep.py: check_bitwise diagnostics, probs=False benchmarks, --only-bf16, permute/unpermute kernel profiling - [`62f7232`](https://github.com/deepseek-ai/DeepEP/commit/62f7232f39035199de59ed1f6744f076b1507397) Optimize sparse prob communication - [`db9880b`](https://github.com/deepseek-ai/DeepEP/commit/db9880bd510d1a6647c0c1770e48159c327c4e37) Fix sparse prob copy edge cases - [`7e65444`](https://github.com/deepseek-ai/DeepEP/commit/7e6544479a8c974425a778811ea9c463ed3091a8) Add dense routing map for scan input - [`482a958`](https://github.com/deepseek-ai/DeepEP/commit/482a958c1b1d6c422b70631c657194399b8d2c13) [HybridEP] Optimize dense scan local expert lookup - [`10358ab`](https://github.com/deepseek-ai/DeepEP/commit/10358abe0172427ba944c891bae47940e6494028) [HybridEP] Use rank bitsets in dense scan - [`88f1b6a`](https://github.com/deepseek-ai/DeepEP/commit/88f1b6a47a477878ea6e12f366d33f66fffcd99b) [HybridEP] Standardize benchmark probability labels - [`18d401a`](https://github.com/deepseek-ai/DeepEP/commit/18d401a0d047b8028e379f9ba54efa17d479a300) [HybridEP] Test dense topk permute routing - [`a139d9e`](https://github.com/deepseek-ai/DeepEP/commit/a139d9e33abae5236798dd1c74fbc5e2d4b6f5f5) [HybridEP] Infer dense routing from topk indices - [`936e059`](https://github.com/deepseek-ai/DeepEP/commit/936e059c1caaaafb8a677235fb7daf2288f3b430) [HybridEP] Guard dense topk routing inference ### 📊 Changes **10 files changed** (+801 additions, -363 deletions) <details> <summary>View changed files</summary> 📝 `csrc/hybrid_ep/backend/hybrid_ep_backend.cuh` (+355 -158) 📝 `csrc/hybrid_ep/config.cuh` (+18 -2) 📝 `csrc/hybrid_ep/executor/executor.cu` (+39 -9) 📝 `csrc/hybrid_ep/extension/allgather.cu` (+9 -4) 📝 `csrc/hybrid_ep/extension/allgather.cuh` (+2 -1) 📝 `csrc/hybrid_ep/jit/compiler.cu` (+5 -4) 📝 `csrc/hybrid_ep/jit/compiler.cuh` (+1 -1) 📝 `csrc/hybrid_ep/pybind_hybrid_ep.cu` (+1 -0) 📝 `deep_ep/hybrid_ep_buffer.py` (+136 -34) 📝 `tests/test_hybrid_ep.py` (+235 -150) </details> ### 📄 Description ## Summary - Add dense int16 top-k routing input for HybridEP metadata preprocessing. - Optimize dense scan with local-expert and rank bitsets. - Improve HybridEP test diagnostics and benchmark labels, including with/without-probs paths. ## Performance Primary impact is scan preprocessing: dense scan is significantly faster on the target dense top-k path. End-to-end dispatch/combine bandwidth is effectively unchanged versus upstream hybrid-ep in single-node B300 tests, with no observed regression. ## Testing - Built on B300 with `TORCH_CUDA_ARCH_LIST="10.0;10.3+PTX"`. - `tests/test_hybrid_ep.py --num-processes 8` on B300 for default config. - Same test for H=512/E=8/K=8 and H=4096/E=8/K=8. - BF16 and FP8 correctness passed, including dense top-k scan and dense top-k scan+permute paths. - Re-ran updated test script against upstream hybrid-ep; dense-routing checks skip cleanly when unsupported. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:05:22 +00:00