deepseek-ai/DeepEP
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00
Code Issues 308 Projects Packages Wiki Activity

[PR #732] Add unordered hybrid kernels for transports without ordered delivery (opt-in via EP_HYBRID_KERNEL) #738

New issue
Open
opened 2026-10-27 21:05:46 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-27 21:05:46 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/732
Author: @Xuan-1998
Created: 8/21/2026
Status: 🔄 Open

Base: main ← Head: unordered-kernels

📝 Commits (7)

  • 43d5a7a gin: add GIN context creation and signal layout
  • 3c34f91 dispatch: batch puts with in-band headers for unordered delivery
  • 4871aa5 combine: gate returns on counting signals for unordered delivery
  • 48ddfff feat: add EP_HYBRID_KERNEL toggle between ordered and unordered kernels
  • 5de2504 fix: resolve the GIN context count on the single-node path
  • 00b0697 feat(qp): clamp unordered num_allocated_qps to [2, 17] with a warning
  • 8d743bc docs(readme): document hybrid kernel variants and AWS EFA support

📊 Changes

21 files changed (+3230 additions, -88 deletions)

View changed files

📝 README.md (+23 -0)
📝 csrc/elastic/buffer.hpp (+146 -13)
📝 csrc/kernels/backend/api.cuh (+14 -0)
📝 csrc/kernels/backend/nccl.cu (+85 -5)
📝 csrc/kernels/elastic/combine.hpp (+80 -9)
📝 csrc/kernels/elastic/dispatch.hpp (+85 -2)
➕ csrc/kernels/elastic/kernel_select.hpp (+55 -0)
📝 csrc/python_api.cpp (+5 -0)
📝 deep_ep/buffers/elastic.py (+57 -13)
📝 deep_ep/include/deep_ep/common/comm.cuh (+78 -22)
📝 deep_ep/include/deep_ep/common/compiled.cuh (+18 -0)
➕ deep_ep/include/deep_ep/common/gin_resource_alloc.cuh (+170 -0)
📝 deep_ep/include/deep_ep/common/layout.cuh (+34 -6)
📝 deep_ep/include/deep_ep/common/ptx.cuh (+39 -0)
➕ deep_ep/include/deep_ep/common/qp_mapping.cuh (+123 -0)
📝 deep_ep/include/deep_ep/impls/combine_reduce_epilogue.cuh (+73 -14)
📝 deep_ep/include/deep_ep/impls/combine_utils.cuh (+41 -0)
➕ deep_ep/include/deep_ep/impls/hybrid_combine_unordered.cuh (+895 -0)
➕ deep_ep/include/deep_ep/impls/hybrid_dispatch_unordered.cuh (+1106 -0)
📝 deep_ep/include/deep_ep/impls/pp_send_recv.cuh (+14 -4)

...and 1 more files

📄 Description

The hybrid (scale-out) dispatch/combine kernels synchronize through a trailing tail signal: the receiver assumes that when the tail arrives, all data written before it has landed. This holds on transports with ordered delivery and VA/strong signal support (e.g. IB RC), but not on unordered transports such as AWS EFA (SRD), it can be overcome with proxy thread but will hurt the performance. This PR adds an alternative kernel pair that only needs weak signals, so DeepEP's hybrid mode can run on those transports.

The default behavior is unchanged: EP_HYBRID_KERNEL defaults to ordered, which runs the existing kernels under the existing communication configuration, untouched and under their original file names. Setting EP_HYBRID_KERNEL=unordered selects the new pair.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/732 **Author:** [@Xuan-1998](https://github.com/Xuan-1998) **Created:** 8/21/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `unordered-kernels` --- ### 📝 Commits (7) - [`43d5a7a`](https://github.com/deepseek-ai/DeepEP/commit/43d5a7ab3e5b3798805fb8d9adf3cd4a32edb803) gin: add GIN context creation and signal layout - [`3c34f91`](https://github.com/deepseek-ai/DeepEP/commit/3c34f919bdc3d8aed82c76e8d52383832a0df146) dispatch: batch puts with in-band headers for unordered delivery - [`4871aa5`](https://github.com/deepseek-ai/DeepEP/commit/4871aa5122c5f4416c931dab35739e08ab8dd763) combine: gate returns on counting signals for unordered delivery - [`48ddfff`](https://github.com/deepseek-ai/DeepEP/commit/48ddfff3a97b051f2af67caa3164a890c3fd1d27) feat: add EP_HYBRID_KERNEL toggle between ordered and unordered kernels - [`5de2504`](https://github.com/deepseek-ai/DeepEP/commit/5de25041447b853f4b75a21cbf3c02b85d68f0b9) fix: resolve the GIN context count on the single-node path - [`00b0697`](https://github.com/deepseek-ai/DeepEP/commit/00b069738883c090878ca89c63056e44fd191b0c) feat(qp): clamp unordered num_allocated_qps to [2, 17] with a warning - [`8d743bc`](https://github.com/deepseek-ai/DeepEP/commit/8d743bcc8b8ca6532f408b022f8e51db8384090b) docs(readme): document hybrid kernel variants and AWS EFA support ### 📊 Changes **21 files changed** (+3230 additions, -88 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+23 -0) 📝 `csrc/elastic/buffer.hpp` (+146 -13) 📝 `csrc/kernels/backend/api.cuh` (+14 -0) 📝 `csrc/kernels/backend/nccl.cu` (+85 -5) 📝 `csrc/kernels/elastic/combine.hpp` (+80 -9) 📝 `csrc/kernels/elastic/dispatch.hpp` (+85 -2) ➕ `csrc/kernels/elastic/kernel_select.hpp` (+55 -0) 📝 `csrc/python_api.cpp` (+5 -0) 📝 `deep_ep/buffers/elastic.py` (+57 -13) 📝 `deep_ep/include/deep_ep/common/comm.cuh` (+78 -22) 📝 `deep_ep/include/deep_ep/common/compiled.cuh` (+18 -0) ➕ `deep_ep/include/deep_ep/common/gin_resource_alloc.cuh` (+170 -0) 📝 `deep_ep/include/deep_ep/common/layout.cuh` (+34 -6) 📝 `deep_ep/include/deep_ep/common/ptx.cuh` (+39 -0) ➕ `deep_ep/include/deep_ep/common/qp_mapping.cuh` (+123 -0) 📝 `deep_ep/include/deep_ep/impls/combine_reduce_epilogue.cuh` (+73 -14) 📝 `deep_ep/include/deep_ep/impls/combine_utils.cuh` (+41 -0) ➕ `deep_ep/include/deep_ep/impls/hybrid_combine_unordered.cuh` (+895 -0) ➕ `deep_ep/include/deep_ep/impls/hybrid_dispatch_unordered.cuh` (+1106 -0) 📝 `deep_ep/include/deep_ep/impls/pp_send_recv.cuh` (+14 -4) _...and 1 more files_ </details> ### 📄 Description The hybrid (scale-out) dispatch/combine kernels synchronize through a trailing tail signal: the receiver assumes that when the tail arrives, all data written before it has landed. This holds on transports with ordered delivery and VA/strong signal support (e.g. IB RC), but not on unordered transports such as AWS EFA (SRD), it can be overcome with proxy thread but will hurt the performance. This PR adds an alternative kernel pair that only needs weak signals, so DeepEP's hybrid mode can run on those transports. The default behavior is unchanged: `EP_HYBRID_KERNEL` defaults to `ordered`, which runs the existing kernels under the existing communication configuration, untouched and under their original file names. Setting `EP_HYBRID_KERNEL=unordered` selects the new pair. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
hybrid-ep
chore/remove-clang-format-yapf
v2.5
nvDev
try-fix-cu128
mori-ep
antgroup-opt
compat-old-torch
add-linter
eager-rdma
v1.2.1
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepEP#738
No description provided.