[PR #574] Replace custom ibgda usage with nvshmem native apis #641

New issue

Open

opened 2026-10-27 21:04:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:04:48 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/574
Author: @seth-howell
Created: 1/27/2026
Status: 🔄 Open

Base: main ← Head: nvshmem_native_apis

📝 Commits (4)

10e8a3b Update internode kernels to use upstream NVSHMEM
35858c9 csrc/kernels:
75a1811 csrc: Only fence when the transport is unordered
5de69c1 check format

📊 Changes

11 files changed (+277 additions, -699 deletions)

View changed files

📝 csrc/config.hpp (+9 -9)
📝 csrc/deep_ep.cpp (+22 -12)
📝 csrc/deep_ep.hpp (+10 -5)
📝 csrc/kernels/api.cuh (+15 -11)
📝 csrc/kernels/configs.cuh (+0 -3)
➖ csrc/kernels/ibgda_device.cuh (+0 -506)
📝 csrc/kernels/internode.cu (+100 -80)
📝 csrc/kernels/internode_ll.cu (+86 -61)
📝 csrc/kernels/runtime.cu (+13 -2)
📝 csrc/kernels/utils.cuh (+4 -0)
📝 deep_ep/buffer.py (+18 -10)

📄 Description

This patch enables native NVSHMEM API usage in DeepEP

Eliminating the custom ibgda_device.cu file from DeepEP and switching to the new public nvshmem_qp APIs extends support to multiple transport backends including IBRC, IBDEVX, and libfabric.

This patch does not degrade performance on IBGDA. In some cases (2N, 4N) the kernels exhibit better performance than IBGDA.

It preserves IBGDA as the default transport unless a specific transport is supplied by the user.

Unordered transports like libfabric + EFA do require a fence before the put_signal operations to maintain functionality. Currently the performance of libfabric + EFA is not close to the performance of IB due to some missing functionality in the libfabric transport:

Multi-rail support. This is scheduled for NVSHMEM 3.6 and is actively under development by AWS engineers
The need for ordering between the put and signal operations with a fence which is costly and quiets the qpair (there is only 1). AWS engineers are aware of the issue and are looking at possible solutions in the NVSHMEM libfabric transport.

These improvements will be given to users for free in future releases of NVSHMEM.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/574 **Author:** [@seth-howell](https://github.com/seth-howell) **Created:** 1/27/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `nvshmem_native_apis` --- ### 📝 Commits (4) - [`10e8a3b`](https://github.com/deepseek-ai/DeepEP/commit/10e8a3b43e5726184b3fda6ae9b46ccfbb1074ce) Update internode kernels to use upstream NVSHMEM - [`35858c9`](https://github.com/deepseek-ai/DeepEP/commit/35858c97d26c0a91849ce330cb9134019a91df07) csrc/kernels: - [`75a1811`](https://github.com/deepseek-ai/DeepEP/commit/75a1811cf9c2ae057f8a28ab8438d0906c459e57) csrc: Only fence when the transport is unordered - [`5de69c1`](https://github.com/deepseek-ai/DeepEP/commit/5de69c1979510100ff86709a45089975f5f25e6b) check format ### 📊 Changes **11 files changed** (+277 additions, -699 deletions) <details> <summary>View changed files</summary> 📝 `csrc/config.hpp` (+9 -9) 📝 `csrc/deep_ep.cpp` (+22 -12) 📝 `csrc/deep_ep.hpp` (+10 -5) 📝 `csrc/kernels/api.cuh` (+15 -11) 📝 `csrc/kernels/configs.cuh` (+0 -3) ➖ `csrc/kernels/ibgda_device.cuh` (+0 -506) 📝 `csrc/kernels/internode.cu` (+100 -80) 📝 `csrc/kernels/internode_ll.cu` (+86 -61) 📝 `csrc/kernels/runtime.cu` (+13 -2) 📝 `csrc/kernels/utils.cuh` (+4 -0) 📝 `deep_ep/buffer.py` (+18 -10) </details> ### 📄 Description This patch enables native NVSHMEM API usage in DeepEP Eliminating the custom ibgda_device.cu file from DeepEP and switching to the new public nvshmem_qp APIs extends support to multiple transport backends including IBRC, IBDEVX, and libfabric. This patch does not degrade performance on IBGDA. In some cases (2N, 4N) the kernels exhibit better performance than IBGDA. It preserves IBGDA as the default transport unless a specific transport is supplied by the user. Unordered transports like libfabric + EFA do require a fence before the put_signal operations to maintain functionality. Currently the performance of libfabric + EFA is not close to the performance of IB due to some missing functionality in the libfabric transport: 1. Multi-rail support. This is scheduled for NVSHMEM 3.6 and is actively under development by AWS engineers 2. The need for ordering between the put and signal operations with a fence which is costly and quiets the qpair (there is only 1). AWS engineers are aware of the issue and are looking at possible solutions in the NVSHMEM libfabric transport. These improvements will be given to users for free in future releases of NVSHMEM. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-27 21:04:48 +00:00

bsakaguchi referenced this issue

2026-10-27 21:05:13 +00:00

[PR #641] [MERGED] Fix internode dispatch args #681