[PR #635] [CLOSED] feat: Add RDMA-Only (Non-NVLink) Topology Support for ElasticBuffer Direct Mode #678

New issue

Closed

opened 2026-10-27 21:05:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:05:11 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/635
Author: @MengYu10151
Created: 5/19/2026
Status: ❌ Closed

Base: main ← Head: pcie

📝 Commits (3)

7b6ba19 Add PCIe-only topology support for ElasticBuffer
4f77720 Add RDMA-only (non-NVLink) topology support for ElasticBuffer
35fd265 Fix buffer size calculation for non-NVLink topology

📊 Changes

5 files changed (+55 additions, -15 deletions)

View changed files

📝 csrc/elastic/buffer.hpp (+17 -5)
📝 csrc/kernels/backend/api.cuh (+1 -0)
📝 csrc/kernels/backend/nccl.cu (+14 -6)
📝 deep_ep/buffers/elastic.py (+20 -4)
📝 deep_ep/utils/envs.py (+3 -0)

📄 Description

Motivation

DeepEP currently assumes NVLink connectivity between intra-node GPUs. On non-NVLink machines
(e.g., RTX professional series cards with IB RDMA only), NCCL's LSA domain grouping may report lsaSize > 1
even without real NVLink hardware, causing the hybrid dispatch kernel to crash (NVLink atomic
barriers fail without NVLink) and the NVLink connectivity check to block initialization.

This PR enables DeepEP to run on non-NVLink topologies under **direct mode only** — all ranks

communicate via RDMA without NVLink forwarding. Hybrid mode (hierarchical RDMA + NVLink) is
automatically disabled when no NVLink hardware is detected.

Changes

Detect NVLink presence via get_nvlink_gbs() at initialization, automatically selecting
the appropriate communication path based on hardware topology: NVLink machines use the
existing hybrid/direct mode logic, non-NVLink machines fall back to RDMA-only direct mode
Pass has_nvlink flag to C++ backend to override NCCL's LSA domain (num_nvl_ranks = 1),
compatible with the existing direct mode dispatch/combine kernel design
Fix division-by-zero in SM estimation when nvlink_traffic == 0
Guard NVLink-dependent paths (AGRS) and skip NVLink connectivity check on non-NVLink machines

Testing

Full test_ep.py suite (144 configs) passes on 1/2 node (RDMA-only, EP 1/2×8, direct mode)
1-node CX7 EP8 topology achieves ~53 GB/s dipsatch RDMA bandwidth;
2-node CX7 EP8*2 topology achieves ~48 GB/s dispatch RDMA bandwidth;
more benchmarks in progress

Notes

On RDMA-only topology, the dispatch kernel achieves high bandwidth utilization with very
few SMs(e.g., 8/12), leaving most SMs free for overlapping computation.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/635 **Author:** [@MengYu10151](https://github.com/MengYu10151) **Created:** 5/19/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `pcie` --- ### 📝 Commits (3) - [`7b6ba19`](https://github.com/deepseek-ai/DeepEP/commit/7b6ba19fa09f2f4ac4153168f689a63c2d06d6cd) Add PCIe-only topology support for ElasticBuffer - [`4f77720`](https://github.com/deepseek-ai/DeepEP/commit/4f77720db4a4db55200b431517bae07d580f1a76) Add RDMA-only (non-NVLink) topology support for ElasticBuffer - [`35fd265`](https://github.com/deepseek-ai/DeepEP/commit/35fd2654daacbf9627ef606cc47dcdf3f075570b) Fix buffer size calculation for non-NVLink topology ### 📊 Changes **5 files changed** (+55 additions, -15 deletions) <details> <summary>View changed files</summary> 📝 `csrc/elastic/buffer.hpp` (+17 -5) 📝 `csrc/kernels/backend/api.cuh` (+1 -0) 📝 `csrc/kernels/backend/nccl.cu` (+14 -6) 📝 `deep_ep/buffers/elastic.py` (+20 -4) 📝 `deep_ep/utils/envs.py` (+3 -0) </details> ### 📄 Description ### Motivation DeepEP currently assumes NVLink connectivity between intra-node GPUs. On non-NVLink machines (e.g., RTX professional series cards with IB RDMA only), NCCL's LSA domain grouping may report `lsaSize > 1` even without real NVLink hardware, causing the hybrid dispatch kernel to crash (NVLink atomic barriers fail without NVLink) and the NVLink connectivity check to block initialization. This PR enables DeepEP to run on non-NVLink topologies under **direct mode only** — all ranks communicate via RDMA without NVLink forwarding. Hybrid mode (hierarchical RDMA + NVLink) is automatically disabled when no NVLink hardware is detected. ### Changes - Detect NVLink presence via `get_nvlink_gbs()` at initialization, automatically selecting the appropriate communication path based on hardware topology: NVLink machines use the existing hybrid/direct mode logic, non-NVLink machines fall back to RDMA-only direct mode - Pass `has_nvlink` flag to C++ backend to override NCCL's LSA domain (`num_nvl_ranks = 1`), compatible with the existing direct mode dispatch/combine kernel design - Fix division-by-zero in SM estimation when `nvlink_traffic == 0` - Guard NVLink-dependent paths (AGRS) and skip NVLink connectivity check on non-NVLink machines ### Testing - Full `test_ep.py` suite (144 configs) passes on 1/2 node (RDMA-only, EP 1/2×8, direct mode) - 1-node CX7 EP8 topology achieves ~53 GB/s dipsatch RDMA bandwidth; - 2-node CX7 EP8*2 topology achieves ~48 GB/s dispatch RDMA bandwidth; - more benchmarks in progress ### Notes - On RDMA-only topology, the dispatch kernel achieves high bandwidth utilization with very few SMs(e.g., 8/12), leaving most SMs free for overlapping computation. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:05:11 +00:00