[PR #130] [MERGED] Support multi-QP for normal kernels #440

New issue

Closed

opened 2026-10-26 10:00:47 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 10:00:47 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/130
Author: @LyricZhao
Created: 4/22/2025
Status: ✅ Merged
Merged: 4/22/2025
Merged by: @LyricZhao

Base: main ← Head: trmt/internode_multi_qp

📝 Commits (9)

5ab80c2 In the Internode Normal Kernel, when using nvshmem ibrc for RDMA data transmission, a single QP is used for data transfer between two GPUs, which limits kernel performance in network card dual-port and RoCE network scenarios.
e2c5784 Revert ibgda_device.cuh and remove some comments.
e0eaaf9 Add the performance data after internode optimization in the Readme file
c07fdd1 Merge branch 'trmt/internode_multi_qp' of github.com:deepseek-ai/DeepEP into trmt/internode_multi_qp
20b2aaa Refactor some code.
3e54b78 Normal kernels always use IBGDA mode.
edbb1bc Several code lints
3b1045d Fix the performance data.
e255d57 Use put_nbi_warp.

📊 Changes

7 files changed (+82 additions, -62 deletions)

View changed files

📝 README.md (+4 -2)
📝 csrc/kernels/ibgda_device.cuh (+17 -11)
📝 csrc/kernels/internode.cu (+38 -26)
📝 csrc/kernels/internode_ll.cu (+1 -1)
📝 csrc/kernels/runtime.cu (+5 -7)
📝 deep_ep/buffer.py (+11 -12)
📝 tests/test_internode.py (+6 -3)

📄 Description

This PR is authored by Tencent Network Platform Department. Thanks for the contribution! Now normal kernels have a huge speedup:

Type	Dispatch #EP	Bottleneck bandwidth	Combine #EP	Bottleneck bandwidth
Internode	32	44 👉🏻 58 GB/s (RDMA)	32	47 👉🏻 57 GB/s (RDMA)
Internode	64	46 👉🏻 51 GB/s (RDMA)	64	45 👉🏻 50 GB/s (RDMA)

Through in-depth optimization, the following enhancements have been implemented in the internode normal kernels.

Replacing IBRC with IBGDA
Utilizing distinct QPs (Queue Pairs) per channel for parallel data transmission

These improvements not only enhance the robustness of the internode normal kernels in scenarios involving dual-port NICs and RoCE networks but also further elevate communication performance.

NOTES: the bandwidth in the table is algorithm bandwidth, for physical bandwidth, e.g. 58 * 3/4 = 43.5.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/130 **Author:** [@LyricZhao](https://github.com/LyricZhao) **Created:** 4/22/2025 **Status:** ✅ Merged **Merged:** 4/22/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `trmt/internode_multi_qp` --- ### 📝 Commits (9) - [`5ab80c2`](https://github.com/deepseek-ai/DeepEP/commit/5ab80c28f3d6c3e4f88ce236f427ab7c81025172) In the Internode Normal Kernel, when using nvshmem ibrc for RDMA data transmission, a single QP is used for data transfer between two GPUs, which limits kernel performance in network card dual-port and RoCE network scenarios. - [`e2c5784`](https://github.com/deepseek-ai/DeepEP/commit/e2c578485cb36dc9e08958cf3213525a6ffd7d05) Revert `ibgda_device.cuh` and remove some comments. - [`e0eaaf9`](https://github.com/deepseek-ai/DeepEP/commit/e0eaaf94fbedd700c83d3a635290ba49047c1404) Add the performance data after internode optimization in the Readme file - [`c07fdd1`](https://github.com/deepseek-ai/DeepEP/commit/c07fdd197cbfe03d25da73da02f563e99748652f) Merge branch 'trmt/internode_multi_qp' of github.com:deepseek-ai/DeepEP into trmt/internode_multi_qp - [`20b2aaa`](https://github.com/deepseek-ai/DeepEP/commit/20b2aaaf9e47ddeb910e1a31affb19ffc2cb6022) Refactor some code. - [`3e54b78`](https://github.com/deepseek-ai/DeepEP/commit/3e54b78fd776108d04e959e6988e96f62d0314d8) Normal kernels always use IBGDA mode. - [`edbb1bc`](https://github.com/deepseek-ai/DeepEP/commit/edbb1bc3ffc73d33a612ad254b6b75fc904ba222) Several code lints - [`3b1045d`](https://github.com/deepseek-ai/DeepEP/commit/3b1045db43ed232691d32dde0517dfed571b0a65) Fix the performance data. - [`e255d57`](https://github.com/deepseek-ai/DeepEP/commit/e255d57befba7452f4677c31677e03eb6ad96711) Use `put_nbi_warp`. ### 📊 Changes **7 files changed** (+82 additions, -62 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+4 -2) 📝 `csrc/kernels/ibgda_device.cuh` (+17 -11) 📝 `csrc/kernels/internode.cu` (+38 -26) 📝 `csrc/kernels/internode_ll.cu` (+1 -1) 📝 `csrc/kernels/runtime.cu` (+5 -7) 📝 `deep_ep/buffer.py` (+11 -12) 📝 `tests/test_internode.py` (+6 -3) </details> ### 📄 Description This PR is authored by **Tencent Network Platform Department**. Thanks for the contribution! Now normal kernels have a huge speedup: | Type | Dispatch #EP | Bottleneck bandwidth | Combine #EP | Bottleneck bandwidth | |:---------:|:------------:|:--------------------:|:-----------:|:--------------------:| | Internode | 32 | 44 👉🏻 58 GB/s (RDMA) | 32 | 47 👉🏻 57 GB/s (RDMA) | | Internode | 64 | 46 👉🏻 51 GB/s (RDMA) | 64 | 45 👉🏻 50 GB/s (RDMA) | Through in-depth optimization, the following enhancements have been implemented in the internode normal kernels. - Replacing IBRC with IBGDA - Utilizing distinct QPs (Queue Pairs) per channel for parallel data transmission These improvements not only enhance the robustness of the internode normal kernels in scenarios involving dual-port NICs and RoCE networks but also further elevate communication performance. NOTES: the bandwidth in the table is algorithm bandwidth, for physical bandwidth, e.g. `58 * 3/4 = 43.5`. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 10:00:47 +00:00