[GH-ISSUE #566] Feature request: Enable multi-NIC RDMA support for GPU = 1:2 (CX8) systems in Hybrid-EP / GIN #347

Open
opened 2026-10-26 09:58:56 +00:00 by bsakaguchi · 3 comments
Owner

Originally created by @skeyyk on GitHub (Jan 18, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/566

On the hybrid-ep branch, on B300 systems equipped with CX8 NICs, where each GPU is connected to two RDMA NICs, inter-node communication during training appears to utilize only one NIC per GPU. As a result, the effective RDMA bandwidth per GPU does not scale with the available NIC resources, even though the hardware topology supports concurrent usage of multiple NICs. This behavior is consistently observed on B300 + CX8 platforms across training runs.

Originally created by @skeyyk on GitHub (Jan 18, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/566 On the hybrid-ep branch, on B300 systems equipped with CX8 NICs, where each GPU is connected to two RDMA NICs, inter-node communication during training appears to utilize only one NIC per GPU. As a result, the effective RDMA bandwidth per GPU does not scale with the available NIC resources, even though the hardware topology supports concurrent usage of multiple NICs. This behavior is consistently observed on B300 + CX8 platforms across training runs.
Author
Owner

@Autumn1998 commented on GitHub (Jan 30, 2026):

Yes, this is a known issue for now. We are working on switching to a more mature backend instead of relying on the low-level API, and we will then try to support this case.

<!-- gh-comment-id:3822619199 --> @Autumn1998 commented on GitHub (Jan 30, 2026): Yes, this is a known issue for now. We are working on switching to a more mature backend instead of relying on the low-level API, and we will then try to support this case.
Author
Owner

@Brook017 commented on GitHub (Apr 2, 2026):

@Autumn1998 Does it mean underlying doca api will no longer be used?

<!-- gh-comment-id:4174922370 --> @Brook017 commented on GitHub (Apr 2, 2026): @Autumn1998 Does it mean underlying doca api will no longer be used?
Author
Owner

@Autumn1998 commented on GitHub (Apr 9, 2026):

We will still fix bugs, but there will be no new features.

<!-- gh-comment-id:4213037698 --> @Autumn1998 commented on GitHub (Apr 9, 2026): We will still fix bugs, but there will be no new features.
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepEP#347
No description provided.