mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #285] [Question]: Why RDMA bandwidth can exceed 50GB/s #207
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#207
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @TianDi101 on GitHub (Jul 9, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/285
Hi folks, I found that the RDMA bandwidth in README.md can exceed 50GB/s, just what to ask how is this number measured? Is this actually algorithm bandwidth instead of bus bandwidth?
To my knowledge, for algorithm bandwidth we can compute with BW = total_bytes / duration and for bus bandwidth, we compute with BWrdma = rdma_total_bybtes / duration.
Please correct me if I'm wrong, thanks!
@TianDi101 commented on GitHub (Jul 9, 2025):
I just noticed that this PR https://github.com/deepseek-ai/DeepEP/pull/130 explains the calculation. However, in this case the EP16 performance is a little wierd since its bus bandwidth is only 43 * 0.5 = 21.5GB/s, far less than 50GB/s. I think under the setting of 4096 token per GPU, it should be able to saturate RDMA bandwidth.
@sphish commented on GitHub (Jul 10, 2025):
On the H800, the RDMA network bandwidth utilization is indeed not high in EP16 case. A main reason for this is that in our EP16 test case, when the RDMA bandwidth reaches 43 GB/s, the NVLink bandwidth will exceeds 140 GB/s, making NVLink the bottleneck.
With GPUs that have higher NVLink bandwidth, you'd get better results.
@TianDi101 commented on GitHub (Jul 14, 2025):
Thanks for the explanation, could you please elaborate on this more?
I'm thinking in this way, for each GPU in the EP16 case, the inbound RDMA bandwidth is 50GB/s while the outbound NVLink bandwidth is 160GB/s, seems that no matter how the algorithm is designed, RDMA bandwidth is always the bottleneck, right?
This is under the assumption that 8 GPUs in a node don't compete for NVLink bandwidth, that being said, each GPU will always get 160GB/s for each direction. But I'm not sure if this assumption is right.
@sphish commented on GitHub (Jul 15, 2025):
Since the tokens received from RDMA may be forwarded multiple times via NVLink, RDMA is not always the bottleneck. he NVLink 160GB/s bandwidth is also just a ideal upper bound.
Based on previous feedback and test results from the Tencent team, testing with H20 servers that have higher NVLink bandwidth shows significantly better performance in the EP16 case, which leads me to believe that this case is NVLink-bottlenecked.