[GH-ISSUE #95] Why does low latency dispatch only use IBGDA? #87

New issue

Closed

opened 2026-10-26 09:54:00 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:54:00 +00:00

Owner

Originally created by @ltrcc on GitHub (Mar 27, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/95

The code is:
if (dst_rank != rank) {
nvshmemi_ibgda_put_nbi_warp(dst_ptr, src_ptr, num_bytes_per_msg, dst_rank, dst_expert_local_idx, lane_id, slot_idx);
} else {
const auto* src_int4_ptr = reinterpret_cast<const int4*>(src_ptr);
const auto* dst_int4_ptr = reinterpret_cast<int4*>(dst_ptr);
UNROLLED_WARP_COPY(8, lane_id, num_int4_per_msg, dst_int4_ptr, src_int4_ptr, ld_nc_global, st_na_global);
}

My issue is:
The code only judges whether dst_rank and rank are the same, and if they are not, it uses IBGDA for transmission. However, the situation where dst_rank and rank are not the same includes the case where they are within the same node. Would using NvLink instead of RDMA for transmission be faster when dst_rank and rank are at the same node?

Thanks for all the answers.

Originally created by @ltrcc on GitHub (Mar 27, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/95 **The code is:** `if (dst_rank != rank) {` ` nvshmemi_ibgda_put_nbi_warp(dst_ptr, src_ptr, num_bytes_per_msg, dst_rank, dst_expert_local_idx, lane_id, slot_idx);` `} else {` ` const auto* src_int4_ptr = reinterpret_cast<const int4*>(src_ptr);` ` const auto* dst_int4_ptr = reinterpret_cast<int4*>(dst_ptr);` ` UNROLLED_WARP_COPY(8, lane_id, num_int4_per_msg, dst_int4_ptr, src_int4_ptr, ld_nc_global, st_na_global);` `}` **My issue is:** The code only judges whether dst_rank and rank are the same, and if they are not, it uses IBGDA for transmission. However, the situation where dst_rank and rank are not the same includes the case where they are within the same node. Would using NvLink instead of RDMA for transmission be faster when dst_rank and rank are at the same node? **Thanks for all the answers.**

bsakaguchi closed this issue

2026-10-26 09:54:01 +00:00

bsakaguchi commented

2026-10-26 09:54:02 +00:00

Author

Owner

@LyricZhao commented on GitHub (Mar 28, 2025):

Would using NvLink instead of RDMA for transmission be faster when dst_rank and rank are at the same node?

Yes, but in-node traffic is just a little part of the whole, if the EP is large (for our online services); secondly, NVLink is of memory semantics, if you do that with NVLink, then it is conflict with hook-based overlapping.

@LyricZhao commented on GitHub (Mar 28, 2025): > Would using NvLink instead of RDMA for transmission be faster when dst_rank and rank are at the same node? Yes, but in-node traffic is just a little part of the whole, if the EP is large (for our online services); secondly, NVLink is of memory semantics, if you do that with NVLink, then it is conflict with hook-based overlapping.

bsakaguchi commented

2026-10-26 09:54:02 +00:00

Author

Owner

@ltrcc commented on GitHub (Mar 28, 2025):

Yes, but in-node traffic is just a little part of the whole, if the EP is large (for our online services); secondly, NVLink is of memory semantics, if you do that with NVLink, then it is conflict with hook-based overlapping.

Okay, I get it. Thank you very much for your answer

@ltrcc commented on GitHub (Mar 28, 2025): > Yes, but in-node traffic is just a little part of the whole, if the EP is large (for our online services); secondly, NVLink is of memory semantics, if you do that with NVLink, then it is conflict with hook-based overlapping. Okay, I get it. Thank you very much for your answer

bsakaguchi referenced this issue

2026-10-26 09:54:16 +00:00

[GH-ISSUE #106] how can I run the test across multi nodes？ #97