[GH-ISSUE #31] Why is a mechanism like PXN in NCCL not used in low-latency kernel? It should be able to achieve even lower latency. #30

New issue

Open

opened 2026-10-26 09:52:46 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-26 09:52:46 +00:00

Owner

Originally created by @weishenmezhememan on GitHub (Feb 28, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/31

NCCL provides the PXN mechanism. During cross-node communication, if the GPUs are not of the same type, data can first be forwarded within the node to a GPU of the same type and then transferred via RDMA to the target node's GPU. On multi-rail networks, this can reduce the number of switch hops, theoretically achieving lower latency. In low-latency kernels, what is the reason for not adopting a similar mechanism?

Originally created by @weishenmezhememan on GitHub (Feb 28, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/31 NCCL provides the PXN mechanism. During cross-node communication, if the GPUs are not of the same type, data can first be forwarded within the node to a GPU of the same type and then transferred via RDMA to the target node's GPU. On multi-rail networks, this can reduce the number of switch hops, theoretically achieving lower latency. In low-latency kernels, what is the reason for not adopting a similar mechanism?

bsakaguchi commented

2026-10-26 09:52:47 +00:00

Author

Owner

@sphish commented on GitHub (Feb 28, 2025):

In our experiments, network latency accounts for only a small portion of the total execution time. The overhead introduced by the control logic of the proposed PXN strategy might exceed the benefits gained from reduced network latency.

@sphish commented on GitHub (Feb 28, 2025): In our experiments, network latency accounts for only a small portion of the total execution time. The overhead introduced by the control logic of the proposed PXN strategy might exceed the benefits gained from reduced network latency.

bsakaguchi commented

2026-10-26 09:52:47 +00:00

Author

Owner

@sphish commented on GitHub (Mar 3, 2025):

In fact, the main bottleneck here is network bandwidth, and PXN's approach does not reduce the total amount of data transmitted.

@sphish commented on GitHub (Mar 3, 2025): In fact, the main bottleneck here is network bandwidth, and PXN's approach does not reduce the total amount of data transmitted.

bsakaguchi commented

2026-10-26 09:52:48 +00:00

Author

Owner

@weishenmezhememan commented on GitHub (Mar 3, 2025):

Why is the main bottleneck network bandwidth? The amount of data during the decoding phase of inference should be relatively small, so communication optimization should primarily focus on reducing latency, right? That's why a point-to-point direct sending approach is adopted, instead of the conventional all-to-all kernel. Could you help explain this further? Thanks!

@weishenmezhememan commented on GitHub (Mar 3, 2025): Why is the main bottleneck network bandwidth? The amount of data during the decoding phase of inference should be relatively small, so communication optimization should primarily focus on reducing latency, right? That's why a point-to-point direct sending approach is adopted, instead of the conventional all-to-all kernel. Could you help explain this further? Thanks!

bsakaguchi referenced this issue

2026-10-26 09:53:39 +00:00

[GH-ISSUE #74] test_internode.py hangs at internode.cu:dispatch(...) #68

bsakaguchi referenced this issue

2026-10-26 09:56:30 +00:00

[GH-ISSUE #270] A timeout error occurred when running test_internode.py on four devices. #200

bsakaguchi referenced this issue

2026-10-26 09:56:40 +00:00

[GH-ISSUE #290] Running the test case on a 2-node H20 3E machine failed with internode.cu:1029 'unspecified launch failure' #208