[GH-ISSUE #22] Question about NVSHMEM and NVLink #24

New issue

Closed

opened 2026-10-26 09:52:41 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:52:41 +00:00

Owner

Originally created by @Puellaquae on GitHub (Feb 26, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/22

Hi, foremost, thanks for your excellent work.

I noticed that in normal kernel, intranode use NVLink and internode use NVSHMEM. However in low-latency mode both use NVSHEMEM. I guess that NVSHMEM might has better latency performance, though NVLink has better bandwidth performance; alternatively, using NVSHMEM both maybe has a better latency performance in overall than handling them separately. Unfortunately, I don't have any Hopper GPU with NVLink or IB for testing by myself.

Originally created by @Puellaquae on GitHub (Feb 26, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/22 Hi, foremost, thanks for your excellent work. I noticed that in normal kernel, intranode use NVLink and internode use NVSHMEM. However in low-latency mode both use NVSHEMEM. I guess that NVSHMEM might has better latency performance, though NVLink has better bandwidth performance; alternatively, using NVSHMEM both maybe has a better latency performance in overall than handling them separately. Unfortunately, I don't have any Hopper GPU with NVLink or IB for testing by myself.

bsakaguchi closed this issue

2026-10-26 09:52:41 +00:00

bsakaguchi commented

2026-10-26 09:52:42 +00:00

Author

Owner

@rubbberrabbit commented on GitHub (Feb 26, 2025):

不知道延迟是不是由dispatch的实现导致的，我猜是IB+NVLINK的通信方式增加了额外的延迟（但是导致internode通信量更小），而直接使用IB延迟更低

@rubbberrabbit commented on GitHub (Feb 26, 2025): 不知道延迟是不是由dispatch的实现导致的，我猜是IB+NVLINK的通信方式增加了额外的延迟（但是导致internode通信量更小），而直接使用IB延迟更低

bsakaguchi commented

2026-10-26 09:52:42 +00:00

Author

Owner

@LyricZhao commented on GitHub (Feb 26, 2025):

In V3 paper, we recommend settings like EP320. Considering only 8 NVL peers and 312 RDMA peers from the view of a single rank, we just simplify all connections into RDMA (even for local-node ranks). For the low-latency kernels are mainly designed for such cases.

Maybe later, we will add pure-NVLink low-latency kernels.

@LyricZhao commented on GitHub (Feb 26, 2025): In V3 paper, we recommend settings like EP320. Considering only 8 NVL peers and 312 RDMA peers from the view of a single rank, we just simplify all connections into RDMA (even for local-node ranks). For the low-latency kernels are mainly designed for such cases. Maybe later, we will add pure-NVLink low-latency kernels.

bsakaguchi referenced this issue

2026-10-26 09:53:39 +00:00

[GH-ISSUE #74] test_internode.py hangs at internode.cu:dispatch(...) #68

bsakaguchi referenced this issue

2026-10-26 09:56:29 +00:00

[GH-ISSUE #270] A timeout error occurred when running test_internode.py on four devices. #200

bsakaguchi referenced this issue

2026-10-26 09:56:40 +00:00

[GH-ISSUE #290] Running the test case on a 2-node H20 3E machine failed with internode.cu:1029 'unspecified launch failure' #208