[GH-ISSUE #164] Is internode api necessary in low_latency mode with a single 8*H200 node? #143

New issue

Closed

opened 2026-10-26 09:55:11 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:55:11 +00:00

Owner

Originally created by @Yi-sir on GitHub (May 14, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/164

I deployed DeepSeek-R1 using DeepEP on an 8×H200 server. Due to incompatibility between the 'normal' mode interface and CUDA Graph(https://github.com/sgl-project/sglang/issues/6218), I had to use 'low_latency' mode. However, the 'low_latency' mode appears to utilize internode communication interfaces, which rely on an InfiniBand (IB) environment. Currently, our machine’s IB environment is not yet ready.

For this single-machine deployment with 'low_latency' mode, is the IB environment (or specifically, the internode interfaces) still mandatory?

Thank you!

Originally created by @Yi-sir on GitHub (May 14, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/164 I deployed DeepSeek-R1 using DeepEP on an 8×H200 server. Due to incompatibility between the 'normal' mode interface and CUDA Graph(https://github.com/sgl-project/sglang/issues/6218), I had to use 'low_latency' mode. However, the 'low_latency' mode appears to utilize internode communication interfaces, which rely on an InfiniBand (IB) environment. Currently, our machine’s IB environment is not yet ready. For this single-machine deployment with 'low_latency' mode, is the IB environment (or specifically, the internode interfaces) still mandatory? Thank you!

bsakaguchi closed this issue

2026-10-26 09:55:11 +00:00

bsakaguchi commented

2026-10-26 09:55:12 +00:00

Author

Owner

@LyricZhao commented on GitHub (May 14, 2025):

is the IB environment (or specifically, the internode interfaces) still mandatory?

Yes, the local node transfer will also use RDMA.

For NVLink impl, we currently don't have time to do. You can refer to the PPLX kernels, their kernels support that: https://github.com/ppl-ai/pplx-kernels.

@LyricZhao commented on GitHub (May 14, 2025): > is the IB environment (or specifically, the internode interfaces) still mandatory? Yes, the local node transfer will also use RDMA. For NVLink impl, we currently don't have time to do. You can refer to the PPLX kernels, their kernels support that: https://github.com/ppl-ai/pplx-kernels.

bsakaguchi commented

2026-10-26 09:55:12 +00:00

Author

Owner

@Yi-sir commented on GitHub (May 14, 2025):

is the IB environment (or specifically, the internode interfaces) still mandatory?

Yes, the local node transfer will also use RDMA.

For NVLink impl, we currently don't have time to do. You can refer to the PPLX kernels, their kernels support that: https://github.com/ppl-ai/pplx-kernels.

thank you

@Yi-sir commented on GitHub (May 14, 2025): > > is the IB environment (or specifically, the internode interfaces) still mandatory? > > Yes, the local node transfer will also use RDMA. > > For NVLink impl, we currently don't have time to do. You can refer to the PPLX kernels, their kernels support that: https://github.com/ppl-ai/pplx-kernels. thank you

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#143

No description provided.

Rows
Columns