[GH-ISSUE #445] Networking requirements(latency, bandwidth, throughput) for EP vs TP? #292

New issue

Closed

opened 2026-10-26 09:58:07 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-26 09:58:07 +00:00

Owner

Originally created by @terrificdm on GitHub (Oct 10, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/445

I'm curious about the differences in networking requirements between EP and TP, particularly regarding latency, bandwidth, and throughput. I know that both distribution strategies are highly demanding in terms of networking performance, but I'm not sure which one has higher requirements from latency, bandwidth, and throughput perspectives. Could someone elaborate on this a little for me?

I also know that NCCL wasn't well-optimized for all-to-all which EP requires, and that's why DeepEP was introduced. But why can NCCL meet TP's requirements? Is it because NCCL is well-suited for allreduce, or is it because TP is more tolerant of networking limitations compared to EP?

Thanks.

Originally created by @terrificdm on GitHub (Oct 10, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/445 I'm curious about the differences in networking requirements between EP and TP, particularly regarding latency, bandwidth, and throughput. I know that both distribution strategies are highly demanding in terms of networking performance, but I'm not sure which one has higher requirements from latency, bandwidth, and throughput perspectives. Could someone elaborate on this a little for me? I also know that NCCL wasn't well-optimized for all-to-all which EP requires, and that's why DeepEP was introduced. But why can NCCL meet TP's requirements? Is it because NCCL is well-suited for allreduce, or is it because TP is more tolerant of networking limitations compared to EP? Thanks.

bsakaguchi closed this issue

2026-10-26 09:58:07 +00:00

bsakaguchi commented

2026-10-26 09:58:08 +00:00

Author

Owner

@sphish commented on GitHub (Oct 11, 2025):

TP essentially consists of two communication patterns: allgather and reduce-scatter, both of which NCCL has optimized very effectively.

Both EP and TP communication patterns have extremely high latency and bandwidth requirements. In fact, because NCCL’s latency optimization is not particularly strong, in many low-latency scenarios (primarily inference), people may replace NCCL with custom communication kernels.

@sphish commented on GitHub (Oct 11, 2025): TP essentially consists of two communication patterns: **allgather** and **reduce-scatter**, both of which NCCL has optimized very effectively. Both EP and TP communication patterns have extremely high latency and bandwidth requirements. In fact, because NCCL’s latency optimization is not particularly strong, in many low-latency scenarios (primarily inference), people may replace NCCL with custom communication kernels.

bsakaguchi commented

2026-10-26 09:58:09 +00:00

Author

Owner

@terrificdm commented on GitHub (Oct 12, 2025):

TP essentially consists of two communication patterns: allgather and reduce-scatter, both of which NCCL has optimized very effectively.

Both EP and TP communication patterns have extremely high latency and bandwidth requirements. In fact, because NCCL’s latency optimization is not particularly strong, in many low-latency scenarios (primarily inference), people may replace NCCL with custom communication kernels.

Well noted. Thank you for your clarification.

@terrificdm commented on GitHub (Oct 12, 2025): > TP essentially consists of two communication patterns: **allgather** and **reduce-scatter**, both of which NCCL has optimized very effectively. > > Both EP and TP communication patterns have extremely high latency and bandwidth requirements. In fact, because NCCL’s latency optimization is not particularly strong, in many low-latency scenarios (primarily inference), people may replace NCCL with custom communication kernels. Well noted. Thank you for your clarification.

bsakaguchi commented

2026-10-26 09:58:09 +00:00

Author

Owner

@polarstormx commented on GitHub (Sep 18, 2026):

Closing as answered, as the author confirmed that the explanation clarified the question.

@polarstormx commented on GitHub (Sep 18, 2026): Closing as answered, as [the author confirmed](https://github.com/deepseek-ai/DeepEP/issues/445#issuecomment-3393871238) that the explanation clarified the question.

bsakaguchi referenced this issue

2026-10-27 21:03:23 +00:00

[PR #292] [MERGED] Explicitly destroy the C++ runtime and release resources. #509