deepseek-ai/DeepEP

Fork 0

mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00

[GH-ISSUE #100] Why normal mode does not support AR? #92

New issue

Closed

opened 2026-10-26 09:54:08 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-26 09:54:08 +00:00

Owner

Originally created by @yejunhong1 on GitHub (Mar 31, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/100

Hi, DeepEP team,

I have two brief questions:

I wonder why normal mode not support AR?
It is because IBRC use Send operation that does not supper network out-of-order delivery?
Or is it because the queue managing fashion that has some issue with AR?
I am not very clear about the key contribution of normal mode.
Is it the low SM usage (but meanwhile provide high throughput)?
Or it can provide supreme throughput that existing frameworks cannot achieve (such as Pytorch alltoall based on NCCL's Send/Recv)?

Thanks.

Originally created by @yejunhong1 on GitHub (Mar 31, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/100 Hi, DeepEP team, I have two brief questions: 1. I wonder why normal mode not support AR? It is because IBRC use Send operation that does not supper network out-of-order delivery? Or is it because the queue managing fashion that has some issue with AR? 2. I am not very clear about the key contribution of normal mode. Is it the low SM usage (but meanwhile provide high throughput)? Or it can provide supreme throughput that existing frameworks cannot achieve (such as Pytorch alltoall based on NCCL's Send/Recv)? Thanks.

bsakaguchi closed this issue

2026-10-26 09:54:08 +00:00

bsakaguchi commented

2026-10-26 09:54:09 +00:00

Author

Owner

@LyricZhao commented on GitHub (Apr 1, 2025):

why normal mode not support AR?

Supported now (in PR #45).

Or it can provide supreme throughput that existing frameworks cannot achieve (such as Pytorch alltoall based on NCCL's Send/Recv)?

NCCL all2all does not have cross-domain multicast/reduction, e.g. a token is sent once via RDMA, broadcast multiple times to several NVL peers.

In terms of performance/precision, DeepEP's performance is also much better.

@LyricZhao commented on GitHub (Apr 1, 2025): > why normal mode not support AR? Supported now (in PR #45). > Or it can provide supreme throughput that existing frameworks cannot achieve (such as Pytorch alltoall based on NCCL's Send/Recv)? NCCL all2all does not have cross-domain multicast/reduction, e.g. a token is sent once via RDMA, broadcast multiple times to several NVL peers. In terms of performance/precision, DeepEP's performance is also much better.

bsakaguchi commented

2026-10-26 09:54:10 +00:00

Author

Owner

@yejunhong1 commented on GitHub (Apr 1, 2025):

Thanks for quick response.

As for the performance, our test result with NCCL alltoall (with DCQCN off) can also reach around 46GB, which is already the upper limit (same as DeepEP).
So I am a bit confused what is the improvement that DeepEP brings? Is it the SM resource reduction?

Meanwhile, I see that in low-latency mode, there is a quantification-to-FP8 before dispatch.
But I cannot find quantification related code in normal mode (maybe there is but not found).
Do you do quantification in normal mode? If so, the quantification may bring 2x modle throughput improvement under the same network bandwidth budget. I wonder if it is true.

@yejunhong1 commented on GitHub (Apr 1, 2025): Thanks for quick response. As for the performance, our test result with NCCL alltoall (with DCQCN off) can also reach around 46GB, which is already the upper limit (same as DeepEP). So I am a bit confused what is the improvement that DeepEP brings? Is it the SM resource reduction? Meanwhile, I see that in low-latency mode, there is a quantification-to-FP8 before dispatch. But I cannot find quantification related code in normal mode (maybe there is but not found). Do you do quantification in normal mode? If so, the quantification may bring 2x modle throughput improvement under the same network bandwidth budget. I wonder if it is true.

bsakaguchi commented

2026-10-26 09:54:10 +00:00

Author

Owner

@LyricZhao commented on GitHub (Apr 2, 2025):

So I am a bit confused what is the improvement that DeepEP brings?

As mentioned before: "cross-domain multicast: a token is sent once via RDMA, broadcast multiple times to several NVL peers". Using NCCL will cost more RDMA communication volumn (not performance cost), for V3 training (see the gate in V3 paper), nearly 2-3x.

But I cannot find quantification related code in normal mode (maybe there is but not found).

You should cast by yourself before launching the dispatch/combine kernel.

@LyricZhao commented on GitHub (Apr 2, 2025): > So I am a bit confused what is the improvement that DeepEP brings? As mentioned before: "cross-domain multicast: a token is sent once via RDMA, broadcast multiple times to several NVL peers". Using NCCL will cost more RDMA communication volumn (not performance cost), for V3 training (see the gate in V3 paper), nearly 2-3x. > But I cannot find quantification related code in normal mode (maybe there is but not found). You should cast by yourself before launching the dispatch/combine kernel.

bsakaguchi commented

2026-10-26 09:54:10 +00:00

Author

Owner

@yejunhong1 commented on GitHub (Apr 2, 2025):

I see. Thanks a lot.

@yejunhong1 commented on GitHub (Apr 2, 2025): I see. Thanks a lot.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#92

No description provided.

Rows
Columns