mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #166] Why does DeepEP LL kernel use double send/recv/signal buffers? #142
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#142
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jammycc on GitHub (May 14, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/166
Hi,
I noticed that in the DeepEP LL kernel implementation, there are two sets of send/recv/signal buffers being maintained. However, it seems that during each dispatch/combine communication,the current buffer is used, while the next buffer is cleared. From my observation, both buffers are not used simultaneously.
Is there a specific reason for using double buffers here? Would creating a single buffer suffice, considering they are not active at the same time?
Appreciate any insights—thanks for your great work on this project!
@LyricZhao commented on GitHub (May 22, 2025):
For extreme low-latency consideration, we only use a "single-direction" signaling mechanism:
This is unsafe if the sender exits and launch the next dispatch/combine, which will rewrite the same buffer if the receiver still reading the buffer of the current dispatch/combine.
Add another direction signal will solve this and save half the VRAM, but the latency will be worse:
@LyricZhao commented on GitHub (May 22, 2025):
For double buffer, even considering 3 contiguous comm kernels, we can proof no buffer-rewrites.
@LyricZhao commented on GitHub (May 22, 2025):
Oh, "the current buffer is used, while the next buffer is cleared" is also important. Otherwise, cleaning should insert here:
With double buffer, the cleaning can be parallelized for lower latency.
@LyricZhao commented on GitHub (May 22, 2025):
Actually, the current online batch-size is large and such operations may be not importart anymore. But when we were designing the V3 family, the target TPS is high and such optimization is somehow worth doing if VRAM is enough.