deepseek-ai/DeepEP
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00
Code Issues 308 Projects Packages Wiki Activity

[GH-ISSUE #262] Question Regarding Dispatch Argument allocate_on_comm_stream #194

New issue
Closed
opened 2026-10-26 09:56:22 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 09:56:22 +00:00
Owner
Copy link

Originally created by @ZhiyiHu1999 on GitHub (Jun 26, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/262

Hello! I am not quite clear about the argument allocate_on_comm_stream, for which the docs said it is to 'control whether all the allocated tensors' ownership to be on the communication stream'.

Could you elaborate on the assertion here and why the previous_mode and 'async_mode' are both True if we set allocate_on_comm_stream to True.

github.com/deepseek-ai/DeepEP@ed3444bf9b/csrc/deep_ep.cpp (L740-L743)

Thanks a lot for the help!

Originally created by @ZhiyiHu1999 on GitHub (Jun 26, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/262 Hello! I am not quite clear about the argument `allocate_on_comm_stream`, for which the docs said it is to 'control whether all the allocated tensors' ownership to be on the communication stream'. Could you elaborate on the assertion here and why the `previous_mode` and 'async_mode' are both `True` if we set `allocate_on_comm_stream` to `True`. https://github.com/deepseek-ai/DeepEP/blob/ed3444bf9ba5da3a01150cb3546d343b6d6de36e/csrc/deep_ep.cpp#L740-L743 Thanks a lot for the help!
bsakaguchi commented 2026-10-26 09:56:23 +00:00
Author
Owner
Copy link

@LyricZhao commented on GitHub (Jun 27, 2025):

This allocate_on_comm_stream is for computation/communication overlapping. For overlapping settings, you must have something to wait (read the right data to send) and use async-finishing to make the computation stream not wait on the communication stream.

<!-- gh-comment-id:3011905451 --> @LyricZhao commented on GitHub (Jun 27, 2025): This `allocate_on_comm_stream` is for computation/communication overlapping. For overlapping settings, you must have something to wait (read the right data to send) and use async-finishing to make the computation stream not wait on the communication stream.
bsakaguchi commented 2026-10-26 09:56:23 +00:00
Author
Owner
Copy link

@LyricZhao commented on GitHub (Jun 27, 2025):

The ownership of the allocated tensors actually does not have direct relationship with overlapping. However, for our internal training, allocate_on_comm_stream=True must indicate overlapping, so we check previous_event.has_value() and async as a strong assertion, for debugging convenience of the training framework.

<!-- gh-comment-id:3011913052 --> @LyricZhao commented on GitHub (Jun 27, 2025): The ownership of the allocated tensors actually does not have direct relationship with overlapping. However, for our internal training, `allocate_on_comm_stream=True` must indicate overlapping, so we check `previous_event.has_value() and async` as a strong assertion, for debugging convenience of the training framework.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
hybrid-ep
chore/remove-clang-format-yapf
v2.5
nvDev
try-fix-cu128
mori-ep
antgroup-opt
compat-old-torch
add-linter
eager-rdma
v1.2.1
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepEP#194
No description provided.