mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #78] Can data copies be optimized in low_latency kernel #70
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#70
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Thunderbrook on GitHub (Mar 18, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/78
Hi
in the low_latency_combine, the data should be copy from x to rdma_send_x because the x may not be registered. So can DeepEP expose the memory allocated by nvshmem through the API or provide an API to register user memory to avoid the data copy.
thanks
@LyricZhao commented on GitHub (Mar 18, 2025):
You are right, I will do it in this week. Thanks for feedback!
@LyricZhao commented on GitHub (Mar 18, 2025):
Emm, but it requires the previous kernel to support strided output. Anyway, I will expose an API in DeepEP firstly.
@LyricZhao commented on GitHub (Mar 18, 2025):
Supported in #79, combine issue/recv time reduces from 38 us to 30 us, thanks! Feel free to reopen the issue if you have other problems.
@ghghliu commented on GitHub (Jun 12, 2025):
buffer.get_next_low_latency_combine_buffer(handle)[:, :, :] = simulated_gemm_x. 你好，我理解这句话本身也会触发一次内存拷贝吧，请问这与在combine kernel中使用UNROLLED_WARP_COPY(7, lane_id, hidden_bf16_int4, buf_int4_ptr, x_int4, ld_nc_global, st_na_global)拷贝的差异是什么呢？