mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #93] Questions About Illegal Memory Access in DeepEP Low-Latency API Integration #85
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#85
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @JimmyWang1850 on GitHub (Mar 25, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/93
Subject:
We've successfully integrated DeepEP's low-latency into our LLM project deployed on H20 GPUs, but encountered an illegal memory access issue during CUDA graph execution. Below are our implementation details and observations:
Implementation Workflow
Called deep_ep.Buffer.get_low_latency_rdma_size_hint() to configure and initialize the low-latency buffer. num_max_dispatch_tokens_per_rank will be 244。
Created multiple CUDA graphs with batch sizes bs = [1, 2, 4, 8, ...] sharing the same DeepEP low-latency buffer. All operations use:
● buffer.low_latency_dispatch
● buffer.low_latency_combine
Note: num_max_dispatch_tokens_per_rank is set equal to batch size during dispatch.
Observed Issue
When executing the bs=1 CUDA graph, we encounter invalid token counts at this critical line:
while ((num_recv_tokens = ld_acquire_global(rdma_recv_count + local_expert_idx * num_ranks + src_rank)) == 0);at https://github.com/deepseek-ai/DeepEP/blob/main/csrc/kernels/internode_ll.cu#L263(Code Reference)
The num_recv_tokens variable receives corrupted values (e.g., 19xxxxxxx), indicating memory access violations. Notably:
●The issue only manifests when multiple CUDA graphs coexist
●Single CUDA graph configurations work correctly
Key Questions
Are there any known constraints when reusing low-latency buffers across multiple CUDA graphs?
The README mentions:
"you may need to restore some buffer status once you replay"
Could this imply required buffer state reset between graph replays? What specific buffer states require maintenance?
Should different batch sizes in concurrent graphs require separate buffer instances or explicit synchronization?
@LyricZhao commented on GitHub (Mar 26, 2025):
The core reason is that the
num_max_dispatch_tokens_per_rankis different for each graph, moved to WeChat discussion, close it now, feel free to reopen it.@Huoyuan100861 commented on GitHub (Mar 26, 2025):
Hello DeepEP author,
Thank you very much for providing such a high-performance EP communication library. I've encountered a similar issue and was wondering if it would be possible to exchange contact information for further discussion?
Thank you.
@LyricZhao commented on GitHub (Mar 26, 2025):
Sure, WeChat: LyricZ_THU