mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #464] Can deep_ep run on environments with more than 160 ranks? #300
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#300
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @ZhenguoYao1 on GitHub (Oct 21, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/464
I noticed a restriction in /csrc/deep_ep.cpp:
EP_HOST_ASSERT(0 <= rank && rank < num_ranks &&
(num_ranks <= NUM_MAX_NVL_PEERS * NUM_MAX_RDMA_PEERS || low_latency_mode));
where NUM_MAX_NVL_PEERS = 8 and NUM_MAX_RDMA_PEERS = 20. This implies the rank count cannot exceed 160 (8*20).
I tested this on a 24-node cluster, and the assertion was triggered. Therefore, my question is:
Does deep_ep actually support training on clusters with more than 20 nodes? Or is there any misunderstanding in my interpretation?
@sphish commented on GitHub (Oct 23, 2025):
Typically, a training job might run on more than 20 nodes, but the EP group size within that job usually does not exceed 160.
@goelayu commented on GitHub (Oct 30, 2025):
@ZhenguoYao1 Using gb200 support from https://github.com/fzyzcjy/DeepEP/tree/feat/dev_20250914, you can scale beyond 160.
@elvircrn commented on GitHub (Nov 11, 2025):
@goelayu Did you modify this
github.com/fzyzcjy/DeepEP@483f00af84/csrc/kernels/launch.cuh (L54)to scale up the ranks?