mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #450] HT Kernels with "--test-ll-compatibility" option #297
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#297
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @yatagaan on GitHub (Oct 10, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/450
The question is in the context of high throughput kernels with the option
--test-ll-compatibility.What is this compatibility checking? It seems that the compatibility of the HT kernel buffer is checked with LL kernel buffer. Is it in the context of prefill stage followed by a decode stage (in inference)? I understand that prefill will use HT kernels and decode will use decode kernels.
With this option, how is the NVSHMEM communicator built? For example, with 2 servers (with 8 GPUs each), do we expect to see 8 communicators built (one per NVLink rank)? We're seeing that this init method in runtime.cu gets called by all 16 GPUs. However, all GPUs on the first server have rank 0 (its' rdma_rank) and all GPUs on the second server have rank 1. This is explained in the figure below. This does not seem correct. Kindly clarify.
@sphish commented on GitHub (Oct 11, 2025):
test-ll-compatibilityis for verifying whether a DeepEP Buffer can switch between normal mode and low-latency mode.One is the low-latency mode, which creates a single NVSHMEM group containing all ranks. This mode supports both normal and low-latency kernels. This mode is suitable for fully connected networks.
The other is the non–low-latency mode, as shown in your diagram, which builds 8 NVSHMEM groups, each containing only the GPUs with the same index. This mode supports only normal kernels and does not support low-latency kernels, but it can operate in multi-plane network configurations.