mirror of
https://github.com/deepseek-ai/DeepGEMM.git
synced 2026-11-05 00:47:26 +00:00
[GH-ISSUE #57] How to use the output of DeepEP as the input of DeepGemm #35
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepGEMM#35
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @hpz4311 on GitHub (Mar 11, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/57
I think that after DeepEP, the activations received by expert 0 are (13, 7168).
2.In this case, calling m_grouped_gemm_fp8_fp8_bf16_nt_masked requires the activations to be arranged in the form of (4, 8, 7168), with masked_m=[1,2,3,7].
So, if we want to use the output of DeepEP as the input, do we need to manually rearrange the data from form 1 to form 2?
I have only run the DeepGemm demo and am not sure if I have misunderstood the output distribution of DeepEP .
@LyricZhao commented on GitHub (Mar 13, 2025):
Not exactly, it can be less than 13, as there can be some tokens selecting more than 1 expert.
Yes, for outputs of the normal kernels of DeepEP, you have to compute a layout and expand the tokens into the format required by the contiguous kernel in DeepGEMM.
For low-latency kernels' output of DeepEP, it can be directly used by the masked kernel in DeepGEMM (but not useful for training, small batch inference only).
@LyricZhao commented on GitHub (Mar 13, 2025):
We plan to open-source the small layout-related kernels later (as an optional utility C library), maybe next month. cc @xuzhean
@xutizhou commented on GitHub (Apr 19, 2025):
Hi @LyricZhao, do you have a timeline for open-sourcing the small layout-related kernels?
@LyricZhao commented on GitHub (Apr 21, 2025):
Sorry, we are still refactoring and optimizing the small and wgrad kernels (optimization took much more time as expected). Hard to tell a specific time, but at least before June.