mirror of
https://github.com/deepseek-ai/DeepEP.git
synced 2026-11-05 00:47:32 +00:00
[GH-ISSUE #123] Why is there no fp8 combine? #109
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepEP#109
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @thefacetakt on GitHub (Apr 15, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/123
Hi!
I was wondering if there why is there no fp8 combine method. Did you find it detrimental to training process even when performing intermediate accumulation in fp32?
@qijiaxing commented on GitHub (Apr 16, 2025):
is it because the MoE Grouped GEMM output in bf16?
@MichaelEk commented on GitHub (Apr 16, 2025):
BF16 combine communication is more expensive than FP8 quantization/dequantization kernels. Also, it is hard to overlap it during the forward pass.
@LyricZhao commented on GitHub (May 14, 2025):
Sorry for my late reply, please see comments in #161.
@thefacetakt commented on GitHub (May 14, 2025):
@LyricZhao, thanks for the detailed reply!