[GH-ISSUE #129] Why do we reduce topk weights for each token? #113

New issue

Closed

opened 2026-10-26 09:54:31 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-26 09:54:31 +00:00

Owner

Originally created by @dearsxx0918 on GitHub (Apr 21, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/129

Hi,
I'm quite confused by the source code here.
github.com/deepseek-ai/DeepEP@a84a24808f/csrc/kernels/intranode.cu (L772)
github.com/deepseek-ai/DeepEP@a84a24808f/csrc/kernels/internode.cu (L1261)

These implementations indicate that we have reduced TOPK weights. But as far as I know, we only need to reduce the weight of the combined tokens, so for each token, we only need to reduce it to one instead of TOPK.

Can you help he on this?

Best regards,
-Edda

Originally created by @dearsxx0918 on GitHub (Apr 21, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/129 Hi, I'm quite confused by the source code here. https://github.com/deepseek-ai/DeepEP/blob/a84a24808fb0ea732f49b874cc456a69dde69076/csrc/kernels/intranode.cu#L772 https://github.com/deepseek-ai/DeepEP/blob/a84a24808fb0ea732f49b874cc456a69dde69076/csrc/kernels/internode.cu#L1261 These implementations indicate that we have reduced TOPK weights. But as far as I know, we only need to reduce the weight of the combined tokens, so for each token, we only need to reduce it to one instead of TOPK. Can you help he on this? Best regards, -Edda

bsakaguchi closed this issue

2026-10-26 09:54:31 +00:00

bsakaguchi commented

2026-10-26 09:54:32 +00:00

Author

Owner

@LyricZhao commented on GitHub (Apr 22, 2025):

This is for dispatch backward, the gradient needs to be reduced.

@LyricZhao commented on GitHub (Apr 22, 2025): This is for dispatch **backward**, the gradient needs to be reduced.

bsakaguchi commented

2026-10-26 09:54:32 +00:00

Author

Owner

@dearsxx0918 commented on GitHub (Apr 22, 2025):

Thanks for your quick reply. I mean why we need reduce weights? Seems we can get this from topk_weights directly ?
And if we need reduce weights, we reduce all topk_weights into one ?

@dearsxx0918 commented on GitHub (Apr 22, 2025): Thanks for your quick reply. I mean why we need reduce weights? Seems we can get this from topk_weights directly ? And if we need reduce weights, we reduce all topk_weights into one ?

bsakaguchi commented

2026-10-26 09:54:32 +00:00

Author

Owner

@LyricZhao commented on GitHub (Apr 23, 2025):

Oh, you are right. It seems that we only need to copy the weights needed, instead of copying 👍🏻

Will add this as a TODO later.

@LyricZhao commented on GitHub (Apr 23, 2025): Oh, you are right. It seems that we only need to copy the weights needed, instead of copying 👍🏻 Will add this as a TODO later.

bsakaguchi commented

2026-10-26 09:54:32 +00:00

Author

Owner

@dearsxx0918 commented on GitHub (Apr 23, 2025):

Thanks for you reply!

@dearsxx0918 commented on GitHub (Apr 23, 2025): Thanks for you reply!

bsakaguchi referenced this issue

2026-10-26 09:59:08 +00:00

[GH-ISSUE #587] Why doesn't low-latency dispatch use system-level fence before issuing IBGDA writes? #360