[GH-ISSUE #359] maybe bug in low latency combine tma ld/st #247

New issue

Closed

opened 2026-10-26 09:57:21 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:57:21 +00:00

Owner

Originally created by @ziannchen on GitHub (Aug 12, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/359

github.com/deepseek-ai/DeepEP@d9767ce05f/csrc/kernels/internode_ll.cu (L691-L695)
github.com/deepseek-ai/DeepEP@d9767ce05f/csrc/kernels/internode_ll.cu (L712-L716)

If the thread with lane_id != 0 wins the elect and enter this if branch, the offset used may be wrong ?

Originally created by @ziannchen on GitHub (Aug 12, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/359 https://github.com/deepseek-ai/DeepEP/blob/d9767ce05f67b4f8314d3f6ebd95846939256d0b/csrc/kernels/internode_ll.cu#L691-L695 https://github.com/deepseek-ai/DeepEP/blob/d9767ce05f67b4f8314d3f6ebd95846939256d0b/csrc/kernels/internode_ll.cu#L712-L716 If the thread with lane_id != 0 wins the elect and enter this if branch, the offset used may be wrong ?

bsakaguchi closed this issue

2026-10-26 09:57:21 +00:00

bsakaguchi commented

2026-10-26 09:57:23 +00:00

Author

Owner

@zheanxu commented on GitHub (Aug 13, 2025):

Through extensive testing, we have consistently observed that elect_one_sync(lane_id) always selects the thread with lane_id = 0. Furthermore, this implementation has been successfully deployed and tested in production environments, where it continues to run reliably. Based on these observations, we can reasonably conclude that elect_one_sync(lane_id) guarantees the selection of the thread with lane_id = 0.

@zheanxu commented on GitHub (Aug 13, 2025): Through extensive testing, we have consistently observed that `elect_one_sync(lane_id)` always selects the thread with `lane_id = 0`. Furthermore, this implementation has been successfully deployed and tested in production environments, where it continues to run reliably. Based on these observations, we can reasonably conclude that `elect_one_sync(lane_id)` guarantees the selection of the thread with `lane_id = 0`.

bsakaguchi commented

2026-10-26 09:57:23 +00:00

Author

Owner

@cyhdmjzzy commented on GitHub (Dec 19, 2025):

const auto& offset_int4 = i + 32 * kNumSendUnrolls
            = lane_id * kNumSendUnrolls + iter_idx * 32 * kNumSendUnrolls + 32 * kNumSendUnrolls
            = lane_id * kNumSendUnrolls + (iter_idx + 1) * 32 * kNumSendUnrolls

Wouldn't the lane_id * kNumSendUnrolls part cause a discrepancy?
elect_one_sync() always elects lane 0, so it's equivalent to:
const auto& offset_int4 = (iter_idx + 1) * 32 * kNumSendUnrolls

@cyhdmjzzy commented on GitHub (Dec 19, 2025): ``` const auto& offset_int4 = i + 32 * kNumSendUnrolls = lane_id * kNumSendUnrolls + iter_idx * 32 * kNumSendUnrolls + 32 * kNumSendUnrolls = lane_id * kNumSendUnrolls + (iter_idx + 1) * 32 * kNumSendUnrolls ``` Wouldn't the `lane_id * kNumSendUnrolls` part cause a discrepancy? `elect_one_sync()` always elects lane 0, so it's equivalent to: `const auto& offset_int4 = (iter_idx + 1) * 32 * kNumSendUnrolls`

bsakaguchi referenced this issue

2026-10-26 10:01:16 +00:00

[PR #247] [MERGED] Optimize intranode combine. #481