[GH-ISSUE #525] [Question] Question on `cpy_src_int4_ptr + offset_int4` #335

New issue

Closed

opened 2026-10-26 09:58:46 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:58:46 +00:00

Owner

Originally created by @cyhdmjzzy on GitHub (Dec 18, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/525

First, I would like to express my great admiration for the authors who have created such an excellent project.

About the code in csrc/kernels/internode_ll.cu, I have a question regarding line 870's cpy_src_int4_ptr + offset_int4. First, tma_load_1d
(or tma_load_and_arrive) is used to transfer kNumTMABufferBytes bytes of data from global
memory to shared memory for the current warp's next_stage_idx in a single TMA operation,
where kNumTMABufferBytes represents the int4 data for 32 threads (one warp) for one stage,
with each lane responsible for kNumSendUnrolls int4s. However, in offset_int4 = i + 32 * kNumSendUnrolls, i is lane-dependent (specifically, i = lane_id * kNumSendUnrolls at the
start of the loop). If elect_one_sync() at line 867 selects a thread in the warp that is
not lane=0, for example lane=2, then cpy_src_int4_ptr + offset_int4 was intended to offset
to the next stage's data for 32 threads (i.e., 32 * kNumSendUnrolls int4s), but it would
skip the kNumSendUnrolls int4s corresponding to lanes 0 and 1. How should this be
understood? Why is (i + 32 * kNumSendUnrolls) used to represent the data for
next_stage_idx that needs to be written from global memory to shared memory?

My personal humble opinion is that line 869 should perhaps be changed to the following to be more reasonable:

const auto& offset_int4 = (iter_idx + 1) * 32 * kNumSendUnrolls;

I believe I must be misunderstanding something, but I cannot figure it out. I very much look forward to your reply. Thank you very much.

Originally created by @cyhdmjzzy on GitHub (Dec 18, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/525 First, I would like to express my great admiration for the authors who have created such an excellent project. About the code in csrc/kernels/internode_ll.cu, I have a question regarding line 870's `cpy_src_int4_ptr + offset_int4`. First, `tma_load_1d` (or `tma_load_and_arrive`) is used to transfer `kNumTMABufferBytes` bytes of data from global memory to shared memory for the current warp's `next_stage_idx` in a single TMA operation, where `kNumTMABufferBytes` represents the int4 data for 32 threads (one warp) for one stage, with each lane responsible for `kNumSendUnrolls` int4s. However, in `offset_int4 = i + 32 * kNumSendUnrolls`, `i` is lane-dependent (specifically, `i = lane_id * kNumSendUnrolls` at the start of the loop). If `elect_one_sync()` at line 867 selects a thread in the warp that is not lane=0, for example lane=2, then `cpy_src_int4_ptr + offset_int4` was intended to offset to the next stage's data for 32 threads (i.e., `32 * kNumSendUnrolls` int4s), but it would skip the `kNumSendUnrolls` int4s corresponding to lanes 0 and 1. How should this be understood? Why is `(i + 32 * kNumSendUnrolls)` used to represent the data for `next_stage_idx` that needs to be written from global memory to shared memory? My personal humble opinion is that line 869 should perhaps be changed to the following to be more reasonable: ```cpp const auto& offset_int4 = (iter_idx + 1) * 32 * kNumSendUnrolls; ``` I believe I must be misunderstanding something, but I cannot figure it out. I very much look forward to your reply. Thank you very much.

bsakaguchi closed this issue

2026-10-26 09:58:46 +00:00

bsakaguchi commented

2026-10-26 09:58:47 +00:00

Author

Owner

@cyhdmjzzy commented on GitHub (Dec 19, 2025):

It turns out someone had asked this before: https://github.com/deepseek-ai/DeepEP/issues/359

@cyhdmjzzy commented on GitHub (Dec 19, 2025): It turns out someone had asked this before: https://github.com/deepseek-ai/DeepEP/issues/359

bsakaguchi commented

2026-10-26 09:58:47 +00:00

Author

Owner

@polarstormx commented on GitHub (Sep 18, 2026):

Closing as a duplicate of #359, as confirmed in the author's comment.

@polarstormx commented on GitHub (Sep 18, 2026): Closing as a duplicate of #359, as confirmed in [the author's comment](https://github.com/deepseek-ai/DeepEP/issues/525#issuecomment-3675085150).

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#335

No description provided.

Rows
Columns

[GH-ISSUE #525] [Question] Question on cpy_src_int4_ptr + offset_int4 #335

[GH-ISSUE #525] [Question] Question on `cpy_src_int4_ptr + offset_int4` #335