mirror of
https://github.com/deepseek-ai/DeepGEMM.git
synced 2026-11-05 00:47:26 +00:00
[GH-ISSUE #33] Use col-major tensor for DeepGemm #20
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepGEMM#20
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @PuALGuo on GitHub (Mar 3, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/33
It is said that the A tensor should be col-major. and to meet the requirement of TMA, it should do some padding or transposition.
'''
LHS scales must be transposed for TMA load, but not for RHS scales
lhs_scales = get_col_major_tma_aligned_tensor(lhs_scales)
'''
Is there any reason why we don't choose the row-major? It seems that the alignment for K is more easier to meet.
Thanks a lot!
@soundOfDestiny commented on GitHub (Mar 4, 2025):
This is because each stage does a GEMM with K_BLOCK_SIZE=128, so each stage only reads one scale in the K direction. If we were to use row-major layout, TMA would only read one scale (4 bytes) of continuous memory at a time, resulting in very low reading efficiency.