mirror of
https://github.com/deepseek-ai/DeepGEMM.git
synced 2026-11-05 00:47:26 +00:00
[GH-ISSUE #23] how to deal with shapes like 7, 9 ? and how to deal with linears with bias ? #18
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepGEMM#18
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @artetaout on GitHub (Feb 27, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/23
when input_len is 7, how to do ?
when forward a linear with bias, how to deal with it ?
@LyricZhao commented on GitHub (Feb 28, 2025):
The normal GEMM natively supports
M == 7, there is an issue in the test and now has been fixed ingithub.com/deepseek-ai/DeepGEMM@b69f630b91.For grouped GEMM kernels, only masked layout supports such shapes.
No bias support currently.
@Li-dongyang commented on GitHub (Feb 28, 2025):
Could you explain the rationale behind the bug fix @LyricZhao ? Is this related to the copy operations in the code? Thanks!
@LyricZhao commented on GitHub (Feb 28, 2025):
This is just related to an assertion
assert lhs_scales.shape == (m, (k + 127) // 128). The previous utillity PyTorch functionget_col_major_tma_aligned_tensorfailed to make a proper shape after transposing for TMA alignment.You can run the test using the commit before/after for comparison.