[GH-ISSUE #97] 在特定参数下，group_gemm_mask会报错Cuda error #52

New issue

Closed

opened 2026-10-26 21:11:19 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:11:19 +00:00

Owner

Originally created by @risemeup1 on GitHub (May 13, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/97

`def test_m_grouped_gemm_masked() -> None:
print('Testing grouped masked GEMM:')

for num_groups, m in ((2, 1536),):
    for k, n in ((2048, 6144),):
        # Test correctness
        masked_m_candidates = list(filter(lambda candidate: candidate <= m, (64, 128, 192, 256, 320, 384)))
        for i in range(10):
            x_fp8, y_fp8, out, ref_out = construct_masked_grouped(num_groups, m, k, n)
            masked_m = torch.empty((num_groups, ), device='cuda', dtype=torch.int)
            for j in range(num_groups):
                masked_m[j] = random.choice(masked_m_candidates)
            expected_m = min(int(masked_m.float().mean()) + 1, m)
            deep_gemm.m_grouped_gemm_fp8_fp8_bf16_nt_masked(x_fp8, y_fp8, out, torch.tensor([1433, 27],dtype = torch.int32), 1536)
            print("out:",out)``

在num_group=2，m,k,n=1536,2048,6144的情况下，m_indicies=torch.tensor([1433, 27],expected_m=1536的情况下会报错，这是我们实际业务下遇到的情况，求助如何解决？

Originally created by @risemeup1 on GitHub (May 13, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/97 `def test_m_grouped_gemm_masked() -> None: print('Testing grouped masked GEMM:') for num_groups, m in ((2, 1536),): for k, n in ((2048, 6144),): # Test correctness masked_m_candidates = list(filter(lambda candidate: candidate <= m, (64, 128, 192, 256, 320, 384))) for i in range(10): x_fp8, y_fp8, out, ref_out = construct_masked_grouped(num_groups, m, k, n) masked_m = torch.empty((num_groups, ), device='cuda', dtype=torch.int) for j in range(num_groups): masked_m[j] = random.choice(masked_m_candidates) expected_m = min(int(masked_m.float().mean()) + 1, m) deep_gemm.m_grouped_gemm_fp8_fp8_bf16_nt_masked(x_fp8, y_fp8, out, torch.tensor([1433, 27],dtype = torch.int32), 1536) print("out:",out)`` 在num_group=2，m,k,n=1536,2048,6144的情况下，m_indicies=torch.tensor([1433, 27],expected_m=1536的情况下会报错，这是我们实际业务下遇到的情况，求助如何解决？ ![Image](https://github.com/user-attachments/assets/4c342f8e-b451-4013-9313-06e6218a2445)

bsakaguchi closed this issue

2026-10-26 21:11:20 +00:00

bsakaguchi commented

2026-10-26 21:11:21 +00:00

Author

Owner

@zheanxu commented on GitHub (May 13, 2025):

By default, tensor.tensor() is on the CPU. Adding device='cuda' can resolve this issue. 😊

@zheanxu commented on GitHub (May 13, 2025): By default, `tensor.tensor()` is on the CPU. Adding `device='cuda'` can resolve this issue. 😊

bsakaguchi commented

2026-10-26 21:11:21 +00:00

Author

Owner

@risemeup1 commented on GitHub (May 13, 2025):

By default, tensor.tensor() is on the CPU. Adding device='cuda' can resolve this issue. 😊

感谢感谢

@risemeup1 commented on GitHub (May 13, 2025): > By default, `tensor.tensor()` is on the CPU. Adding `device='cuda'` can resolve this issue. 😊 感谢感谢

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepGEMM#52

No description provided.

Rows
Columns