deepseek-ai/DeepGEMM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00
Code Issues 162 Projects Packages Wiki Activity

[GH-ISSUE #405] CUDA Illegal Instruction in test_k_grouped_gemm_contiguous #185

New issue
Closed
opened 2026-10-26 21:14:24 +00:00 by bsakaguchi · 2 comments
bsakaguchi commented 2026-10-26 21:14:24 +00:00
Owner
Copy link

Originally created by @henny-bee on GitHub (Aug 12, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/405

I encountered a CUDA kernel error while running the test_k_grouped_gemm_contiguous test with the matrix dimensions explicitly set to:

  • M = 768
  • N = 1536

The test fails with the following error:

torch.AcceleratorError: CUDA error: an illegal instruction was encountered

The error appears to originate from a CUDA kernel execution. Since CUDA errors can be reported asynchronously, the reported stack trace may not point to the actual source of the failure.

For debugging, I also considered using:

CUDA_LAUNCH_BLOCKING=1

and enabling device-side assertions with TORCH_USE_CUDA_DSA.

Question

Is the M=768, N=1536 shape combination currently supported by the grouped GEMM implementation?

If this shape is expected to be supported, could there be a kernel configuration or shape-related issue causing the illegal instruction?

Environment

  • Test: test_k_grouped_gemm_contiguous
  • M: 768
  • N: 1536
  • Error: CUDA error: an illegal instruction was encountered
Originally created by @henny-bee on GitHub (Aug 12, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/405 I encountered a CUDA kernel error while running the `test_k_grouped_gemm_contiguous` test with the matrix dimensions explicitly set to: * `M = 768` * `N = 1536` The test fails with the following error: ```text torch.AcceleratorError: CUDA error: an illegal instruction was encountered ``` The error appears to originate from a CUDA kernel execution. Since CUDA errors can be reported asynchronously, the reported stack trace may not point to the actual source of the failure. For debugging, I also considered using: ```bash CUDA_LAUNCH_BLOCKING=1 ``` and enabling device-side assertions with `TORCH_USE_CUDA_DSA`. ### Question Is the `M=768, N=1536` shape combination currently supported by the grouped GEMM implementation? If this shape is expected to be supported, could there be a kernel configuration or shape-related issue causing the illegal instruction? ### Environment * Test: `test_k_grouped_gemm_contiguous` * M: `768` * N: `1536` * Error: `CUDA error: an illegal instruction was encountered`
bsakaguchi commented 2026-10-26 21:14:26 +00:00
Author
Owner
Copy link

@PavelPaha commented on GitHub (Aug 25, 2026):

Hi @henny-bee, this has already been fixed on the current main branch by:
github.com/deepseek-ai/DeepGEMM@1f6f3f3789

<!-- gh-comment-id:5416965555 --> @PavelPaha commented on GitHub (Aug 25, 2026): Hi @henny-bee, this has already been fixed on the current main branch by: https://github.com/deepseek-ai/DeepGEMM/commit/1f6f3f378920ccb5cc036ef43eb3f5972e921713
bsakaguchi commented 2026-10-26 21:14:26 +00:00
Author
Owner
Copy link

@XFDG commented on GitHub (Sep 13, 2026):

Hi, I verified this on the current main branch (66081d4c) on an NVIDIA B200 (SM100) with CUDA 13.1.

Using 8 small K groups with the reported M=768, N=1536 shape, 50/50 synchronized k_grouped_fp8_gemm_tn_contiguous launches completed without an illegal instruction. The output stayed finite and bitwise deterministic across all launches. This confirms that the tensor-map race fix in 1f6f3f378920ccb5cc036ef43eb3f5972e921713 covers this failure on current main.

<!-- gh-comment-id:5650735651 --> @XFDG commented on GitHub (Sep 13, 2026): Hi, I verified this on the current main branch (`66081d4c`) on an NVIDIA B200 (SM100) with CUDA 13.1. Using 8 small K groups with the reported `M=768, N=1536` shape, 50/50 synchronized `k_grouped_fp8_gemm_tn_contiguous` launches completed without an illegal instruction. The output stayed finite and bitwise deterministic across all launches. This confirms that the tensor-map race fix in `1f6f3f378920ccb5cc036ef43eb3f5972e921713` covers this failure on current main.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
public-sync-20261014
0930-release
nv_dev
fix-mega-moe-fence
public-release-2609
public-release-2607
public-release-2606
mega-update
public-release-260416
ko3n1g/ci/build-wheels
nv_dev_f8e8fb5
nv_dev_950d31a
nv_dev_67fc648
nv_dev_c491439
nv_dev_4ff3f54
v2.1.1.post3
v2.1.1.post2
v2.1.1.post1
v2.1.1
v2.1.0
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepGEMM#185
No description provided.