deepseek-ai/DeepGEMM

Fork 0

mirror of https://github.com/deepseek-ai/DeepGEMM.git synced 2026-11-05 00:47:26 +00:00

[GH-ISSUE #461] Update CUTLASS to 4.8.0 for SM107 (Rubin) support #196

New issue

Open

opened 2026-10-26 21:14:36 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:14:36 +00:00

Owner

Originally created by @lisawym on GitHub (Sep 28, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/461

Summary

We encountered TMA/TMEM device assertions while running DeepSeek-V4.1-Flash through vLLM on Rubin (SM107). Could DeepGEMM update its vendored CUTLASS dependency from 4.2.1 to 4.8.0, which adds initial SM107 support?

The failure was observed with vllm-project/DeepGEMM at e1f418c2a4f20818221f6b0e578b4c2f634d4c3f. We also confirmed that upstream deepseek-ai/DeepGEMM at 78b6900 carries the same CUTLASS pin, f3fde58372d33e9a5650ba7b80fc48b3b49d40c8. The dependency limitation is therefore shared, although we have not reproduced the runtime failure on an unmodified upstream checkout.

The DSV4.1-Flash error output is included below. A colleague reported similar behavior with GLM5.3 on Rubin, which suggests the issue is not model-specific.

Observed failure

In our September 22, 2026 run, model weights loaded, but startup profiling failed before requests could be served. With CUDA_LAUNCH_BLOCKING=1, the stack pointed to tf32_hc_prenorm_gemm:

Trying to use TMA Descriptor Prefetch without CUTE_ARCH_TMA_SM90_ENABLED.
Attempting to use TMEM allocation PTX without CUTE_ARCH_TCGEN05_TMEM_ENABLED
CUDA error 710 (CUDA_ERROR_ASSERT): device-side assert triggered

DeepGEMM's pinned architecture selector selects sm_107f for capability 10.7. However, the pinned CUTLASS headers lack SM107 enablement; their SM100 feature guards require __CUDA_ARCH__ == 1000. The errors indicate that the affected CuTe TMA/TMEM helpers were compiled without their feature macros enabled, selecting assertion fallbacks. This is consistent with an architecture-guard mismatch.

Environment

Component	Recorded value
GPU	Rubin, SM107 / compute capability 10.7
Host architecture	Linux ARM64
Driver	620.43
CUDA runtime	13.4; exact NVCC patch version was not captured
Python	3.12.3
PyTorch	`2.15.0.dev20260908+cu134`
vLLM	`0.29.1rc1.dev513+g0961bbae2.d20260922.cu134`
vLLM source	`0961bbae2894d574be790d219651824eb199318e`
DeepGEMM	vLLM fork, `e1f418c2a4f20818221f6b0e578b4c2f634d4c3f`
Current CUTLASS	4.2.1, `f3fde58372d33e9a5650ba7b80fc48b3b49d40c8`
Model	`deepseek-ai/DeepSeek-V4.1-Flash`
Model revision	`dba1be0a40aa45a94ad051997016db3960a90277`

Base container:

vllm/vllm-openai:cu134-nightly-0961bbae2894d574be790d219651824eb199318e@sha256:551a8c08c5119b5dbc7e902aabcd6ce2d903f084f76d4fa13fd887008bfd83d3

Configuration: TP4, V2 model runner, eager execution, CPU-offloaded Engram, no speculative decoding, maximum model length 32768, maximum sequences 8, and maximum batched tokens 8192. The optional mooncake-transfer-engine-cuda13==0.3.13.post1 package was installed, but the Mooncake connector was disabled for this diagnostic run.

Proposed update and current workaround

Update the CUTLASS submodule pin from 4.2.1 to 4.8.0, released September 21, 2026 PDT / September 22 UTC. This release adds initial Rubin support, including enabling existing SM100-compatible kernels for sm_107a and sm_107f and updating TMEM handling.

The release notes specify the R615 driver associated with CUDA 13.4 GA for SM107 execution and state that R610 from the Developer Preview is insufficient. Our recorded run used driver 620.43.

Our temporary workaround sets DeepGEMM's JIT compiler target to sm_100f when the device reports capability 10.7. This allowed the exercised DSV4.1-Flash paths to run in a separate custom build; it was not a controlled A/B against the failing container above.

Validation and questions

We have not tested the 4.8.0 upgrade. We do have SM107 access and can run DSV4.1-Flash against a candidate branch to confirm whether the failure clears.

Is updating CUTLASS to 4.8.0 the recommended approach for SM107 support? Are there known compatibility or regression concerns, or required DeepGEMM changes beyond the submodule bump?

Originally created by @lisawym on GitHub (Sep 28, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepGEMM/issues/461 ## Summary We encountered TMA/TMEM device assertions while running DeepSeek-V4.1-Flash through vLLM on Rubin (SM107). Could DeepGEMM update its vendored CUTLASS dependency from **4.2.1 to 4.8.0**, which adds initial SM107 support? The failure was observed with `vllm-project/DeepGEMM` at `e1f418c2a4f20818221f6b0e578b4c2f634d4c3f`. We also confirmed that upstream [`deepseek-ai/DeepGEMM` at `78b6900`](https://github.com/deepseek-ai/DeepGEMM/tree/78b69000794d0937b47ae3387eff7663410264d1/third-party) carries the same CUTLASS pin, `f3fde58372d33e9a5650ba7b80fc48b3b49d40c8`. The dependency limitation is therefore shared, although we have not reproduced the runtime failure on an unmodified upstream checkout. The DSV4.1-Flash error output is included below. A colleague reported similar behavior with GLM5.3 on Rubin, which suggests the issue is not model-specific. ## Observed failure In our September 22, 2026 run, model weights loaded, but startup profiling failed before requests could be served. With `CUDA_LAUNCH_BLOCKING=1`, the stack pointed to `tf32_hc_prenorm_gemm`: ```text Trying to use TMA Descriptor Prefetch without CUTE_ARCH_TMA_SM90_ENABLED. Attempting to use TMEM allocation PTX without CUTE_ARCH_TCGEN05_TMEM_ENABLED CUDA error 710 (CUDA_ERROR_ASSERT): device-side assert triggered ``` DeepGEMM's [pinned architecture selector](https://github.com/deepseek-ai/DeepJIT/blob/e5bdee2bc4ca519eba00cfc5f0c6e950e6a96a16/include/deep_jit/backend/cuda/device.hpp#L68-L79) selects `sm_107f` for capability 10.7. However, the [pinned CUTLASS headers](https://github.com/NVIDIA/cutlass/blob/f3fde58372d33e9a5650ba7b80fc48b3b49d40c8/include/cutlass/arch/config.h) lack SM107 enablement; their SM100 feature guards require `__CUDA_ARCH__ == 1000`. The errors indicate that the affected CuTe TMA/TMEM helpers were compiled without their feature macros enabled, selecting assertion fallbacks. This is consistent with an architecture-guard mismatch. ## Environment | Component | Recorded value | |---|---| | GPU | Rubin, SM107 / compute capability 10.7 | | Host architecture | Linux ARM64 | | Driver | 620.43 | | CUDA runtime | 13.4; exact NVCC patch version was not captured | | Python | 3.12.3 | | PyTorch | `2.15.0.dev20260908+cu134` | | vLLM | `0.29.1rc1.dev513+g0961bbae2.d20260922.cu134` | | vLLM source | `0961bbae2894d574be790d219651824eb199318e` | | DeepGEMM | vLLM fork, `e1f418c2a4f20818221f6b0e578b4c2f634d4c3f` | | Current CUTLASS | **4.2.1**, `f3fde58372d33e9a5650ba7b80fc48b3b49d40c8` | | Model | `deepseek-ai/DeepSeek-V4.1-Flash` | | Model revision | `dba1be0a40aa45a94ad051997016db3960a90277` | Base container: ```text vllm/vllm-openai:cu134-nightly-0961bbae2894d574be790d219651824eb199318e@sha256:551a8c08c5119b5dbc7e902aabcd6ce2d903f084f76d4fa13fd887008bfd83d3 ``` Configuration: TP4, V2 model runner, eager execution, CPU-offloaded Engram, no speculative decoding, maximum model length 32768, maximum sequences 8, and maximum batched tokens 8192. The optional `mooncake-transfer-engine-cuda13==0.3.13.post1` package was installed, but the Mooncake connector was disabled for this diagnostic run. ## Proposed update and current workaround Update the CUTLASS submodule pin from **4.2.1 to [4.8.0](https://github.com/NVIDIA/cutlass/releases/tag/v4.8.0)**, released **September 21, 2026 PDT / September 22 UTC**. This release adds initial Rubin support, including enabling existing SM100-compatible kernels for `sm_107a` and `sm_107f` and updating TMEM handling. The release notes specify the R615 driver associated with CUDA 13.4 GA for SM107 execution and state that R610 from the Developer Preview is insufficient. Our recorded run used driver **620.43**. Our temporary workaround sets DeepGEMM's JIT compiler target to `sm_100f` when the device reports capability 10.7. This allowed the exercised DSV4.1-Flash paths to run in a separate custom build; it was not a controlled A/B against the failing container above. ## Validation and questions We have not tested the 4.8.0 upgrade. We do have SM107 access and can run DSV4.1-Flash against a candidate branch to confirm whether the failure clears. Is updating CUTLASS to 4.8.0 the recommended approach for SM107 support? Are there known compatibility or regression concerns, or required DeepGEMM changes beyond the submodule bump?

bsakaguchi commented

2026-10-26 21:14:37 +00:00

Author

Owner

@lucifer1004 commented on GitHub (Oct 17, 2026):

Another data point, from TensorRT-LLM: it pins DeepGEMM b64107f2b9, which also vendors CUTLASS 4.2.1. There, fp8_gemm_nt (the sm100_fp8_fp4_gemm_1d1d_impl kernel) hits the same assert on SM107, at cute/arch/tmem_allocator_sm100.hpp:143 with "Attempting to use TMEM allocation PTX without CUTE_ARCH_TCGEN05_TMEM_ENABLED". DG_JIT_DEBUG=1 shows --gpu-architecture=sm_107f.

Changing only the JIT target from sm_107f to sm_100f, with no other changes, makes the kernel run correctly on SM107 for (m, k, n) = (12, 512, 1024), (1, 7168, 2048) and (128, 7168, 2048). The error against an fp32 reference matches a non-DeepGEMM FP8 block-scale GEMM. So until the CUTLASS bump lands, mapping 10.7 to 100f in Device::get_arch() looks like a workable interim fix.

TensorRT-LLM tracking issue: NVIDIA/TensorRT-LLM#20508.

@lucifer1004 commented on GitHub (Oct 17, 2026): Another data point, from TensorRT-LLM: it pins DeepGEMM `b64107f2b9`, which also vendors CUTLASS 4.2.1. There, `fp8_gemm_nt` (the `sm100_fp8_fp4_gemm_1d1d_impl` kernel) hits the same assert on SM107, at `cute/arch/tmem_allocator_sm100.hpp:143` with "Attempting to use TMEM allocation PTX without CUTE_ARCH_TCGEN05_TMEM_ENABLED". `DG_JIT_DEBUG=1` shows `--gpu-architecture=sm_107f`. Changing only the JIT target from `sm_107f` to `sm_100f`, with no other changes, makes the kernel run correctly on SM107 for (m, k, n) = (12, 512, 1024), (1, 7168, 2048) and (128, 7168, 2048). The error against an fp32 reference matches a non-DeepGEMM FP8 block-scale GEMM. So until the CUTLASS bump lands, mapping 10.7 to `100f` in `Device::get_arch()` looks like a workable interim fix. TensorRT-LLM tracking issue: NVIDIA/TensorRT-LLM#20508.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepGEMM#196

No description provided.

Rows
Columns