[PR #113] [MERGED] Fix illegal memory address when skipping -1 m indices #248

New issue

Closed

opened 2026-10-26 21:15:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:15:19 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/113
Author: @shixianc
Created: 6/15/2025
Status: ✅ Merged
Merged: 6/16/2025
Merged by: @LyricZhao

Base: main ← Head: main

📝 Commits (1)

b7ccb13 Fix illegal memory address when skipping -1 m indices

📊 Changes

1 file changed (+1 additions, -1 deletions)

View changed files

📝 deep_gemm/include/deep_gemm/scheduler.cuh (+1 -1)

📄 Description

I'm testing vllm integration with latest m_grouped_gemm_fp8_fp8_bf16_nt_contiguous but I faced an issue when I pad m_indices with -1 for skipping useless compute.
When VLLM tries to capture the whole CUDA Graph for the model, it fails with error

RuntimeError: CUDA error: an illegal memory access was encountered

However if I benchmark this kernel separately it passes with -1. After debugging this is related to tma_copy operation at this line. scheduler.get_global_idx will return an invalid index for -1 m block.

With the changes in the PR it fixes the issue. However after some research, it seems invalid address is an no-op for tma_copy, and maybe by design to skip useless loading? Because if I benchmark this kernel separately it passes with -1.

Can anyone help clarifying this and suggest on the different behavior by CUDA Graph capturing? Thanks!!

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/113 **Author:** [@shixianc](https://github.com/shixianc) **Created:** 6/15/2025 **Status:** ✅ Merged **Merged:** 6/16/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`b7ccb13`](https://github.com/deepseek-ai/DeepGEMM/commit/b7ccb135bc493dd378b3b252f21efdc39b582993) Fix illegal memory address when skipping -1 m indices ### 📊 Changes **1 file changed** (+1 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `deep_gemm/include/deep_gemm/scheduler.cuh` (+1 -1) </details> ### 📄 Description I'm testing vllm integration with latest `m_grouped_gemm_fp8_fp8_bf16_nt_contiguous` but I faced an issue when I pad m_indices with `-1` for skipping useless compute. When VLLM tries to capture the whole CUDA Graph for the model, it fails with error ``` RuntimeError: CUDA error: an illegal memory access was encountered ``` However if I benchmark this kernel separately it passes with -1. After debugging this is related to tma_copy operation at [this line](https://github.com/deepseek-ai/DeepGEMM/blob/8dfa3298274bfe6b242f6f8a3e6f3eff2707dd9f/deep_gemm/include/deep_gemm/fp8_gemm.cuh#L211). `scheduler.get_global_idx` will return an invalid index for -1 m block. With the changes in the PR it fixes the issue. However after some research, it seems invalid address is an no-op for tma_copy, and maybe by design to skip useless loading? Because if I benchmark this kernel separately it passes with -1. Can anyone help clarifying this and suggest on the different behavior by CUDA Graph capturing? Thanks!! --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 21:15:19 +00:00