[PR #399] fix: SM90 paged mqa `prefix_sum` out of range (`CUDBG_EXCEPTION_WARP_OUT_OF_RANGE_ADDRESS`) #418

New issue

Open

opened 2026-10-26 21:17:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:17:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepGEMM/pull/399
Author: @cjackal
Created: 8/5/2026
Status: 🔄 Open

Base: main ← Head: sm90-paged-mqa-logits-warp-out-of-range

📝 Commits (1)

75d9d49 fix prefix_sum out of index error

📊 Changes

1 file changed (+1 additions, -1 deletions)

View changed files

📝 deep_gemm/include/deep_gemm/scheduler/sm90_paged_mqa_logits.cuh (+1 -1)

📄 Description

Description

In sm90_paged_mqa_logits.cuh, we compute num_segs using diff of prefix_sum, but current diff calculation logic is unsafe as q_idx can be batch_size after the binary search which is out of bound of prefix_sum if batch_size is aligned. In this case, we should instead refer to the already computed segment sum (the would-be prefix_sum[batch_size]).

The overall math is trivial, so no further comments added in this ternary.

We have hit this error in production, the following is the CUDA coredump - sm90_paged_mqa_logits_metadata<kAlignedBatchSize=96, SPLIT_KV=256, kNumSMs=132, kIsVarlen=false> hits the problematic line with 16 sequences with next_n=6:

...
[19:25:48.404432] coredump: Detected an exception of type CUDBG_EXCEPTION_WARP_OUT_OF_RANGE_ADDRESS (5)
[19:25:48.404435] coredump:   - Device: 4
[19:25:48.404437] coredump:   - SM: 124
[19:25:48.404439] coredump:   - Warp: 1
[19:25:48.404440] coredump:   - PC 0x7ef928bc46b0
[19:25:48.404688] coredump: Stack trace (lane masks: active 0xFFFFFFFF, valid 0xFFFFFFFF):
[19:25:48.404692] coredump:   # 0   0x7ef928bc46b0  _ZN9deep_gemm5sched30sm90_paged_mqa_logits_metadataILj96ELj256ELj132ELb0EEEvjjbPKjS3_Pj
...

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepGEMM/pull/399 **Author:** [@cjackal](https://github.com/cjackal) **Created:** 8/5/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `sm90-paged-mqa-logits-warp-out-of-range` --- ### 📝 Commits (1) - [`75d9d49`](https://github.com/deepseek-ai/DeepGEMM/commit/75d9d49aef9e6d1b1c1f579cae21998d54ac34f2) fix `prefix_sum` out of index error ### 📊 Changes **1 file changed** (+1 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `deep_gemm/include/deep_gemm/scheduler/sm90_paged_mqa_logits.cuh` (+1 -1) </details> ### 📄 Description # Description In `sm90_paged_mqa_logits.cuh`, [we compute `num_segs` using diff of `prefix_sum`](https://github.com/deepseek-ai/DeepGEMM/blob/559d79fb6994a58b8a15b4b93bf13ccc16edf247/deep_gemm/include/deep_gemm/scheduler/sm90_paged_mqa_logits.cuh#L113), but current diff calculation logic is unsafe as `q_idx` can be `batch_size` after the binary search which is out of bound of `prefix_sum` if `batch_size` is aligned. In this case, we should instead refer to the already computed segment sum (the would-be `prefix_sum[batch_size]`). The overall math is trivial, so no further comments added in this ternary. We have hit this error in production, the following is the CUDA coredump - `sm90_paged_mqa_logits_metadata<kAlignedBatchSize=96, SPLIT_KV=256, kNumSMs=132, kIsVarlen=false>` hits the problematic line with 16 sequences with next_n=6: ```log ... [19:25:48.404432] coredump: Detected an exception of type CUDBG_EXCEPTION_WARP_OUT_OF_RANGE_ADDRESS (5) [19:25:48.404435] coredump: - Device: 4 [19:25:48.404437] coredump: - SM: 124 [19:25:48.404439] coredump: - Warp: 1 [19:25:48.404440] coredump: - PC 0x7ef928bc46b0 [19:25:48.404688] coredump: Stack trace (lane masks: active 0xFFFFFFFF, valid 0xFFFFFFFF): [19:25:48.404692] coredump: # 0 0x7ef928bc46b0 _ZN9deep_gemm5sched30sm90_paged_mqa_logits_metadataILj96ELj256ELj132ELb0EEEvjjbPKjS3_Pj ... ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 21:17:34 +00:00

bsakaguchi referenced this issue

2026-10-26 21:17:41 +00:00

[PR #418] [CLOSED] feat: add compact M-grouped GEMM execution #429