[GH-ISSUE #178] int8_attention: OverflowError past 299,520 key rows (int32 strides), reachable on long-video workloads #57

New issue

Open

opened 2026-10-26 22:19:47 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 22:19:47 +00:00

Owner

Originally created by @shootthesound on GitHub (Sep 15, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/178

int8_attention raises OverflowError: sage_sdpa: tensor strides exceed int32 range; reduce batch/seq/head dimensions once the key sequence passes 299,520 rows at 56 heads × 128, with batch size 1.

This is reachable on ordinary long-video work rather than an exotic shape. It came up generating a 50-second MiniMax H3 video at 720p: the packed sequence is 314,437 rows, and the call that fails is the one where the text and reference rows attend to the whole timeline.

Where the boundary is

Bisected on a 5090:

largest working key rows: 299520   (first failing: 299521)
  k.stride(0) at 299520 = 2,146,959,360   int32 max = 2,147,483,647

The interesting part: 299,520 works and 299,521 fails, but neither stride exceeds int32. At 299,521 the batch stride is 2,147,425,792, still under the limit — so something internal (padding to cta_k, or a derived stride) crosses it first. 299,520 is exactly 64 × 4680, i.e. CTA_K-aligned, which fits that guess but I have not verified it.

Also worth noting: batch is 1 here, so stride(0) is never used to index anything. If the check is on the batch stride specifically, skipping it when batch == 1 might be a cheap fix for the common single-batch inference case.

Repro

Needs about 14 GB of VRAM, since the overflow is a function of element count.

import torch, comfy_kitchen

DEV, HEADS, HD = "cuda", 56, 128
for rows in (299520, 299521):
    q = torch.randn(1, HEADS, 512, HD, device=DEV, dtype=torch.bfloat16)
    k = torch.randn(1, HEADS, rows, HD, device=DEV, dtype=torch.bfloat16)
    v = torch.randn(1, HEADS, rows, HD, device=DEV, dtype=torch.bfloat16)
    print("%d rows, k.stride(0)=%d" % (rows, k.stride(0)), end=" -> ")
    try:
        comfy_kitchen.int8_attention(q, k, v, scale=HD ** -0.5)
        print("ok")
    except Exception as e:
        print("%s: %s" % (type(e).__name__, e))
    del q, k, v
    torch.cuda.empty_cache()

What it means in practice

The limit is a row count, so it arrives sooner the larger the frame — for MiniMax H3 at 24 fps, roughly 101 s at 832×480 but only 47 s at 720p. Video models are exactly the workload that reaches it, and a hard raise means a long render dies at the last step rather than degrading.

For what it is worth, the error is a good one: it refuses rather than computing something wrong, and OverflowError is specific enough to catch. We now do catch it, remember the width, and send those rows to PyTorch attention while everything narrower keeps using Kitchen — so this is not blocking us. But an attention backend quietly having a maximum sequence length is worth documenting even if it is not raised, and int64 offsets (or the batch == 1 shortcut) would remove it.

Related: #177, which is about the masked path's cost on the same workload.

Environment

comfy-kitchen 0.2.31 · RTX 5090, driver 616.86 · torch 2.9.1+cu130 · Python 3.10 · Windows 11

Mentioning it here since it is the same call path. _int8_attention_cuda allocates q_int8,
k_int8, v_int8 and their scales on top of the bf16 tensors it is handed. On one block of the
same workload — 4096 queries against a 58,000-key window, 56 heads of 128 — that is roughly:

k_int8   416 MB
v_int8   416 MB   (padded to cta_k)
q_int8    29 MB
output    59 MB
         ------
         ~920 MB   where PyTorch SDPA allocates the 59 MB output alone

Which is a fair trade for 3.4× and not a bug. But it does mean a workload can be fast enough and
still not fit, and the failure mode is a hard OutOfMemoryError mid-render rather than anything
recoverable from inside the kernel. A note in the docs that the int8 path trades memory for speed
would have saved us a render; we now catch it and fall back to PyTorch for the affected calls.

Originally created by @shootthesound on GitHub (Sep 15, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/178 `int8_attention` raises `OverflowError: sage_sdpa: tensor strides exceed int32 range; reduce batch/seq/head dimensions` once the key sequence passes **299,520 rows** at 56 heads × 128, with batch size 1. This is reachable on ordinary long-video work rather than an exotic shape. It came up generating a 50-second MiniMax H3 video at 720p: the packed sequence is 314,437 rows, and the call that fails is the one where the text and reference rows attend to the whole timeline. ## Where the boundary is Bisected on a 5090: ``` largest working key rows: 299520 (first failing: 299521) k.stride(0) at 299520 = 2,146,959,360 int32 max = 2,147,483,647 ``` The interesting part: **299,520 works and 299,521 fails, but neither stride exceeds int32.** At 299,521 the batch stride is 2,147,425,792, still under the limit — so something internal (padding to `cta_k`, or a derived stride) crosses it first. 299,520 is exactly `64 × 4680`, i.e. `CTA_K`-aligned, which fits that guess but I have not verified it. Also worth noting: **batch is 1 here**, so `stride(0)` is never used to index anything. If the check is on the batch stride specifically, skipping it when `batch == 1` might be a cheap fix for the common single-batch inference case. ## Repro Needs about 14 GB of VRAM, since the overflow is a function of element count. ```python import torch, comfy_kitchen DEV, HEADS, HD = "cuda", 56, 128 for rows in (299520, 299521): q = torch.randn(1, HEADS, 512, HD, device=DEV, dtype=torch.bfloat16) k = torch.randn(1, HEADS, rows, HD, device=DEV, dtype=torch.bfloat16) v = torch.randn(1, HEADS, rows, HD, device=DEV, dtype=torch.bfloat16) print("%d rows, k.stride(0)=%d" % (rows, k.stride(0)), end=" -> ") try: comfy_kitchen.int8_attention(q, k, v, scale=HD ** -0.5) print("ok") except Exception as e: print("%s: %s" % (type(e).__name__, e)) del q, k, v torch.cuda.empty_cache() ``` ## What it means in practice The limit is a **row count**, so it arrives sooner the larger the frame — for MiniMax H3 at 24 fps, roughly 101 s at 832×480 but only 47 s at 720p. Video models are exactly the workload that reaches it, and a hard raise means a long render dies at the last step rather than degrading. For what it is worth, the error is a good one: it refuses rather than computing something wrong, and `OverflowError` is specific enough to catch. We now do catch it, remember the width, and send those rows to PyTorch attention while everything narrower keeps using Kitchen — so this is not blocking us. But an attention backend quietly having a maximum sequence length is worth documenting even if it is not raised, and int64 offsets (or the `batch == 1` shortcut) would remove it. Related: #177, which is about the masked path's cost on the same workload. ## Environment `comfy-kitchen` 0.2.31 · RTX 5090, driver 616.86 · torch 2.9.1+cu130 · Python 3.10 · Windows 11 --- ## Related: the memory cost is not obvious either Mentioning it here since it is the same call path. `_int8_attention_cuda` allocates `q_int8`, `k_int8`, `v_int8` and their scales on top of the bf16 tensors it is handed. On one block of the same workload — 4096 queries against a 58,000-key window, 56 heads of 128 — that is roughly: ``` k_int8 416 MB v_int8 416 MB (padded to cta_k) q_int8 29 MB output 59 MB ------ ~920 MB where PyTorch SDPA allocates the 59 MB output alone ``` Which is a fair trade for 3.4× and not a bug. But it does mean a workload can be fast enough and still not fit, and the failure mode is a hard `OutOfMemoryError` mid-render rather than anything recoverable from inside the kernel. A note in the docs that the int8 path trades memory for speed would have saved us a render; we now catch it and fall back to PyTorch for the affected calls.

bsakaguchi commented

2026-10-26 22:19:48 +00:00

Author

Owner

@potoior commented on GitHub (Oct 22, 2026):

I'd like to take a look at this — planning to reproduce the boundary with synthetic tensors on a local GPU (batch 1, 56 heads, 128 dim), verify whether it's the unused batch stride or a CTA_K-padded derived stride that actually crosses int32 first, and fix accordingly. PR once I've confirmed.

@potoior commented on GitHub (Oct 22, 2026): I'd like to take a look at this — planning to reproduce the boundary with synthetic tensors on a local GPU (batch 1, 56 heads, 128 dim), verify whether it's the unused batch stride or a CTA_K-padded derived stride that actually crosses int32 first, and fix accordingly. PR once I've confirmed.

bsakaguchi commented

2026-10-26 22:19:48 +00:00

Author

Owner

@potoior commented on GitHub (Oct 22, 2026):

Verified on current main that this is fixed — and added regression coverage in #267.

The check the report bisected (0.2.31's tensor strides exceed int32 range) was replaced in b54f851 ("Support larger shapes in cuda int8 attention", Sep 28), which was never linked here. The launcher now bounds only the offsets the kernel actually computes in 32-bit: within-head tile offsets (kv_len * head_dim against int32) and the per-product batch/head base strides (each checked against uint32, with a 64-bit Offset kernel variant selected when a product would exceed it).

Reproduced the report's shapes on an NVIDIA Thor (sm_110, torch 2.11.0+cu130, comfy-kitchen main), compared against scaled_dot_product_attention (nrmse < 0.03):

shape (B : heads : head_dim : key rows)	result
1 : 56 : 128 : 299520 (old working boundary)	OK, nrmse 0.0175
1 : 56 : 128 : 299521 (old failing boundary, CTA_K-unaligned)	OK, nrmse 0.0175
1 : 56 : 128 : 314437 (the 50 s 720p MiniMax H3 case)	OK, nrmse 0.0175
2 : 64 : 128 : 320000 (multi-batch)	OK, nrmse 0.0175
1 : 64 : 128 : 540000 (head-base products > uint32, 64-bit `Offset` variant)	OK, nrmse 0.0177

The remaining hard limit is the within-head int32 offset check: kv_len * head_dim > 2^31, i.e. roughly 16.7 M key rows at head_dim 128, which raises a clean overflow_error instead of computing something wrong — beyond any current hardware. The HIP backend is not affected: it passes every stride as int64.

The 64-bit head-base path had no test coverage at all before; #267 locks all of the above in as regression tests (with VRAM guards).

@potoior commented on GitHub (Oct 22, 2026): Verified on current main that this is fixed — and added regression coverage in #267. The check the report bisected (0.2.31's `tensor strides exceed int32 range`) was replaced in b54f851 ("Support larger shapes in cuda int8 attention", Sep 28), which was never linked here. The launcher now bounds only the offsets the kernel actually computes in 32-bit: within-head tile offsets (`kv_len * head_dim` against int32) and the per-product batch/head base strides (each checked against uint32, with a 64-bit `Offset` kernel variant selected when a product would exceed it). Reproduced the report's shapes on an NVIDIA Thor (sm_110, torch 2.11.0+cu130, comfy-kitchen main), compared against `scaled_dot_product_attention` (nrmse < 0.03): | shape (B : heads : head_dim : key rows) | result | |---|---| | 1 : 56 : 128 : 299520 (old working boundary) | OK, nrmse 0.0175 | | 1 : 56 : 128 : 299521 (old failing boundary, CTA_K-unaligned) | OK, nrmse 0.0175 | | 1 : 56 : 128 : 314437 (the 50 s 720p MiniMax H3 case) | OK, nrmse 0.0175 | | 2 : 64 : 128 : 320000 (multi-batch) | OK, nrmse 0.0175 | | 1 : 64 : 128 : 540000 (head-base products > uint32, 64-bit `Offset` variant) | OK, nrmse 0.0177 | The remaining hard limit is the within-head int32 offset check: `kv_len * head_dim > 2^31`, i.e. roughly 16.7 M key rows at head_dim 128, which raises a clean `overflow_error` instead of computing something wrong — beyond any current hardware. The HIP backend is not affected: it passes every stride as int64. The 64-bit head-base path had no test coverage at all before; #267 locks all of the above in as regression tests (with VRAM guards).

bsakaguchi referenced this issue

2026-10-26 22:20:30 +00:00

[PR #57] [MERGED] Support int8 on turing (20 series) GPUs. #113