[GH-ISSUE #210] gated_delta.is_available() returns True on sm_70 (V100); deltanet_conv_step then raises launch rejected #63

New issue

Closed

opened 2026-10-26 22:19:51 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:19:51 +00:00

Owner

Originally created by @itssutch on GitHub (Sep 28, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/210

Summary

On Tesla V100 (sm_70), comfy_kitchen.gated_delta.is_available() returns True, so ComfyUI 0.37's Qwen 3.5 TextGenerate path calls deltanet_conv_step / gated_delta_decode_fused. The CUDA launcher then returns false and Python raises:

RuntimeError: deltanet_conv_step launch rejected

ComfyUI already has a working eager/PyTorch fallback in comfy/text_encoders/qwen35.py. It never reaches that path because it trusts gated_delta_decode_is_available.

Environment

GPU: Tesla V100-SXM2-16GB (sm_70)
PyTorch: 2.14.0+cu126 (correct for Volta; cu130 dropped < sm_75)
ComfyUI: 0.37.0
comfy-kitchen: 0.2.35
Startup: kitchen CUDA backend available=True, disabled=True (ComfyUI disables it below cu130); eager backend enabled
Kitchen CUDA capabilities list does not include deltanet_conv_step

Why the gate is wrong

is_available() in comfy_kitchen/gated_delta.py only checks:

_EXT_AVAILABLE and that _C has gated_delta_decode_fused / deltanet_conv_step
head-dim shape constraints
shared_memory_per_block_optin vs _fused_shmem_bytes()

It does not check device compute capability against the archs actually compiled into the wheel (Linux default is 75-virtual;80;89;90a;100f;120f). V100 reports ~96 KB opt-in shared memory, the fused kernel asks for ~74 KB, so the shmem test passes even though there is no sm_70 cubin.

This path also ignores ComfyUI having disabled the kitchen CUDA backend. gated_delta talks to _C directly and does not go through the registry constraint system described in the README.

Expected

is_available(device=V100) returns False so callers use eager/PyTorch. A rejected launch should not be the first signal that the device is unsupported.

Suggested fix

In is_available() (CUDA branch), also require:

torch.cuda.get_device_capability(device) >= (7, 5), matching sage_attention._NATIVE_MINIMUM_CAPABILITY and the wheel floor
optionally: do not claim availability when the CUDA backend is disabled

deltanet_conv_step currently calls is_available(proj.device) with default head dims, so it inherits the fused-decode shmem gate even for the conv-only kernel. A dedicated conv-step capability check would be more accurate, but the CC / compiled-arch check is the one that stops this crash.

ComfyUI call site: comfy/text_encoders/qwen35.py (use_fused + comfy_kitchen.deltanet_conv_step). If kitchen cannot fix the gate quickly, ComfyUI should treat launch rejected as “fused unavailable” and fall through to the existing F.conv1d path instead of failing the prompt.

Originally created by @itssutch on GitHub (Sep 28, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/210 ## Summary On Tesla V100 (sm_70), `comfy_kitchen.gated_delta.is_available()` returns `True`, so ComfyUI 0.37's Qwen 3.5 `TextGenerate` path calls `deltanet_conv_step` / `gated_delta_decode_fused`. The CUDA launcher then returns false and Python raises: ``` RuntimeError: deltanet_conv_step launch rejected ``` ComfyUI already has a working eager/PyTorch fallback in `comfy/text_encoders/qwen35.py`. It never reaches that path because it trusts `gated_delta_decode_is_available`. ## Environment - GPU: Tesla V100-SXM2-16GB (sm_70) - PyTorch: 2.14.0+cu126 (correct for Volta; cu130 dropped < sm_75) - ComfyUI: 0.37.0 - comfy-kitchen: 0.2.35 - Startup: kitchen CUDA backend `available=True, disabled=True` (ComfyUI disables it below cu130); eager backend enabled - Kitchen CUDA capabilities list does **not** include `deltanet_conv_step` ## Why the gate is wrong `is_available()` in `comfy_kitchen/gated_delta.py` only checks: 1. `_EXT_AVAILABLE` and that `_C` has `gated_delta_decode_fused` / `deltanet_conv_step` 2. head-dim shape constraints 3. `shared_memory_per_block_optin` vs `_fused_shmem_bytes()` It does **not** check device compute capability against the archs actually compiled into the wheel (Linux default is `75-virtual;80;89;90a;100f;120f`). V100 reports ~96 KB opt-in shared memory, the fused kernel asks for ~74 KB, so the shmem test passes even though there is no sm_70 cubin. This path also ignores ComfyUI having disabled the kitchen CUDA backend. `gated_delta` talks to `_C` directly and does not go through the registry constraint system described in the README. ## Expected `is_available(device=V100)` returns `False` so callers use eager/PyTorch. A rejected launch should not be the first signal that the device is unsupported. ## Suggested fix In `is_available()` (CUDA branch), also require: - `torch.cuda.get_device_capability(device) >= (7, 5)`, matching `sage_attention._NATIVE_MINIMUM_CAPABILITY` and the wheel floor - optionally: do not claim availability when the CUDA backend is disabled `deltanet_conv_step` currently calls `is_available(proj.device)` with default head dims, so it inherits the fused-decode shmem gate even for the conv-only kernel. A dedicated conv-step capability check would be more accurate, but the CC / compiled-arch check is the one that stops this crash. ## Related ComfyUI call site: `comfy/text_encoders/qwen35.py` (`use_fused` + `comfy_kitchen.deltanet_conv_step`). If kitchen cannot fix the gate quickly, ComfyUI should treat `launch rejected` as “fused unavailable” and fall through to the existing `F.conv1d` path instead of failing the prompt.

bsakaguchi closed this issue

2026-10-26 22:19:51 +00:00

bsakaguchi referenced this issue

2026-10-26 22:20:33 +00:00

[PR #63] int4_tensorwise: W4A4 layout (+ConvRot) with fused CUDA kernels #118