[PR #256] cuda: mark attn_mask .none() on sage_sdpa and sage_sdpa_prequantized for nanobind < 2.11 #263

New issue

Open

opened 2026-10-26 22:22:14 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:22:14 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/256
Author: @fblissjr
Created: 10/18/2026
Status: 🔄 Open

Base: main ← Head: fix-sage-sdpa-attn-mask-none

📝 Commits (1)

c2ea126 cuda bindings: mark attn_mask .none() on sage_sdpa and sage_sdpa_prequantized for nanobind < 2.11

📊 Changes

1 file changed (+2 additions, -2 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+2 -2)

📄 Description

When comfy-kitchen is built from source with nanobind < 2.11.0 (pyproject.toml allows nanobind>=2.0.0), calling int8_attention or int8_attention_from_prequantized with attn_mask=None (the default) raises a TypeError:

TypeError: sage_sdpa_prequantized(): incompatible function arguments. The following argument types are supported:
    1. sage_sdpa_prequantized(q_int8: ndarray[device='cuda'], k_int8: ndarray[device='cuda'], v_int8: ndarray[device='cuda'], o: ndarray[device='cuda'], q_scale: ndarray[device='cuda'], k_scale: ndarray[device='cuda'], v_scale: ndarray[device='cuda'], cta_k: int, sm_scale: float, output_dtype_code: int, stream_ptr: int, attn_mask: ndarray[device='cuda']) -> None

Invoked with types: PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, int, float, int, int, NoneType

Why this happens

In #207 (690f765), the CUDA bindings for sage_sdpa and sage_sdpa_prequantized changed nb::arg("attn_mask") = nb::none() to a bare nb::arg("attn_mask") (while the HIP backend kept = nb::none()).
nanobind >= 2.11.0 (used to build the PyPI 0.2.38 wheel) automatically treats std::optional<T> parameters as .none() (wjakob/nanobind#1262), so None is accepted even with a bare nb::arg("attn_mask").
nanobind 2.0.0–2.10.2 requires an explicit .none() (or = nb::none()) on nb::arg(...), otherwise None is rejected at runtime.

Fix

Mark nb::arg("attn_mask").none() on both sage_sdpa_prequantized and sage_sdpa in comfy_kitchen/backends/cuda/dlpack_bindings.cpp (matching the other std::optional bindings in the file).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/256 **Author:** [@fblissjr](https://github.com/fblissjr) **Created:** 10/18/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix-sage-sdpa-attn-mask-none` --- ### 📝 Commits (1) - [`c2ea126`](https://github.com/Comfy-Org/comfy-kitchen/commit/c2ea1263e86c6f05c267583366a89f9a6862744f) cuda bindings: mark attn_mask .none() on sage_sdpa and sage_sdpa_prequantized for nanobind < 2.11 ### 📊 Changes **1 file changed** (+2 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+2 -2) </details> ### 📄 Description When `comfy-kitchen` is built from source with `nanobind < 2.11.0` (`pyproject.toml` allows `nanobind>=2.0.0`), calling `int8_attention` or `int8_attention_from_prequantized` with `attn_mask=None` (the default) raises a `TypeError`: ``` TypeError: sage_sdpa_prequantized(): incompatible function arguments. The following argument types are supported: 1. sage_sdpa_prequantized(q_int8: ndarray[device='cuda'], k_int8: ndarray[device='cuda'], v_int8: ndarray[device='cuda'], o: ndarray[device='cuda'], q_scale: ndarray[device='cuda'], k_scale: ndarray[device='cuda'], v_scale: ndarray[device='cuda'], cta_k: int, sm_scale: float, output_dtype_code: int, stream_ptr: int, attn_mask: ndarray[device='cuda']) -> None Invoked with types: PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, int, float, int, int, NoneType ``` ### Why this happens - In #207 (`690f765`), the CUDA bindings for `sage_sdpa` and `sage_sdpa_prequantized` changed `nb::arg("attn_mask") = nb::none()` to a bare `nb::arg("attn_mask")` (while the HIP backend kept `= nb::none()`). - **`nanobind >= 2.11.0`** (used to build the PyPI `0.2.38` wheel) automatically treats `std::optional<T>` parameters as `.none()` ([wjakob/nanobind#1262](https://github.com/wjakob/nanobind/pull/1262)), so `None` is accepted even with a bare `nb::arg("attn_mask")`. - **`nanobind 2.0.0`–`2.10.2`** requires an explicit `.none()` (or `= nb::none()`) on `nb::arg(...)`, otherwise `None` is rejected at runtime. ### Fix Mark `nb::arg("attn_mask").none()` on both `sage_sdpa_prequantized` and `sage_sdpa` in `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (matching the other `std::optional` bindings in the file). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 22:22:14 +00:00

bsakaguchi referenced this issue

2026-10-26 22:22:16 +00:00

[PR #263] [MERGED] Head dim 64 support in flash attention. #267