Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

[GH-ISSUE #106] [Bug] comfy-kitchen routes to _cuda_backend on AMD HIP (RDNA 4), causing PyCapsule TypeError during INT8 attention #33

New issue

Closed

opened 2026-10-26 22:19:25 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-26 22:19:25 +00:00

Owner

Originally created by @zychuk on GitHub (Aug 11, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/106

Environment:

OS: Linux
GPU: AMD Radeon RX 9070 (gfx1201)
ROCm Version: 7.13
PyTorch Version: 2.11.0+rocm7.13.0
Python Version: 3.14.7
ComfyUI Version: 0.31.0 (Latest master, testing PR #15479)
comfy-kitchen Version: 0.2.30

Description:
When attempting to use the newly implemented Comfy Kitchen INT8 attention on an AMD RDNA 4 GPU, the generation crashes.

This happens using both implementation methods:
-Using the ModelAttentionBackend node (crashes during the diffusion model sampling).
-Using the --use-ck-attention CLI argument (crashes earlier during the text encoder's preprocess_text_embeds).

Steps to Reproduce:

-Run ComfyUI on an AMD Linux ROCm environment.
-Load a standard workflow (e.g., MiniMaxH3).
-Apply comfy kitchen attention via the ModelAttentionBackend node OR launch ComfyUI with --use-ck-attention.
-Queue a prompt.

Relevant Logs & Traceback:

Startup confirmation showing HIP is available:
[INFO] Found comfy_kitchen backend hip: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': [...]
[INFO] Device: cuda:0 AMD Radeon RX 9070 : native
[INFO] Using Comfy Kitchen attention

Crash Traceback:
[ERROR] !!! Exception during processing !!! sage_sdpa_quantize(): incompatible function arguments. The following argument types are supported:

sage_sdpa_quantize(q: ndarray[device='cuda'], k: ndarray[device='cuda'], v: ndarray[device='cuda'], q_int8: ndarray[device='cuda'], q_scale: ndarray[device='cuda'], k_int8: ndarray[device='cuda'], k_scale: ndarray[device='cuda'], v_int8: ndarray[device='cuda'], v_scale: ndarray[device='cuda'], cta_k: int, input_dtype_code: int, stream_ptr: int, anchor_indices_ptr: int) -> None

Invoked with types: PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, int, int, int, int

[ERROR] Traceback (most recent call last):
...
File "/workspace/ComfyUI/comfy/ldm/modules/attention.py", line 628, in _attention_comfy_kitchen_int8_containers
quantized = comfy_kitchen.prequantize_int8_attention(
q,
k,
v,
scale=kwargs.get("scale", None),
attn_mask=mask,
)
File "/usr/local/lib/python3.14/site-packages/comfy_kitchen/sage_attention.py", line 322, in prequantize_int8_attention
_cuda_backend._C.sage_sdpa_quantize(
_cuda_backend._wrap_for_dlpack(q),
...
anchor_indices_ptr,
)
TypeError: sage_sdpa_quantize(): incompatible function arguments.

Originally created by @zychuk on GitHub (Aug 11, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/106 Environment: OS: Linux GPU: AMD Radeon RX 9070 (gfx1201) ROCm Version: 7.13 PyTorch Version: 2.11.0+rocm7.13.0 Python Version: 3.14.7 ComfyUI Version: 0.31.0 (Latest master, testing PR #15479) comfy-kitchen Version: 0.2.30 Description: When attempting to use the newly implemented Comfy Kitchen INT8 attention on an AMD RDNA 4 GPU, the generation crashes. This happens using both implementation methods: -Using the ModelAttentionBackend node (crashes during the diffusion model sampling). -Using the --use-ck-attention CLI argument (crashes earlier during the text encoder's preprocess_text_embeds). Steps to Reproduce: -Run ComfyUI on an AMD Linux ROCm environment. -Load a standard workflow (e.g., MiniMaxH3). -Apply comfy kitchen attention via the ModelAttentionBackend node OR launch ComfyUI with --use-ck-attention. -Queue a prompt. Relevant Logs & Traceback: Startup confirmation showing HIP is available: [INFO] Found comfy_kitchen backend hip: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': [...] [INFO] Device: cuda:0 AMD Radeon RX 9070 : native [INFO] Using Comfy Kitchen attention Crash Traceback: [ERROR] !!! Exception during processing !!! sage_sdpa_quantize(): incompatible function arguments. The following argument types are supported: 1. sage_sdpa_quantize(q: ndarray[device='cuda'], k: ndarray[device='cuda'], v: ndarray[device='cuda'], q_int8: ndarray[device='cuda'], q_scale: ndarray[device='cuda'], k_int8: ndarray[device='cuda'], k_scale: ndarray[device='cuda'], v_int8: ndarray[device='cuda'], v_scale: ndarray[device='cuda'], cta_k: int, input_dtype_code: int, stream_ptr: int, anchor_indices_ptr: int) -> None Invoked with types: PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, int, int, int, int [ERROR] Traceback (most recent call last): ... File "/workspace/ComfyUI/comfy/ldm/modules/attention.py", line 628, in _attention_comfy_kitchen_int8_containers quantized = comfy_kitchen.prequantize_int8_attention( q, k, v, scale=kwargs.get("scale", None), attn_mask=mask, ) File "/usr/local/lib/python3.14/site-packages/comfy_kitchen/sage_attention.py", line 322, in prequantize_int8_attention _cuda_backend._C.sage_sdpa_quantize( _cuda_backend._wrap_for_dlpack(q), ... anchor_indices_ptr, ) TypeError: sage_sdpa_quantize(): incompatible function arguments.

bsakaguchi closed this issue

2026-10-26 22:19:25 +00:00

bsakaguchi commented

2026-10-26 22:19:26 +00:00

Author

Owner

@coderabbitai[bot] commented on GitHub (Aug 11, 2026):

⚠️ Possible Duplicate Issue(s)

https://github.com/Comfy-Org/comfy-kitchen/issues/78

🔗 Related PRs

Comfy-Org/comfy-kitchen#103 - Implement optimized int8 attention. [merged]

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

@coderabbitai[bot] commented on GitHub (Aug 11, 2026):  <details> <summary>⚠️ Possible Duplicate Issue(s)</summary> - https://github.com/Comfy-Org/comfy-kitchen/issues/78 </details> <details> <summary>🔗 Related PRs</summary> Comfy-Org/comfy-kitchen#103 - Implement optimized int8 attention. [merged] </details> --- <details> <summary> 🧪 Issue enrichment is currently in open beta.</summary> You can configure auto-planning by selecting labels in the issue_enrichment configuration. To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`: ```yaml issue_enrichment: auto_enrich: enabled: false ``` </details> 💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit)!

bsakaguchi commented

2026-10-26 22:19:27 +00:00

Author

Owner

@0xDELUXA commented on GitHub (Aug 11, 2026):

What do you mean by "newly implemented Comfy Kitchen INT8 attention on an AMD"? https://github.com/Comfy-Org/comfy-kitchen/pull/104 is still a draft and hasn’t been merged yet. https://github.com/Comfy-Org/comfy-kitchen/pull/103, which has been merged, is NVIDIA-only. So the behavior described in this issue is perfectly fine for now.

@0xDELUXA commented on GitHub (Aug 11, 2026): What do you mean by "newly implemented Comfy Kitchen INT8 attention on an AMD"? https://github.com/Comfy-Org/comfy-kitchen/pull/104 is still a draft and hasn’t been merged yet. https://github.com/Comfy-Org/comfy-kitchen/pull/103, which has been merged, is NVIDIA-only. So the behavior described in this issue is perfectly fine for now.

bsakaguchi commented

2026-10-26 22:19:27 +00:00

Author

Owner

@zychuk commented on GitHub (Aug 11, 2026):

What do you mean by "newly implemented Comfy Kitchen INT8 attention on an AMD"? #104 is still a draft and hasn’t been merged yet. #103, which has been merged, is NVIDIA-only. So the behavior described in this issue is perfectly fine for now.

My bad, saw 103 merged and didn't realize they were separate

@zychuk commented on GitHub (Aug 11, 2026): > What do you mean by "newly implemented Comfy Kitchen INT8 attention on an AMD"? [#104](https://github.com/Comfy-Org/comfy-kitchen/pull/104) is still a draft and hasn’t been merged yet. [#103](https://github.com/Comfy-Org/comfy-kitchen/pull/103), which has been merged, is NVIDIA-only. So the behavior described in this issue is perfectly fine for now. My bad, saw 103 merged and didn't realize they were separate

bsakaguchi referenced this issue

2026-10-26 22:20:18 +00:00

[PR #33] [CLOSED] Add ROCm backend #93