Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

[GH-ISSUE #87] CUDA apply_rope kernel errors on sequence-length broadcast that eager backend handles #28

New issue

Closed

opened 2026-10-26 22:19:22 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-26 22:19:22 +00:00

Owner

Originally created by @LacklusterOpsec on GitHub (Jul 29, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/87

Issue:
The CUDA backend's apply_rope kernel raises freqs shape is not broadcastable to input when freqs_cis.shape[2] (sequence length) differs from the input tensor's sequence length, even when the shapes are valid for broadcast (e.g., one is 1 or the mismatch is otherwise legal).

The eager backend handles this correctly in comfy_kitchen/backends/eager/rope.py (lines 12–13) by trimming freqs to match the input:

if x_.shape[2] != 1 and freqs_cis.shape[2] != 1 and x_.shape[2] != freqs_cis.shape[2]:
freqs_cis = freqs_cis[:, :, :x_.shape[2]]

Repro: This occurs when using the Flux KV cache (FluxKVCache node), which concatenates cached key tokens, making k longer than the original
freqs_cis. The CUDA kernel rejects the shape outright instead of trimming to the available freqs range.

Workaround: Disabling the FluxKVCache node avoids the error, since the model runs without cached key concatenation and sequence lengths match.

Expected: The CUDA kernel should either broadcast or handle the mismatch similarly to the eager fallback.

Originally created by @LacklusterOpsec on GitHub (Jul 29, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/87 Issue: The CUDA backend's apply_rope kernel raises freqs shape is not broadcastable to input when freqs_cis.shape[2] (sequence length) differs from the input tensor's sequence length, even when the shapes are valid for broadcast (e.g., one is 1 or the mismatch is otherwise legal). The eager backend handles this correctly in comfy_kitchen/backends/eager/rope.py (lines 12–13) by trimming freqs to match the input: if x_.shape[2] != 1 and freqs_cis.shape[2] != 1 and x_.shape[2] != freqs_cis.shape[2]: freqs_cis = freqs_cis[:, :, :x_.shape[2]] Repro: This occurs when using the Flux KV cache (FluxKVCache node), which concatenates cached key tokens, making k longer than the original freqs_cis. The CUDA kernel rejects the shape outright instead of trimming to the available freqs range. Workaround: Disabling the FluxKVCache node avoids the error, since the model runs without cached key concatenation and sequence lengths match. Expected: The CUDA kernel should either broadcast or handle the mismatch similarly to the eager fallback.

bsakaguchi closed this issue

2026-10-26 22:19:22 +00:00

bsakaguchi commented

2026-10-26 22:19:22 +00:00

Author

Owner

@coderabbitai[bot] commented on GitHub (Jul 29, 2026):

🔗 Related PRs

Comfy-Org/comfy-kitchen#85 - Refactor RoPE kernels and add in-place variants [merged]

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

@coderabbitai[bot] commented on GitHub (Jul 29, 2026):  <details> <summary>🔗 Related PRs</summary> Comfy-Org/comfy-kitchen#85 - Refactor RoPE kernels and add in-place variants [merged] </details> --- <details> <summary> 🧪 Issue enrichment is currently in open beta.</summary> You can configure auto-planning by selecting labels in the issue_enrichment configuration. To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`: ```yaml issue_enrichment: auto_enrich: enabled: false ``` </details> 💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit)!

bsakaguchi commented

2026-10-26 22:19:22 +00:00

Author

Owner

@LacklusterOpsec commented on GitHub (Jul 29, 2026):

Update: Further testing shows this also fails when all sequence lengths match — the CUDA kernel's internal broadcast check on the rotation matrix dimensions (2, 2) vs the reshaped input's (1, 2) is stricter than the eager backend, which handles this via standard PyTorch broadcasting. The try/except RuntimeError fallback to the eager path is the safest workaround on the ComfyUI side.

@LacklusterOpsec commented on GitHub (Jul 29, 2026): Update: Further testing shows this also fails when all sequence lengths match — the CUDA kernel's internal broadcast check on the rotation matrix dimensions (2, 2) vs the reshaped input's (1, 2) is stricter than the eager backend, which handles this via standard PyTorch broadcasting. The try/except RuntimeError fallback to the eager path is the safest workaround on the ComfyUI side.

bsakaguchi commented

2026-10-26 22:19:23 +00:00

Author

Owner

@comfyanonymous commented on GitHub (Jul 30, 2026):

is this fixed now?

@comfyanonymous commented on GitHub (Jul 30, 2026): is this fixed now?

bsakaguchi commented

2026-10-26 22:19:23 +00:00

Author

Owner

@LacklusterOpsec commented on GitHub (Aug 1, 2026):

Yes, issue is now resolved.

@LacklusterOpsec commented on GitHub (Aug 1, 2026): Yes, issue is now resolved.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#28

No description provided.

Rows
Columns