[PR #186] [MERGED] Support 256 head dim in Flash attention kernel. #204

New issue

Closed

opened 2026-10-26 22:21:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:33 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/186
Author: @comfyanonymous
Created: 9/18/2026
Status: ✅ Merged
Merged: 9/24/2026
Merged by: @comfyanonymous

Base: main ← Head: dev/fa256

📝 Commits (2)

9d20fd2 Flash attention 256 head dim.
a098d2b Fix flash attention is_available returning True on CPU device.

📊 Changes

7 files changed (+195 additions, -124 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+7 -6)
📝 comfy_kitchen/backends/cuda/ops/flash_decode.cu (+21 -15)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+7 -7)
📝 comfy_kitchen/backends/hip/launchers.h (+2 -2)
📝 comfy_kitchen/backends/hip/ops/flash_decode.hip (+78 -60)
📝 comfy_kitchen/flash_attention.py (+11 -5)
📝 tests/test_flash_attention.py (+69 -29)

📄 Description

No description provided

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/186 **Author:** [@comfyanonymous](https://github.com/comfyanonymous) **Created:** 9/18/2026 **Status:** ✅ Merged **Merged:** 9/24/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `dev/fa256` --- ### 📝 Commits (2) - [`9d20fd2`](https://github.com/Comfy-Org/comfy-kitchen/commit/9d20fd2e131fc64e5328912eb78c1a621bbf495a) Flash attention 256 head dim. - [`a098d2b`](https://github.com/Comfy-Org/comfy-kitchen/commit/a098d2b84b1758059dd6e5ac37e3c2f4b670ef36) Fix flash attention is_available returning True on CPU device. ### 📊 Changes **7 files changed** (+195 additions, -124 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+7 -6) 📝 `comfy_kitchen/backends/cuda/ops/flash_decode.cu` (+21 -15) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+7 -7) 📝 `comfy_kitchen/backends/hip/launchers.h` (+2 -2) 📝 `comfy_kitchen/backends/hip/ops/flash_decode.hip` (+78 -60) 📝 `comfy_kitchen/flash_attention.py` (+11 -5) 📝 `tests/test_flash_attention.py` (+69 -29) </details> ### 📄 Description _No description provided_ --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:33 +00:00