[PR #259] [MERGED] AMD int8 performance #264

New issue

Closed

opened 2026-10-26 22:22:14 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:22:14 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/259
Author: @rattus128
Created: 10/19/2026
Status: ✅ Merged
Merged: 10/21/2026
Merged by: @comfyanonymous

Base: main ← Head: prs/for-amd-linears

📝 Commits (10+)

6a94830 Add native gfx12 HIP attention and Z-Image INT8 fusions
f8ef6fc Address review feedback for gfx12 HIP kernels
eb6270f reslve code rabbit review comments
e70bc08 Merge branch 'main' into hip-gfx12x-native-attention
638e94c Use atomic WMMA state snapshots, scope ConvRot capability checks to the input device, and preserve float32 validation for HIP attention scales.
99ff431 Fix WMMA races and validate attention scales
8f0b518 Fix WMMA races and validate attention scales
ba44363 Fix Ruff lint
71c5a34 hip: enable native gfx1170 attention and GEMM schedules
9518b7e Merge branch 'main' into hip-gfx12x-native-attention

📊 Changes

24 files changed (+2931 additions, -151 deletions)

View changed files

📝 comfy_kitchen/__init__.py (+92 -0)
📝 comfy_kitchen/backends/eager/__init__.py (+47 -0)
📝 comfy_kitchen/backends/eager/quantization.py (+52 -0)
➕ comfy_kitchen/backends/eager/residual.py (+18 -0)
📝 comfy_kitchen/backends/hip/CMakeLists.txt (+2 -0)
📝 comfy_kitchen/backends/hip/__init__.py (+387 -42)
➕ comfy_kitchen/backends/hip/device_arch.h (+45 -0)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+211 -23)
📝 comfy_kitchen/backends/hip/epilogue.h (+14 -0)
📝 comfy_kitchen/backends/hip/gemm_wmma.h (+71 -10)
📝 comfy_kitchen/backends/hip/hadamard.h (+612 -31)
📝 comfy_kitchen/backends/hip/launchers.h (+13 -0)
📝 comfy_kitchen/backends/hip/mma.h (+1 -1)
📝 comfy_kitchen/backends/hip/ops/gemm_int8.hip (+201 -0)
📝 comfy_kitchen/backends/hip/ops/quantize_int8.hip (+103 -17)
➕ comfy_kitchen/backends/hip/ops/rms_convrot_quant.hip (+41 -0)
➕ comfy_kitchen/backends/hip/ops/rms_gated_residual.hip (+180 -0)
📝 comfy_kitchen/backends/hip/ops/rms_rope.hip (+215 -6)
📝 comfy_kitchen/backends/hip/rope_math.h (+23 -0)
📝 comfy_kitchen/tensor/int8.py (+68 -11)

...and 4 more files

📄 Description

Improve performance for AMD int8 convrot.

Based on work form AMD (preserved in git history).

Check git history for piece by piece changes to generalize and scope down.

Linear Kernel selection is decoupled from fusion choice. This led to a competition between dual-M and the new swiglu implementations with the latter prevailing accross the board and dual-M becoming obsolete
All new fusions are removed. This is a latter phase of development.
- multiple linear support is going to be made transparent to CK, instead its on ComfyUI to concatentate before CK so we can just use optimized single linears.
Multiple hadamard kernel were consolidated onto one new solution that was more performant across the board.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/259 **Author:** [@rattus128](https://github.com/rattus128) **Created:** 10/19/2026 **Status:** ✅ Merged **Merged:** 10/21/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `prs/for-amd-linears` --- ### 📝 Commits (10+) - [`6a94830`](https://github.com/Comfy-Org/comfy-kitchen/commit/6a94830bedf1c2cf59f8b2e1b304e6e05edc884a) Add native gfx12 HIP attention and Z-Image INT8 fusions - [`f8ef6fc`](https://github.com/Comfy-Org/comfy-kitchen/commit/f8ef6fc5958b0f6475927526d3c1b9f9d84184d1) Address review feedback for gfx12 HIP kernels - [`eb6270f`](https://github.com/Comfy-Org/comfy-kitchen/commit/eb6270f317fa5e7b5751a614447eccef52545c57) reslve code rabbit review comments - [`e70bc08`](https://github.com/Comfy-Org/comfy-kitchen/commit/e70bc0878928901e76ec0022523be98c2261c94f) Merge branch 'main' into hip-gfx12x-native-attention - [`638e94c`](https://github.com/Comfy-Org/comfy-kitchen/commit/638e94c247b727b6699e0e4ad0115882a3e4328e) Use atomic WMMA state snapshots, scope ConvRot capability checks to the input device, and preserve float32 validation for HIP attention scales. - [`99ff431`](https://github.com/Comfy-Org/comfy-kitchen/commit/99ff4310c78eec5d9cfd58359092a33e6055eee7) Fix WMMA races and validate attention scales - [`8f0b518`](https://github.com/Comfy-Org/comfy-kitchen/commit/8f0b518a2685fdcadba755f1f4ce821dc1463000) Fix WMMA races and validate attention scales - [`ba44363`](https://github.com/Comfy-Org/comfy-kitchen/commit/ba4436363892e5765cae7e7ef5e07f504eefb1ad) Fix Ruff lint - [`71c5a34`](https://github.com/Comfy-Org/comfy-kitchen/commit/71c5a34ed205dcca53299fd118222b3b74e8ad0c) hip: enable native gfx1170 attention and GEMM schedules - [`9518b7e`](https://github.com/Comfy-Org/comfy-kitchen/commit/9518b7e2b1ca1804d78352c7421f9fbc2383091c) Merge branch 'main' into hip-gfx12x-native-attention ### 📊 Changes **24 files changed** (+2931 additions, -151 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/__init__.py` (+92 -0) 📝 `comfy_kitchen/backends/eager/__init__.py` (+47 -0) 📝 `comfy_kitchen/backends/eager/quantization.py` (+52 -0) ➕ `comfy_kitchen/backends/eager/residual.py` (+18 -0) 📝 `comfy_kitchen/backends/hip/CMakeLists.txt` (+2 -0) 📝 `comfy_kitchen/backends/hip/__init__.py` (+387 -42) ➕ `comfy_kitchen/backends/hip/device_arch.h` (+45 -0) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+211 -23) 📝 `comfy_kitchen/backends/hip/epilogue.h` (+14 -0) 📝 `comfy_kitchen/backends/hip/gemm_wmma.h` (+71 -10) 📝 `comfy_kitchen/backends/hip/hadamard.h` (+612 -31) 📝 `comfy_kitchen/backends/hip/launchers.h` (+13 -0) 📝 `comfy_kitchen/backends/hip/mma.h` (+1 -1) 📝 `comfy_kitchen/backends/hip/ops/gemm_int8.hip` (+201 -0) 📝 `comfy_kitchen/backends/hip/ops/quantize_int8.hip` (+103 -17) ➕ `comfy_kitchen/backends/hip/ops/rms_convrot_quant.hip` (+41 -0) ➕ `comfy_kitchen/backends/hip/ops/rms_gated_residual.hip` (+180 -0) 📝 `comfy_kitchen/backends/hip/ops/rms_rope.hip` (+215 -6) 📝 `comfy_kitchen/backends/hip/rope_math.h` (+23 -0) 📝 `comfy_kitchen/tensor/int8.py` (+68 -11) _...and 4 more files_ </details> ### 📄 Description Improve performance for AMD int8 convrot. Based on work form AMD (preserved in git history). Check git history for piece by piece changes to generalize and scope down. - Linear Kernel selection is decoupled from fusion choice. This led to a competition between dual-M and the new swiglu implementations with the latter prevailing accross the board and dual-M becoming obsolete - All new fusions are removed. This is a latter phase of development. - multiple linear support is going to be made transparent to CK, instead its on ComfyUI to concatentate before CK so we can just use optimized single linears. - Multiple hadamard kernel were consolidated onto one new solution that was more performant across the board. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:22:14 +00:00