[PR #206] [HIP] Integrate iGPU Optimizations (gfx1035 + gfx1103) #225

New issue

Open

opened 2026-10-26 22:21:45 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:45 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/206
Author: @LuXuxue
Created: 9/26/2026
Status: 🔄 Open

Base: main ← Head: main

📝 Commits (10+)

7c6d845 调整配置
210cdb9 提升gemm性能
3ec28e2 融合 RMSNorm+RoPE 重写
d1eac4b hip: int8 attention to the SageAttention gfx110x schedule
847152b hip: fix v buffer overrun in the short-key direct attention path
c83392a hip: gate the 780M tuning behind the gfx1103 architecture
0998ff6 优化gfx1103性能
b3ccecd hip: gate all gfx1103 iGPU tuning behind architecture selection
eb66bb2 Merge remote-tracking branch 'upstream/main'
6a712c2 hip: route partial-key-tile D128 attention to upstream int8_attn

📊 Changes

62 files changed (+14478 additions, -573 deletions)

View changed files

📝 comfy_kitchen/__init__.py (+55 -1)
📝 comfy_kitchen/backends/eager/conv3d.py (+16 -3)
📝 comfy_kitchen/backends/eager/quantization.py (+62 -0)
📝 comfy_kitchen/backends/hip/CMakeLists.txt (+47 -0)
📝 comfy_kitchen/backends/hip/__init__.py (+1632 -105)
📝 comfy_kitchen/backends/hip/architecture_config.h.in (+6 -0)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+470 -36)
➕ comfy_kitchen/backends/hip/dlpack_bindings_gfx103x.cpp (+436 -0)
📝 comfy_kitchen/backends/hip/epilogue.h (+84 -13)
➕ comfy_kitchen/backends/hip/gemm_gfx10.h (+751 -0)
📝 comfy_kitchen/backends/hip/gemm_wmma.h (+417 -19)
📝 comfy_kitchen/backends/hip/hadamard.h (+598 -80)
📝 comfy_kitchen/backends/hip/launchers.h (+176 -0)
📝 comfy_kitchen/backends/hip/mma.h (+119 -31)
➕ comfy_kitchen/backends/hip/mma_gfx103x.h (+674 -0)
➕ comfy_kitchen/backends/hip/nvfp4_gfx10.h (+840 -0)
➕ comfy_kitchen/backends/hip/nvfp4_utils.h (+391 -0)
📝 comfy_kitchen/backends/hip/ops/conv3d_fp16.hip (+59 -6)
📝 comfy_kitchen/backends/hip/ops/convrot_w4a4.hip (+83 -4)
📝 comfy_kitchen/backends/hip/ops/flash_decode.hip (+4 -4)

...and 42 more files

📄 Description

This PR integrates all iGPU-specific optimizations from the comfy-kitchen-igpu fork into the mainline. The fork adds support for AMD Radeon integrated GPUs (RDNA2 gfx103x and RDNA3 gfx1103) with performance competitive to discrete GPUs, while preserving full upstream compatibility for discrete AMD and NVIDIA hardware.

1. New Hardware Support

1.1 RDNA2 (gfx103x) — VALU Extension

No matrix cores on RDNA2. This adds a completely separate extension using VALU v_dot4_i32_i8 and v_dot2_f32_f16 instructions.

1.2 RDNA3 iGPU (gfx1103) — WMMA Tuning

The Radeon 780M (6 WGPs, 128 CU) gets architecture-specific tuning behind COMFY_MMA_GFX1103:

2. Performance Results

Sageatten used below is a iGPU optimized fork, used to show how much gemm get improved.

gfx1103 (Radeon 780M, RDNA3 iGPU, 64GB RAM, Windows 11)

Workload	Speed	Notes
SDXL(int8) 1024x1536 cfg=4 steps=30 upstream gemm+upstream ckatten	3.27 s/it	baseline
SDXL(int8) 1024x1536 cfg=4 steps=30 upstream gemm+sageatten	3.22 s/it	0.98x
SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten	2.86 s/it	0.87x
SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten	2.89 s/it	0.88x
Anima(int8) 1024x1536 cfg=4 steps=30 upstream gemm+upstream ckatten	8.29 s/it	baseline
Anima(int8) 1024x1536 cfg=4 steps=30 upstream gemm+sageatten	8.01 s/it	0.97x
Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten	7.55 s/it	0.91x
Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten	7.46 s/it	0.90x

gfx1035 (Radeon 680M, RDNA2 iGPU, 32GB RAM, Windows 11)

Workload	Speed	Notes
SDXL(int8) 1024x1536 cfg=4 steps=30 upstream gemm(triton backend)+sageatten	6.41 s/it	baseline
SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten	5.84 s/it	0.91x
SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten	5.90 s/it	0.92x
Anima(int8) 1024x1536 cfg=4 steps=30 upstream gemm(triton backend)+sageatten	18.16 s/it	baseline
Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten	17.44 s/it	0.96x
Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten	16.95 s/it	0.93x

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/206 **Author:** [@LuXuxue](https://github.com/LuXuxue) **Created:** 9/26/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (10+) - [`7c6d845`](https://github.com/Comfy-Org/comfy-kitchen/commit/7c6d8457b457665e7677e4f302d2df237eaa044c) 调整配置 - [`210cdb9`](https://github.com/Comfy-Org/comfy-kitchen/commit/210cdb949fca9394d5f3fc4d7fc9eb79bed91fe3) 提升gemm性能 - [`3ec28e2`](https://github.com/Comfy-Org/comfy-kitchen/commit/3ec28e2c23a9f5ef98ead5329a16c121e910a732) 融合 RMSNorm+RoPE 重写 - [`d1eac4b`](https://github.com/Comfy-Org/comfy-kitchen/commit/d1eac4be890ad3da010d156eacab49d0d85a50eb) hip: int8 attention to the SageAttention gfx110x schedule - [`847152b`](https://github.com/Comfy-Org/comfy-kitchen/commit/847152b4642f75167d4ac5b0e57ae326649b1845) hip: fix v buffer overrun in the short-key direct attention path - [`c83392a`](https://github.com/Comfy-Org/comfy-kitchen/commit/c83392a27775006d8a826b9a7d9783c4b4c451a3) hip: gate the 780M tuning behind the gfx1103 architecture - [`0998ff6`](https://github.com/Comfy-Org/comfy-kitchen/commit/0998ff66964a03e3ce701fbdc0ec14a51c9f78b9) 优化gfx1103性能 - [`b3ccecd`](https://github.com/Comfy-Org/comfy-kitchen/commit/b3ccecdaabd6b83e49dd5c75bbc1268c51454b35) hip: gate all gfx1103 iGPU tuning behind architecture selection - [`eb66bb2`](https://github.com/Comfy-Org/comfy-kitchen/commit/eb66bb2ea4ff39aff145c7ad7df2a47d97152814) Merge remote-tracking branch 'upstream/main' - [`6a712c2`](https://github.com/Comfy-Org/comfy-kitchen/commit/6a712c22a82cdb27e604d701e8e46863fdc13169) hip: route partial-key-tile D128 attention to upstream int8_attn ### 📊 Changes **62 files changed** (+14478 additions, -573 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/__init__.py` (+55 -1) 📝 `comfy_kitchen/backends/eager/conv3d.py` (+16 -3) 📝 `comfy_kitchen/backends/eager/quantization.py` (+62 -0) 📝 `comfy_kitchen/backends/hip/CMakeLists.txt` (+47 -0) 📝 `comfy_kitchen/backends/hip/__init__.py` (+1632 -105) 📝 `comfy_kitchen/backends/hip/architecture_config.h.in` (+6 -0) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+470 -36) ➕ `comfy_kitchen/backends/hip/dlpack_bindings_gfx103x.cpp` (+436 -0) 📝 `comfy_kitchen/backends/hip/epilogue.h` (+84 -13) ➕ `comfy_kitchen/backends/hip/gemm_gfx10.h` (+751 -0) 📝 `comfy_kitchen/backends/hip/gemm_wmma.h` (+417 -19) 📝 `comfy_kitchen/backends/hip/hadamard.h` (+598 -80) 📝 `comfy_kitchen/backends/hip/launchers.h` (+176 -0) 📝 `comfy_kitchen/backends/hip/mma.h` (+119 -31) ➕ `comfy_kitchen/backends/hip/mma_gfx103x.h` (+674 -0) ➕ `comfy_kitchen/backends/hip/nvfp4_gfx10.h` (+840 -0) ➕ `comfy_kitchen/backends/hip/nvfp4_utils.h` (+391 -0) 📝 `comfy_kitchen/backends/hip/ops/conv3d_fp16.hip` (+59 -6) 📝 `comfy_kitchen/backends/hip/ops/convrot_w4a4.hip` (+83 -4) 📝 `comfy_kitchen/backends/hip/ops/flash_decode.hip` (+4 -4) _...and 42 more files_ </details> ### 📄 Description This PR integrates **all iGPU-specific optimizations** from the `comfy-kitchen-igpu` fork into the mainline. The fork adds support for **AMD Radeon integrated GPUs** (RDNA2 gfx103x and RDNA3 gfx1103) with performance competitive to discrete GPUs, while preserving full upstream compatibility for discrete AMD and NVIDIA hardware. --- ## 1. New Hardware Support ### 1.1 RDNA2 (gfx103x) — VALU Extension **No matrix cores** on RDNA2. This adds a completely separate extension using VALU `v_dot4_i32_i8` and `v_dot2_f32_f16` instructions. ### 1.2 RDNA3 iGPU (gfx1103) — WMMA Tuning The Radeon 780M (6 WGPs, 128 CU) gets architecture-specific tuning behind `COMFY_MMA_GFX1103`: --- ## 2. Performance Results Sageatten used below is a [iGPU optimized fork](https://github.com/LuXuxue/SageAttention), used to show how much gemm get improved. ### gfx1103 (Radeon 780M, RDNA3 iGPU, 64GB RAM, Windows 11) | Workload | Speed | Notes | |----------|-----------|-------| | SDXL(int8) 1024x1536 cfg=4 steps=30 upstream gemm+upstream ckatten | 3.27 s/it | baseline | | SDXL(int8) 1024x1536 cfg=4 steps=30 upstream gemm+sageatten | 3.22 s/it | 0.98x | | SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten | **2.86 s/it** | **0.87x** | | SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten | 2.89 s/it | 0.88x | | Anima(int8) 1024x1536 cfg=4 steps=30 upstream gemm+upstream ckatten | 8.29 s/it | baseline | | Anima(int8) 1024x1536 cfg=4 steps=30 upstream gemm+sageatten | 8.01 s/it | 0.97x | | Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten | 7.55 s/it | 0.91x | | Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten | **7.46 s/it** | **0.90x** | ### gfx1035 (Radeon 680M, RDNA2 iGPU, 32GB RAM, Windows 11) | Workload | Speed | Notes | |----------|-----------|-------| | SDXL(int8) 1024x1536 cfg=4 steps=30 upstream gemm(triton backend)+sageatten | 6.41 s/it | baseline | | SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten | **5.84 s/it** | **0.91x** | | SDXL(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten | 5.90 s/it | 0.92x | | Anima(int8) 1024x1536 cfg=4 steps=30 upstream gemm(triton backend)+sageatten | 18.16 s/it | baseline | | Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+sageatten | 17.44 s/it | 0.96x | | Anima(int8) 1024x1536 cfg=4 steps=30 fork gemm+fork ckatten | **16.95 s/it** | **0.93x** | --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 22:21:45 +00:00

bsakaguchi referenced this issue

2026-10-26 22:21:53 +00:00

[PR #225] [MERGED] HIP port of #216: fused ConvRot requant kernel for W4A8/W6A8 #237