Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

4) #158

New issue

Closed

opened 2026-10-26 22:20:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:58 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/116
Author: @tvukovic-amd
Created: 8/13/2026
Status: ✅ Merged
Merged: 8/18/2026
Merged by: @comfyanonymous

Base: main ← Head: convrot-int8-quant-fused

📝 Commits (10+)

acbcad3 HIP: fused and global spill ConvRot INT8 quant for G=256
9640968 HIP ConvRot INT8: pick fused block size by LDS fit before spill
7c490ec Merge branch 'Comfy-Org:main' into convrot-int8-quant-fused
4e48439 Fix HIP ConvRot spill workspace and LDS dispatch per review.
8fc007c Validate spill dtype and allocate spill workspace only when needed.
60828a7 Pin convrot perf microbench to the tensor device for route probe.
8ca5b7b Fix fused convrot None spill args and dtype rounding contract.
8a6748a Fix ruff import formatting in convrot perf test
e2a4297 Align ConvRot LDS dispatch with RowT element size, pass dtype into spill probe, and tune K=10240 block size.
d24e341 Address PR review: block-aware legacy LDS sizing, fresh per-device LDS query, conservative needs_spill probe.

📊 Changes

7 files changed (+1036 additions, -72 deletions)

View changed files

📝 comfy_kitchen/backends/hip/__init__.py (+37 -18)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+25 -4)
📝 comfy_kitchen/backends/hip/hadamard.h (+622 -34)
📝 comfy_kitchen/backends/hip/ops/quantize_int8.hip (+65 -5)
📝 tests/conftest.py (+1 -0)
➕ tests/test_hip_convrot_perf.py (+220 -0)
📝 tests/test_hip_wmma.py (+66 -11)

📄 Description

Summary

Adds a fused HIP GPU path for INT8 ConvRot G=256 activation quantization on WMMA-capable RDNA3 / RDNA3.5 / RDNA4 GPUs: a fused single-kernel path when the rotated row fits in LDS, and a global spill path when it does not.

Replaces slow legacy HIP and eager Python Hadamard + quant paths for large-K layers (e.g. Z-Image Turbo INT8 at K=3840/10240).

Fused single-kernel path when the rotated row fits in LDS (typical K=3840).
Global spill when no fused block configuration fits in LDS.
Shape-aware dispatch: picks block size (64/512/640/768/1024) by per-(M,K) LDS fit, trying narrower blocks before spilling — keeps K=10240 on a single fused kernel on RDNA dGPUs (e.g. 512-thread config on gfx1201) instead of 2-kernel spill.

Target workload: int8_linear(..., convrot=True, convrot_groupsize=256)

Changes

hadamard.h: fused convrot_quant_fused_kernel (FHT stage64, G=256); global spill kernels; legacy G=16/64/int4 unchanged; convrot_pick_fused_block_threads() for LDS-aware dispatch.
quantize_int8.hip: global spill workspace (hipMalloc) for large K when fused does not fit.
hip/__init__.py: route INT8 ConvRot G=256 through fused/spill via _rotate_quant_int8; int8_global_spill in _convrot_supported so Python no longer rejects large K.
Tests: correctness K=3840/10240; microbench vs eager; performance pytest marker.

Running tests

Correctness — fused LDS (K=3840) and large K (K=10240) vs eager:

pytest tests/test_hip_wmma.py::test_int8_linear_convrot_large_k_matches_eager -v

Performance — Z-Image Turbo INT8 shapes (M=4128, K=3840/10240, G=256); prints timing tables with -s:

pytest tests/test_hip_convrot_perf.py -v -s

Benchmarks (Windows)

Setup

PR branch = fused FHT kernel or global spill (this PR).
main = legacy HIP ConvRot kernel and/or eager Python fallback.
Microbench “Eager ConvRot” = Python Hadamard + quant in the PR checkout.
Z-Image: PR vs main built checkouts, ComfyUI Z-Image Turbo INT8, 8 steps @ 1024², n=1.

gfx1201 (Navi 48 XTX, RDNA4)

Z-Image Turbo INT8

Benchmark	PR branch	main	Uplift (speedup)
Quant / step	57.0 ms	103.6 ms	+45.0% (1.82x)
E2E it/s	1.78 it/s	1.64 it/s	+8.6% (1.09x)

Microbench

Shape	HIP fused/spill	Eager ConvRot	Uplift (speedup)
K=3840 (M=4128, G=256)	0.25 ms	0.65 ms	+61.1% (2.57x)
K=10240 (M=4128, G=256)	0.42 ms	2.05 ms	+79.3% (4.83x)

gfx1100 (7900 XTX, RDNA3)

Z-Image Turbo INT8

Benchmark	PR branch	main	Uplift (speedup)
Quant / step	57.6 ms	85.9 ms	+32.9% (1.49x)
E2E it/s	1.28 it/s	1.21 it/s	+5.3% (1.06x)

Microbench

Shape	HIP fused/spill	Eager ConvRot	Uplift (speedup)
K=3840 (M=4128, G=256)	0.31 ms	0.75 ms	+58.6% (2.41x)
K=10240 (M=4128, G=256)	0.35 ms	1.61 ms	+78.5% (4.66x)

gfx1151 (Strix Halo, RDNA3.5)

Z-Image Turbo INT8

Benchmark	PR branch	main	Uplift (speedup)
Quant / step	95.4 ms	151.8 ms	+37.2% (1.59x)
E2E it/s	0.54 it/s	0.52 it/s	+2.9% (1.04x)

Microbench

Shape	HIP fused/spill	Eager ConvRot	Uplift (speedup)
K=3840 (M=4128, G=256)	0.37 ms	1.81 ms	+79.4% (4.87x)
K=10240 (M=4128, G=256)	0.79 ms	5.29 ms	+85.0% (6.66x)

gfx1150 (Strix Point APU, RDNA3.5)

Z-Image Turbo INT8

Benchmark	PR branch	main	Uplift (speedup)
Quant / step	265.7 ms	340.3 ms	+21.9% (1.28x)
E2E it/s	0.16 it/s	0.16 it/s	+2.3% (1.02x)

Microbench

Shape	HIP fused/spill	Eager ConvRot	Uplift (speedup)
K=3840 (M=4128, G=256)	0.83 ms	5.56 ms	+85.0% (6.67x)
K=10240 (M=4128, G=256)	4.19 ms	12.85 ms	+67.4% (3.07x)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/116 **Author:** [@tvukovic-amd](https://github.com/tvukovic-amd) **Created:** 8/13/2026 **Status:** ✅ Merged **Merged:** 8/18/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `convrot-int8-quant-fused` --- ### 📝 Commits (10+) - [`acbcad3`](https://github.com/Comfy-Org/comfy-kitchen/commit/acbcad34e8d5de716efae7dcdc83e352344d8616) HIP: fused and global spill ConvRot INT8 quant for G=256 - [`9640968`](https://github.com/Comfy-Org/comfy-kitchen/commit/964096862eea26fb99e0dd51e53887b6f2a42346) HIP ConvRot INT8: pick fused block size by LDS fit before spill - [`7c490ec`](https://github.com/Comfy-Org/comfy-kitchen/commit/7c490ecc9d43815502723579a16fa510033a8060) Merge branch 'Comfy-Org:main' into convrot-int8-quant-fused - [`4e48439`](https://github.com/Comfy-Org/comfy-kitchen/commit/4e48439316f1964b9137a868e0c3e884393b57ac) Fix HIP ConvRot spill workspace and LDS dispatch per review. - [`8fc007c`](https://github.com/Comfy-Org/comfy-kitchen/commit/8fc007c790cf4abcaea2c221555317dc6d95d003) Validate spill dtype and allocate spill workspace only when needed. - [`60828a7`](https://github.com/Comfy-Org/comfy-kitchen/commit/60828a71fc5f90ee67648a1aae6310a32d74d5af) Pin convrot perf microbench to the tensor device for route probe. - [`8ca5b7b`](https://github.com/Comfy-Org/comfy-kitchen/commit/8ca5b7b7035d17baa43a339aec878725991bca7e) Fix fused convrot None spill args and dtype rounding contract. - [`8a6748a`](https://github.com/Comfy-Org/comfy-kitchen/commit/8a6748a4433b1cfcc50f09a794eaa97a7a9ec5dd) Fix ruff import formatting in convrot perf test - [`e2a4297`](https://github.com/Comfy-Org/comfy-kitchen/commit/e2a4297251d1e91860b456dc969595556be82de1) Align ConvRot LDS dispatch with RowT element size, pass dtype into spill probe, and tune K=10240 block size. - [`d24e341`](https://github.com/Comfy-Org/comfy-kitchen/commit/d24e3413844c293baf4fe274033f75cbe5fbaee8) Address PR review: block-aware legacy LDS sizing, fresh per-device LDS query, conservative needs_spill probe. ### 📊 Changes **7 files changed** (+1036 additions, -72 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/__init__.py` (+37 -18) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+25 -4) 📝 `comfy_kitchen/backends/hip/hadamard.h` (+622 -34) 📝 `comfy_kitchen/backends/hip/ops/quantize_int8.hip` (+65 -5) 📝 `tests/conftest.py` (+1 -0) ➕ `tests/test_hip_convrot_perf.py` (+220 -0) 📝 `tests/test_hip_wmma.py` (+66 -11) </details> ### 📄 Description ## Summary Adds a fused HIP GPU path for INT8 ConvRot G=256 activation quantization on WMMA-capable **RDNA3 / RDNA3.5 / RDNA4** GPUs: a fused single-kernel path when the rotated row fits in LDS, and a global spill path when it does not. Replaces slow **legacy HIP** and **eager Python** Hadamard + quant paths for large-K layers (e.g. Z-Image Turbo INT8 at K=3840/10240). - **Fused single-kernel** path when the rotated row fits in LDS (typical K=3840). - **Global spill** when no fused block configuration fits in LDS. - **Shape-aware dispatch**: picks block size (64/512/640/768/1024) by per-(M,K) LDS fit, trying narrower blocks before spilling — keeps K=10240 on a **single fused kernel** on RDNA dGPUs (e.g. 512-thread config on gfx1201) instead of 2-kernel spill. Target workload: `int8_linear(..., convrot=True, convrot_groupsize=256)` ## Changes - **`hadamard.h`**: fused `convrot_quant_fused_kernel` (FHT stage64, G=256); global spill kernels; legacy G=16/64/int4 unchanged; `convrot_pick_fused_block_threads()` for LDS-aware dispatch. - **`quantize_int8.hip`**: global spill workspace (`hipMalloc`) for large K when fused does not fit. - **`hip/__init__.py`**: route INT8 ConvRot G=256 through fused/spill via `_rotate_quant_int8`; `int8_global_spill` in `_convrot_supported` so Python no longer rejects large K. - **Tests**: correctness K=3840/10240; microbench vs eager; `performance` pytest marker. ## Running tests **Correctness** — fused LDS (K=3840) and large K (K=10240) vs eager: ```bash pytest tests/test_hip_wmma.py::test_int8_linear_convrot_large_k_matches_eager -v ``` **Performance** — Z-Image Turbo INT8 shapes (M=4128, K=3840/10240, G=256); prints timing tables with `-s`: ```bash pytest tests/test_hip_convrot_perf.py -v -s ``` ## Benchmarks (Windows) **Setup** - **PR branch** = fused FHT kernel or global spill (this PR). - **main** = legacy HIP ConvRot kernel and/or eager Python fallback. - **Microbench “Eager ConvRot”** = Python Hadamard + quant in the PR checkout. - **Z-Image:** PR vs `main` built checkouts, ComfyUI Z-Image Turbo INT8, 8 steps @ 1024², n=1. ### gfx1201 (Navi 48 XTX, RDNA4) #### Z-Image Turbo INT8 | Benchmark | PR branch | main | Uplift (speedup) | |-----------|-----------|------|------------------| | Quant / step | 57.0 ms | 103.6 ms | +45.0% (1.82x) | | E2E it/s | 1.78 it/s | 1.64 it/s | +8.6% (1.09x) | #### Microbench | Shape | HIP fused/spill | Eager ConvRot | Uplift (speedup) | |-------|-----------------|---------------|------------------| | K=3840 (M=4128, G=256) | 0.25 ms | 0.65 ms | +61.1% (2.57x) | | K=10240 (M=4128, G=256) | 0.42 ms | 2.05 ms | +79.3% (4.83x) | ### gfx1100 (7900 XTX, RDNA3) #### Z-Image Turbo INT8 | Benchmark | PR branch | main | Uplift (speedup) | |-----------|-----------|------|------------------| | Quant / step | 57.6 ms | 85.9 ms | +32.9% (1.49x) | | E2E it/s | 1.28 it/s | 1.21 it/s | +5.3% (1.06x) | #### Microbench | Shape | HIP fused/spill | Eager ConvRot | Uplift (speedup) | |-------|-----------------|---------------|------------------| | K=3840 (M=4128, G=256) | 0.31 ms | 0.75 ms | +58.6% (2.41x) | | K=10240 (M=4128, G=256) | 0.35 ms | 1.61 ms | +78.5% (4.66x) | ### gfx1151 (Strix Halo, RDNA3.5) #### Z-Image Turbo INT8 | Benchmark | PR branch | main | Uplift (speedup) | |-----------|-----------|------|------------------| | Quant / step | 95.4 ms | 151.8 ms | +37.2% (1.59x) | | E2E it/s | 0.54 it/s | 0.52 it/s | +2.9% (1.04x) | #### Microbench | Shape | HIP fused/spill | Eager ConvRot | Uplift (speedup) | |-------|-----------------|---------------|------------------| | K=3840 (M=4128, G=256) | 0.37 ms | 1.81 ms | +79.4% (4.87x) | | K=10240 (M=4128, G=256) | 0.79 ms | 5.29 ms | +85.0% (6.66x) | ### gfx1150 (Strix Point APU, RDNA3.5) #### Z-Image Turbo INT8 | Benchmark | PR branch | main | Uplift (speedup) | |-----------|-----------|------|------------------| | Quant / step | 265.7 ms | 340.3 ms | +21.9% (1.28x) | | E2E it/s | 0.16 it/s | 0.16 it/s | +2.3% (1.02x) | #### Microbench | Shape | HIP fused/spill | Eager ConvRot | Uplift (speedup) | |-------|-----------------|---------------|------------------| | K=3840 (M=4128, G=256) | 0.83 ms | 5.56 ms | +85.0% (6.67x) | | K=10240 (M=4128, G=256) | 4.19 ms | 12.85 ms | +67.4% (3.07x) | --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:58 +00:00