[PR #108] [CLOSED] Optimize HIP ConvRot and deep-K INT8 kernels #150

New issue

Closed

opened 2026-10-26 22:20:52 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:52 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/108
Author: @Yasei-no-otoko
Created: 8/11/2026
Status: ❌ Closed

Base: main ← Head: codex/gfx1151-hip-extreme

📝 Commits (3)

db4eb8c Optimize HIP ConvRot and deep-K INT8 kernels
f588fb7 Add HIP ConvRot dispatch and tail checks
3f86661 Use expf for SwiGLU parity

📊 Changes

3 files changed (+84 additions, -27 deletions)

View changed files

📝 comfy_kitchen/backends/hip/hadamard.h (+45 -26)
📝 comfy_kitchen/backends/hip/ops/gemm_int8.hip (+1 -1)
📝 tests/test_hip_wmma.py (+38 -0)

📄 Description

Summary

specialize the HIP ConvRot group size at compile time and unroll its radix-4 stages
use ping-pong LDS buffers to remove the read/write barrier from each butterfly stage
use the HIP fast exponential in the fused SwiGLU load
double the deep-K INT8 WMMA K block from 64 to 128
cover the fused quantizer and deep-K GEMM with a representative MiniMax H3 MLP test

Performance

Measured on an AMD Radeon 8060S (gfx1151) on Windows with PyTorch 2.12.0+rocm10.1.0a20260811 from the ROCm 10.1 Nightly 2026-08-11 package set (torch.version.hip == 7.16.26315). Inputs use BF16 raw SwiGLU [M, 28672], INT8 weights [256, 14336], and ConvRot group size 256.

Each value is the median of three independent run medians. Each run used 10 warmups and 40 HIP-event-timed iterations. The baseline is the published 0.2.30 wheel under the same Python/ROCm process setup.

M	Operation	0.2.30	This PR	Reduction
4,096	fused SwiGLU + ConvRot quantizer	4.0015 ms	3.1761 ms	20.6%
4,096	deep-K INT8 GEMM	1.1796 ms	0.9896 ms	16.1%
4,096	public `int8_linear`	5.1741 ms	4.0788 ms	21.2%
16,500	fused SwiGLU + ConvRot quantizer	15.6379 ms	12.4303 ms	20.5%
16,500	deep-K INT8 GEMM	5.1023 ms	3.7009 ms	27.5%
16,500	public `int8_linear`	21.2995 ms	15.9544 ms	25.1%

These performance results are specific to gfx1151; other WMMA-capable AMD targets were not benchmarked here.

Validation

built the HIP extension for gfx1151 with the ROCm 10.1 Nightly toolchain
pytest tests/test_hip_wmma.py -q: 337 passed
ruff check tests/test_hip_wmma.py
Magpie correctness analysis: hadamard 1.00 and gemm_int8 1.00
ComfyUI startup/API smoke test with the built extension on the same GPU

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/108 **Author:** [@Yasei-no-otoko](https://github.com/Yasei-no-otoko) **Created:** 8/11/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `codex/gfx1151-hip-extreme` --- ### 📝 Commits (3) - [`db4eb8c`](https://github.com/Comfy-Org/comfy-kitchen/commit/db4eb8c0ac36664f053032f244835abbeaf00ca0) Optimize HIP ConvRot and deep-K INT8 kernels - [`f588fb7`](https://github.com/Comfy-Org/comfy-kitchen/commit/f588fb7c9c9747ad8688863f9022d5467809ccfb) Add HIP ConvRot dispatch and tail checks - [`3f86661`](https://github.com/Comfy-Org/comfy-kitchen/commit/3f866610e66993fb27b4fb982454c7599a946ebc) Use expf for SwiGLU parity ### 📊 Changes **3 files changed** (+84 additions, -27 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/hadamard.h` (+45 -26) 📝 `comfy_kitchen/backends/hip/ops/gemm_int8.hip` (+1 -1) 📝 `tests/test_hip_wmma.py` (+38 -0) </details> ### 📄 Description ## Summary - specialize the HIP ConvRot group size at compile time and unroll its radix-4 stages - use ping-pong LDS buffers to remove the read/write barrier from each butterfly stage - use the HIP fast exponential in the fused SwiGLU load - double the deep-K INT8 WMMA K block from 64 to 128 - cover the fused quantizer and deep-K GEMM with a representative MiniMax H3 MLP test ## Performance Measured on an AMD Radeon 8060S (`gfx1151`) on Windows with PyTorch `2.12.0+rocm10.1.0a20260811` from the ROCm 10.1 Nightly 2026-08-11 package set (`torch.version.hip == 7.16.26315`). Inputs use BF16 raw SwiGLU `[M, 28672]`, INT8 weights `[256, 14336]`, and ConvRot group size 256. Each value is the median of three independent run medians. Each run used 10 warmups and 40 HIP-event-timed iterations. The baseline is the published 0.2.30 wheel under the same Python/ROCm process setup. | M | Operation | 0.2.30 | This PR | Reduction | |---:|---|---:|---:|---:| | 4,096 | fused SwiGLU + ConvRot quantizer | 4.0015 ms | 3.1761 ms | 20.6% | | 4,096 | deep-K INT8 GEMM | 1.1796 ms | 0.9896 ms | 16.1% | | 4,096 | public `int8_linear` | 5.1741 ms | 4.0788 ms | 21.2% | | 16,500 | fused SwiGLU + ConvRot quantizer | 15.6379 ms | 12.4303 ms | 20.5% | | 16,500 | deep-K INT8 GEMM | 5.1023 ms | 3.7009 ms | 27.5% | | 16,500 | public `int8_linear` | 21.2995 ms | 15.9544 ms | 25.1% | These performance results are specific to `gfx1151`; other WMMA-capable AMD targets were not benchmarked here. ## Validation - built the HIP extension for `gfx1151` with the ROCm 10.1 Nightly toolchain - `pytest tests/test_hip_wmma.py -q`: 337 passed - `ruff check tests/test_hip_wmma.py` - Magpie correctness analysis: `hadamard` 1.00 and `gemm_int8` 1.00 - ComfyUI startup/API smoke test with the built extension on the same GPU --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:52 +00:00