[PR #225] [MERGED] HIP port of #216: fused ConvRot requant kernel for W4A8/W6A8 #237

New issue

Closed

opened 2026-10-26 22:21:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:53 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/225
Author: @0xDELUXA
Created: 9/30/2026
Status: ✅ Merged
Merged: 9/30/2026
Merged by: @comfyanonymous

Base: main ← Head: amd/hip-fused-requant

📝 Commits (1)

b367c4d HIP port of #216: fused ConvRot requant kernel for W4A8/W6A8

📊 Changes

5 files changed (+693 additions, -370 deletions)

View changed files

📝 comfy_kitchen/backends/hip/__init__.py (+60 -56)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+38 -23)
📝 comfy_kitchen/backends/hip/launchers.h (+11 -8)
📝 comfy_kitchen/backends/hip/ops/w4a8_requant.hip (+459 -177)
📝 tests/test_hip_wmma.py (+125 -106)

📄 Description

Performance port of #216. The HIP requantize now rotates and quantizes the raw weight in one launch for 4-bit codebook g16 and 6-bit g16/32/64 (fp8 s_rel, scale search off), replacing the eager rotation plus the old 4-bit-only kernel. 6-bit no longer falls to the chunked eager quantizer. Declines mirror CUDA: fp32 weights, fp32 scales, asymmetric, other group sizes and rows past the LDS budget stay on eager.

Measured on gfx1200, bf16, end to end through quantize_w4a8_int8_weight: H3 fc1 [28672, 5376] goes 7.8 -> 6.2 ms at 4 bits and 270 -> 7.8 ms at 6 bits g32. fc2 [5376, 14336] goes 3.8 -> 2.6 ms and 21.6 -> 3.4 ms.

Two places differ from the CUDA kernel on purpose, both toward eager. Divisions follow torch's semantics (correctly rounded tensor-by-tensor, reciprocal multiply for the scalar ones), because -ffast-math approximates x / y, and bf16 rows hit exact .5 ties in w / gs often enough to move a whole row's s_channel by about 1%. The 4-bit assignment returns the last copy of a repeated int8 level, as _assign_grid does. With both, the 6-bit output is bit-identical to eager on every tested shape, and 4-bit packed codes match eager byte for byte. The only remaining 4-bit difference is an occasional single e4m3 step from fp32 summation order. The block is also capped at the group count rounded up to a wave, because threads past it sit idle through both quantize phases.

Compile-verified for gfx1030, gfx1100 and gfx1200 with no scratch in any instantiation. Run-verified on gfx1200 only.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/225 **Author:** [@0xDELUXA](https://github.com/0xDELUXA) **Created:** 9/30/2026 **Status:** ✅ Merged **Merged:** 9/30/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `amd/hip-fused-requant` --- ### 📝 Commits (1) - [`b367c4d`](https://github.com/Comfy-Org/comfy-kitchen/commit/b367c4d5df45519f3a0e98b7c5f419127642ac98) HIP port of #216: fused ConvRot requant kernel for W4A8/W6A8 ### 📊 Changes **5 files changed** (+693 additions, -370 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/__init__.py` (+60 -56) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+38 -23) 📝 `comfy_kitchen/backends/hip/launchers.h` (+11 -8) 📝 `comfy_kitchen/backends/hip/ops/w4a8_requant.hip` (+459 -177) 📝 `tests/test_hip_wmma.py` (+125 -106) </details> ### 📄 Description Performance port of #216. The HIP requantize now rotates and quantizes the raw weight in one launch for 4-bit codebook g16 and 6-bit g16/32/64 (fp8 `s_rel`, scale search off), replacing the eager rotation plus the old 4-bit-only kernel. 6-bit no longer falls to the chunked eager quantizer. Declines mirror CUDA: fp32 weights, fp32 scales, asymmetric, other group sizes and rows past the LDS budget stay on eager. Measured on gfx1200, bf16, end to end through `quantize_w4a8_int8_weight`: H3 fc1 `[28672, 5376]` goes 7.8 -> 6.2 ms at 4 bits and 270 -> 7.8 ms at 6 bits g32. fc2 `[5376, 14336]` goes 3.8 -> 2.6 ms and 21.6 -> 3.4 ms. Two places differ from the CUDA kernel on purpose, both toward eager. Divisions follow torch's semantics (correctly rounded tensor-by-tensor, reciprocal multiply for the scalar ones), because `-ffast-math` approximates `x / y`, and bf16 rows hit exact .5 ties in `w / gs` often enough to move a whole row's `s_channel` by about 1%. The 4-bit assignment returns the last copy of a repeated int8 level, as `_assign_grid` does. With both, the 6-bit output is bit-identical to eager on every tested shape, and 4-bit packed codes match eager byte for byte. The only remaining 4-bit difference is an occasional single e4m3 step from fp32 summation order. The block is also capped at the group count rounded up to a wave, because threads past it sit idle through both quantize phases. Compile-verified for gfx1030, gfx1100 and gfx1200 with no scratch in any instantiation. Run-verified on gfx1200 only. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:53 +00:00