Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

[PR #229] [MERGED] HIP port of `12389a3`: staged W4A8 requant for fp32 and wide rows #238

New issue

Closed

opened 2026-10-26 22:21:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:53 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/229
Author: @0xDELUXA
Created: 10/1/2026
Status: ✅ Merged
Merged: 10/1/2026
Merged by: @comfyanonymous

Base: main ← Head: amd/hip-w4a8-staged-requant

📝 Commits (1)

1f2aa5b HIP port of 12389a3: staged W4A8 requant for fp32 and wide rows

📊 Changes

5 files changed (+283 additions, -23 deletions)

View changed files

📝 comfy_kitchen/backends/hip/__init__.py (+64 -5)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+29 -0)
📝 comfy_kitchen/backends/hip/launchers.h (+7 -0)
📝 comfy_kitchen/backends/hip/ops/w4a8_requant.hip (+83 -6)
📝 tests/test_hip_wmma.py (+100 -12)

📄 Description

Ports 12389a3 to the HIP backend. This is a performance port: the affected inputs already worked through the eager quantizer, but slowly.

The fused ConvRot requant now stages fp32 weights and rows that do not fit in LDS, mirroring CUDA. Rows are rotated in _QUANT_ROW_ELEM_BUDGET-bounded blocks, then quantize_w4a8_convrot requantizes each block directly into its output slice, keeping only K/16 group scales in LDS. It reuses w4_quantize_row, preserving the fused kernel's division, tie handling, and fp8 rounding. The staged path is limited by CUDA's _W4A8_STAGED_MAX_K; 6-bit and other layouts still use eager. The smaller thread blocks from #216 were already included in the HIP port.

On gfx1200, 4-bit quantization improved from 176.9 ms to 14.8 ms for a 28672x5376 fp32 weight, and 169.7 ms to 15.3 ms for an 8192x28672 bf16 weight too wide for fused LDS.

The fused path is unchanged: 6-bit kernels produce identical ISA, and 4-bit bf16 28672x5376 measures 5.86 ms before vs. 5.91 ms after. Outputs match eager up to fp32 summation order; differences are limited to the expected codebook-tie and rounding behavior.

Tests cover fp32 and wide strided weights, multiple row blocks, fused-to-staged-to-eager fallback, stochastic rounding, and graph capture. The two test_hip_wmma.py failures also occur on main. Run-verified on gfx1200 and compile-verified as a fat build for all 17 targets in architectures.json, with no scratch on gfx1030, gfx1100, gfx1151, gfx1200 and gfx1201.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/229 **Author:** [@0xDELUXA](https://github.com/0xDELUXA) **Created:** 10/1/2026 **Status:** ✅ Merged **Merged:** 10/1/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `amd/hip-w4a8-staged-requant` --- ### 📝 Commits (1) - [`1f2aa5b`](https://github.com/Comfy-Org/comfy-kitchen/commit/1f2aa5bf92e04b7d28f9efcaa4b2394d0209ed1d) HIP port of 12389a3: staged W4A8 requant for fp32 and wide rows ### 📊 Changes **5 files changed** (+283 additions, -23 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/__init__.py` (+64 -5) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+29 -0) 📝 `comfy_kitchen/backends/hip/launchers.h` (+7 -0) 📝 `comfy_kitchen/backends/hip/ops/w4a8_requant.hip` (+83 -6) 📝 `tests/test_hip_wmma.py` (+100 -12) </details> ### 📄 Description Ports 12389a3 to the HIP backend. This is a performance port: the affected inputs already worked through the eager quantizer, but slowly. The fused ConvRot requant now stages fp32 weights and rows that do not fit in LDS, mirroring CUDA. Rows are rotated in `_QUANT_ROW_ELEM_BUDGET`-bounded blocks, then `quantize_w4a8_convrot` requantizes each block directly into its output slice, keeping only `K/16` group scales in LDS. It reuses `w4_quantize_row`, preserving the fused kernel's division, tie handling, and fp8 rounding. The staged path is limited by CUDA's `_W4A8_STAGED_MAX_K`; 6-bit and other layouts still use eager. The smaller thread blocks from #216 were already included in the HIP port. On gfx1200, 4-bit quantization improved from 176.9 ms to 14.8 ms for a 28672x5376 fp32 weight, and 169.7 ms to 15.3 ms for an 8192x28672 bf16 weight too wide for fused LDS. The fused path is unchanged: 6-bit kernels produce identical ISA, and 4-bit bf16 28672x5376 measures 5.86 ms before vs. 5.91 ms after. Outputs match eager up to fp32 summation order; differences are limited to the expected codebook-tie and rounding behavior. Tests cover fp32 and wide strided weights, multiple row blocks, fused-to-staged-to-eager fallback, stochastic rounding, and graph capture. The two `test_hip_wmma.py` failures also occur on `main`. Run-verified on gfx1200 and compile-verified as a fat build for all 17 targets in `architectures.json`, with no scratch on gfx1030, gfx1100, gfx1151, gfx1200 and gfx1201. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:53 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-26 22:22:05 +00:00

[PR #238] [Feat] [RDNA1] Add RDNA1 INT8 Attention backend separate from the existing HIP backend #250

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#238

No description provided.

Rows
Columns

[PR #229] [MERGED] HIP port of 12389a3: staged W4A8 requant for fp32 and wide rows #238

📋 Pull Request Information

📝 Commits (1)

📊 Changes

📄 Description

[PR #229] [MERGED] HIP port of `12389a3`: staged W4A8 requant for fp32 and wide rows #238