[PR #257] [MERGED] HIP port of #255: fuse input activations into the W4A8/W6A8 quantizer #262

New issue

Closed

opened 2026-10-26 22:22:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:22:13 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/257
Author: @0xDELUXA
Created: 10/19/2026
Status: ✅ Merged
Merged: 10/21/2026
Merged by: @comfyanonymous

Base: main ← Head: amd/hip-w4a8-fused-input-act

📝 Commits (1)

2010047 HIP port of #255: fuse input activations into the W4A8/W6A8 quantizer

📊 Changes

2 files changed (+84 additions, -25 deletions)

View changed files

📝 comfy_kitchen/backends/hip/__init__.py (+50 -25)
📝 tests/test_hip_wmma.py (+34 -0)

📄 Description

HIP port of #255. w4a8_int8_linear applied the input activation as a separate pass before the ConvRot quantizer, which already fuses gelu_tanh, swiglu and situ on every group size and the global spill path, plus rms_norm on the fused G=256 kernel. The activation now goes to the quantizer. HIP quantizes every row count with that kernel, so decode rows are covered too, not only M >= 512. The int8_linear logic that picks which activation to fuse moved into a shared _split_input_act helper; the asymmetric correction layout and unsupported ConvRot shapes still apply the activation eagerly.

Measured on gfx1200, bf16, N=3072, K=8192: situ at M=4096 goes 7.30 -> 3.35 ms and at M=1024 1.65 -> 1.09 ms, swiglu at M=4096 3.94 -> 3.01 ms. rms_norm is 2-10% faster across the shapes tried.

test_w4a8_linear_fuses_the_input_act covers the four activations at 4 and 6 bits for M=1, 9 and 600, and fails if the eager activation runs. No kernel changes: the fused activation variants are the ones int8_linear already launches, so the device code is identical to main for every arch.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/257 **Author:** [@0xDELUXA](https://github.com/0xDELUXA) **Created:** 10/19/2026 **Status:** ✅ Merged **Merged:** 10/21/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `amd/hip-w4a8-fused-input-act` --- ### 📝 Commits (1) - [`2010047`](https://github.com/Comfy-Org/comfy-kitchen/commit/2010047ea966492ed50792ab6c30513559b4cf72) HIP port of #255: fuse input activations into the W4A8/W6A8 quantizer ### 📊 Changes **2 files changed** (+84 additions, -25 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/__init__.py` (+50 -25) 📝 `tests/test_hip_wmma.py` (+34 -0) </details> ### 📄 Description HIP port of #255. `w4a8_int8_linear` applied the input activation as a separate pass before the ConvRot quantizer, which already fuses `gelu_tanh`, `swiglu` and `situ` on every group size and the global spill path, plus `rms_norm` on the fused G=256 kernel. The activation now goes to the quantizer. HIP quantizes every row count with that kernel, so decode rows are covered too, not only M >= 512. The `int8_linear` logic that picks which activation to fuse moved into a shared `_split_input_act` helper; the asymmetric `correction` layout and unsupported ConvRot shapes still apply the activation eagerly. Measured on gfx1200, bf16, N=3072, K=8192: `situ` at M=4096 goes 7.30 -> 3.35 ms and at M=1024 1.65 -> 1.09 ms, `swiglu` at M=4096 3.94 -> 3.01 ms. `rms_norm` is 2-10% faster across the shapes tried. `test_w4a8_linear_fuses_the_input_act` covers the four activations at 4 and 6 bits for M=1, 9 and 600, and fails if the eager activation runs. No kernel changes: the fused activation variants are the ones `int8_linear` already launches, so the device code is identical to `main` for every arch. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:22:13 +00:00