[PR #182] [MERGED] HIP port of #176: W4A8 decode GEMV and fused GatedDeltaNet decode #201

New issue

Closed

opened 2026-10-26 22:21:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:32 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/182
Author: @0xDELUXA
Created: 9/16/2026
Status: ✅ Merged
Merged: 9/16/2026
Merged by: @comfyanonymous

Base: main ← Head: amd/hip-w4a8-gemv

📝 Commits (1)

406719b HIP port of #176: W4A8 decode GEMV and fused GatedDeltaNet decode

📊 Changes

10 files changed (+1283 additions, -45 deletions)

View changed files

📝 comfy_kitchen/backends/hip/CMakeLists.txt (+2 -0)
📝 comfy_kitchen/backends/hip/__init__.py (+146 -2)
📝 comfy_kitchen/backends/hip/dlpack_bindings.cpp (+176 -0)
📝 comfy_kitchen/backends/hip/mma.h (+17 -0)
➕ comfy_kitchen/backends/hip/ops/gated_delta.hip (+451 -0)
📝 comfy_kitchen/backends/hip/ops/w4a8_dequant.hip (+2 -41)
➕ comfy_kitchen/backends/hip/ops/w4a8_gemv.hip (+169 -0)
➕ comfy_kitchen/backends/hip/w4a8_codec.h (+70 -0)
📝 comfy_kitchen/gated_delta.py (+35 -2)
📝 tests/test_hip_wmma.py (+215 -0)

📄 Description

Ports the two HIP-compatible parts of #176. ops/w4a8_gemv.hip replaces the chunked path's full INT4->INT8 decode with register-side decoding and v_dot4_i32_iu8, cutting memory traffic while remaining bit-identical via the shared w4a8_codec.h and EpiRowwise epilogue.

ops/gated_delta.hip adds fused GatedDeltaNet decode and depthwise conv. Its fp32 [DK, DV] state does not fit in RDNA LDS, so R threads split each value column across registers, using LDS only for the two DK reductions per step. This keeps state traffic to one read/write per kernel. R is instantiated for 16, 32, and 64 keys/thread.

On RDNA4 bf16:

W4A8 GEMV: 1.9-2.6x faster across tested shapes, reaching ~220 GB/s packed-weight bandwidth.
Fused GatedDelta: 2.7-12.4x faster than eager execution.
Depthwise conv: 2.4-4.9x faster.
DV=512 spills ~300 B/lane but is still 4.7x faster; no shipping model uses this shape.

The warp-shuffle INT8 rowwise convrot quantizer is intentionally not ported, the existing HIP quantizer already uses a shared 64-thread radix-4 FHT path.

All 17 manifest targets compile into one fat binary. ISA checks confirm the expected v_dot4_i32_iu8 use on gfx11/gfx12 and scalar fallback on RDNA2, with no scratch except the noted DV=512 case.

Validation passes on gfx1200: gated-delta tests remain 16/16, and the HIP suite increases from 1938 to 2021 passing, with the same 102 pre-existing failures.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/182 **Author:** [@0xDELUXA](https://github.com/0xDELUXA) **Created:** 9/16/2026 **Status:** ✅ Merged **Merged:** 9/16/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `amd/hip-w4a8-gemv` --- ### 📝 Commits (1) - [`406719b`](https://github.com/Comfy-Org/comfy-kitchen/commit/406719b04492fb09f305267c55fd7ce5c32653c1) HIP port of #176: W4A8 decode GEMV and fused GatedDeltaNet decode ### 📊 Changes **10 files changed** (+1283 additions, -45 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/CMakeLists.txt` (+2 -0) 📝 `comfy_kitchen/backends/hip/__init__.py` (+146 -2) 📝 `comfy_kitchen/backends/hip/dlpack_bindings.cpp` (+176 -0) 📝 `comfy_kitchen/backends/hip/mma.h` (+17 -0) ➕ `comfy_kitchen/backends/hip/ops/gated_delta.hip` (+451 -0) 📝 `comfy_kitchen/backends/hip/ops/w4a8_dequant.hip` (+2 -41) ➕ `comfy_kitchen/backends/hip/ops/w4a8_gemv.hip` (+169 -0) ➕ `comfy_kitchen/backends/hip/w4a8_codec.h` (+70 -0) 📝 `comfy_kitchen/gated_delta.py` (+35 -2) 📝 `tests/test_hip_wmma.py` (+215 -0) </details> ### 📄 Description Ports the two HIP-compatible parts of #176. `ops/w4a8_gemv.hip` replaces the chunked path's full INT4->INT8 decode with register-side decoding and `v_dot4_i32_iu8`, cutting memory traffic while remaining bit-identical via the shared `w4a8_codec.h` and `EpiRowwise` epilogue. `ops/gated_delta.hip` adds fused GatedDeltaNet decode and depthwise conv. Its fp32 `[DK, DV]` state does not fit in RDNA LDS, so `R` threads split each value column across registers, using LDS only for the two DK reductions per step. This keeps state traffic to one read/write per kernel. `R` is instantiated for 16, 32, and 64 keys/thread. On RDNA4 bf16: - W4A8 GEMV: 1.9-2.6x faster across tested shapes, reaching ~220 GB/s packed-weight bandwidth. - Fused GatedDelta: 2.7-12.4x faster than eager execution. - Depthwise conv: 2.4-4.9x faster. - `DV=512` spills ~300 B/lane but is still 4.7x faster; no shipping model uses this shape. The warp-shuffle INT8 rowwise convrot quantizer is intentionally not ported, the existing HIP quantizer already uses a shared 64-thread radix-4 FHT path. All 17 manifest targets compile into one fat binary. ISA checks confirm the expected `v_dot4_i32_iu8` use on gfx11/gfx12 and scalar fallback on RDNA2, with no scratch except the noted `DV=512` case. Validation passes on gfx1200: gated-delta tests remain 16/16, and the HIP suite increases from 1938 to 2021 passing, with the same 102 pre-existing failures. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:32 +00:00