[PR #127] [CLOSED] HIP: vectorize rowwise INT8 quantization with 16-byte loads #162

New issue

Closed

opened 2026-10-26 22:21:02 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:02 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/127
Author: @tangzzycc
Created: 8/21/2026
Status: ❌ Closed

Base: main ← Head: exp/hip-rowwise-int8-vector-load-gfx1151

📝 Commits (1)

6b38b04 Optimize HIP rowwise INT8 quantization with vectorized loads

📊 Changes

2 files changed (+85 additions, -2 deletions)

View changed files

📝 comfy_kitchen/backends/hip/ops/quantize_int8.hip (+70 -2)
📝 tests/test_hip_wmma.py (+15 -0)

📄 Description

Summary

Implements Change 2 proposed in #102:

Load 16 bytes per thread with uint4 for FP32, FP16, and BF16 inputs.
Use the vectorized kernel only when row alignment and K permit it; otherwise
keep the existing scalar fallback.
Preserve the existing rounding, clamping, and scale behavior.

The vectorized path uses the portable HIP uint4 type and has no architecture-specific guard, so it also applies to other supported HIP architectures. Performance was measured on gfx1151.

Performance

Measured on AMD Radeon 8060S (gfx1151), ROCm 7.2, with aligned BF16 inputs:

Shape (M x K)	Before	After	Speedup
4096 x 1024	0.0564 ms	0.0269 ms	2.09x
8192 x 2048	0.4417 ms	0.2196 ms	2.01x
4096 x 4096	0.4107 ms	0.2268 ms	1.81x
18432 x 2048	0.9579 ms	0.4999 ms	1.92x
512 x 14336	0.0996 ms	0.0417 ms	2.39x
4096 x 14336	1.3498 ms	0.8246 ms	1.64x

Geometric-mean speedup: 1.96x.

For large BF16 shapes, one-read DRAM-equivalent bandwidth is about 214–229 GB/s, consistent with the 256 GB/s bandwidth of this gfx1151 system.

Testing

CMAKE_BUILD_PARALLEL_LEVEL=16 python setup.py \
  build_ext --inplace --no-cuda --hip --hip-archs=gfx1151

pytest tests/test_qdq.py::TestQuantizeINT8 -q
pytest tests/test_hip_wmma.py -q
pytest tests/test_int8.py -q

INT8 quantization: 4 passed
HIP WMMA: 349 passed
INT8: 62 passed, 39 skipped
Vectorized and fallback outputs match bit-for-bit for FP32, FP16, and BF16,
including aligned and offset-contiguous inputs.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/127 **Author:** [@tangzzycc](https://github.com/tangzzycc) **Created:** 8/21/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `exp/hip-rowwise-int8-vector-load-gfx1151` --- ### 📝 Commits (1) - [`6b38b04`](https://github.com/Comfy-Org/comfy-kitchen/commit/6b38b04d8c3e6bb9c6b6a7462a7afd5eef5fbc28) Optimize HIP rowwise INT8 quantization with vectorized loads ### 📊 Changes **2 files changed** (+85 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/hip/ops/quantize_int8.hip` (+70 -2) 📝 `tests/test_hip_wmma.py` (+15 -0) </details> ### 📄 Description ## Summary Implements Change 2 proposed in #102: - Load 16 bytes per thread with `uint4` for FP32, FP16, and BF16 inputs. - Use the vectorized kernel only when row alignment and K permit it; otherwise keep the existing scalar fallback. - Preserve the existing rounding, clamping, and scale behavior. The vectorized path uses the portable HIP `uint4` type and has no architecture-specific guard, so it also applies to other supported HIP architectures. Performance was measured on gfx1151. ## Performance Measured on AMD Radeon 8060S (gfx1151), ROCm 7.2, with aligned BF16 inputs: | Shape (M x K) | Before | After | Speedup | |---|---:|---:|---:| | 4096 x 1024 | 0.0564 ms | 0.0269 ms | 2.09x | | 8192 x 2048 | 0.4417 ms | 0.2196 ms | 2.01x | | 4096 x 4096 | 0.4107 ms | 0.2268 ms | 1.81x | | 18432 x 2048 | 0.9579 ms | 0.4999 ms | 1.92x | | 512 x 14336 | 0.0996 ms | 0.0417 ms | 2.39x | | 4096 x 14336 | 1.3498 ms | 0.8246 ms | 1.64x | Geometric-mean speedup: **1.96x**. For large BF16 shapes, one-read DRAM-equivalent bandwidth is about 214–229 GB/s, consistent with the 256 GB/s bandwidth of this gfx1151 system. ## Testing ```bash CMAKE_BUILD_PARALLEL_LEVEL=16 python setup.py \ build_ext --inplace --no-cuda --hip --hip-archs=gfx1151 pytest tests/test_qdq.py::TestQuantizeINT8 -q pytest tests/test_hip_wmma.py -q pytest tests/test_int8.py -q ``` - INT8 quantization: 4 passed - HIP WMMA: 349 passed - INT8: 62 passed, 39 skipped - Vectorized and fallback outputs match bit-for-bit for FP32, FP16, and BF16, including aligned and offset-contiguous inputs. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:02 +00:00