[PR #216] [MERGED] Fused ConvRot requant kernel for w4a8/w6a8 #229

New issue

Closed

opened 2026-10-26 22:21:47 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:47 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/216
Author: @kijai
Created: 9/30/2026
Status: ✅ Merged
Merged: 9/30/2026
Merged by: @comfyanonymous

Base: main ← Head: w6a8-fused-requant

📝 Commits (4)

da7bbf0 Fused CUDA requant for W6A8
b79adb6 Fused ConvRot requant kernel for w4a8/w6a8
8faa81a Fix test
6ab248d Rotate in registers instead of shared memory in the fused requant kernel

📊 Changes

5 files changed (+510 additions, -201 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/__init__.py (+41 -63)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+36 -32)
📝 comfy_kitchen/backends/cuda/ops/w4a8_gemm.cu (+332 -106)
➕ tests/test_w4a8_requant.py (+48 -0)
📝 tests/test_w6a8.py (+53 -0)

📄 Description

One fused CUDA kernel for w4a8/w6a8 requantization: ConvRot rotation in shared memory, then codebook (4-bit) or uniform (6-bit) quantization, replacing the two-pass rotate → copy → quantize path.

No rotated copy in memory, one launch per weight; H3 fc1 requant 1.6 → 0.8 ms (4-bit) and 29 → 0.6 ms (6-bit)

Results match the eager quantizer up to fp32 rounding order.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/216 **Author:** [@kijai](https://github.com/kijai) **Created:** 9/30/2026 **Status:** ✅ Merged **Merged:** 9/30/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `w6a8-fused-requant` --- ### 📝 Commits (4) - [`da7bbf0`](https://github.com/Comfy-Org/comfy-kitchen/commit/da7bbf0d412d03bd0fdc30c438ee8777c3f9621b) Fused CUDA requant for W6A8 - [`b79adb6`](https://github.com/Comfy-Org/comfy-kitchen/commit/b79adb655969c7a263a3384a9dc595303e93b39b) Fused ConvRot requant kernel for w4a8/w6a8 - [`8faa81a`](https://github.com/Comfy-Org/comfy-kitchen/commit/8faa81a4421aa248d664abd8d3876155b525fcb2) Fix test - [`6ab248d`](https://github.com/Comfy-Org/comfy-kitchen/commit/6ab248da12135c7c84697c659bdb2f68d7c2e448) Rotate in registers instead of shared memory in the fused requant kernel ### 📊 Changes **5 files changed** (+510 additions, -201 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/__init__.py` (+41 -63) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+36 -32) 📝 `comfy_kitchen/backends/cuda/ops/w4a8_gemm.cu` (+332 -106) ➕ `tests/test_w4a8_requant.py` (+48 -0) 📝 `tests/test_w6a8.py` (+53 -0) </details> ### 📄 Description One fused CUDA kernel for w4a8/w6a8 requantization: ConvRot rotation in shared memory, then codebook (4-bit) or uniform (6-bit) quantization, replacing the two-pass rotate → copy → quantize path. No rotated copy in memory, one launch per weight; H3 fc1 requant 1.6 → 0.8 ms (4-bit) and 29 → 0.6 ms (6-bit) Results match the eager quantizer up to fp32 rounding order. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:47 +00:00