[PR #35] [CLOSED] Feat/svdquant w4a4 kitchen native #92

New issue

Closed

opened 2026-10-26 22:20:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:18 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/35
Author: @HK416-TYPED
Created: 4/21/2026
Status: ❌ Closed

Base: main ← Head: feat/svdquant-w4a4-kitchen-native

📝 Commits (2)

82fdd71 Add SVDQuant W4A4 + AWQ W4A16 eager backends and public API
71419ed Add kitchen-native CUDA int4 MMA kernels for SVDQuant W4A4

📊 Changes

14 files changed (+2376 additions, -0 deletions)

View changed files

📝 comfy_kitchen/__init__.py (+105 -0)
📝 comfy_kitchen/backends/cuda/CMakeLists.txt (+6 -0)
📝 comfy_kitchen/backends/cuda/__init__.py (+169 -0)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+124 -0)
➕ comfy_kitchen/backends/cuda/ops/quantize_svdquant_w4a4.cu (+186 -0)
➕ comfy_kitchen/backends/cuda/ops/scaled_mm_svdquant_w4a4.cu (+348 -0)
➕ comfy_kitchen/backends/cuda/ops/svdquant_utils.cuh (+177 -0)
➕ comfy_kitchen/backends/cuda/ops/svdquant_w4a4_PLAN.md (+196 -0)
📝 comfy_kitchen/backends/eager/__init__.py (+47 -0)
➕ comfy_kitchen/backends/eager/awq.py (+108 -0)
➕ comfy_kitchen/backends/eager/svdquant.py (+327 -0)
📝 comfy_kitchen/tensor/__init__.py (+3 -0)
➕ comfy_kitchen/tensor/svdquant_w4a4.py (+273 -0)
➕ tests/test_svdquant_w4a4.py (+307 -0)

📄 Description

Add SVDQuant W4A4 int4 quantization support for Qwen-Image-Edit and similar SVD-LoRA checkpoints. Built as a kitchen-native row-major backend .

Targets sm_80+ (Ampere).

Changes

comfy_kitchen.quantize_svdquant_w4a4 / scaled_mm_svdquant_w4a4 as torch.library.custom_op entry points with eager pure-PyTorch reference
TensorCoreSVDQuantW4A4Layout dispatching aten.t / aten.mm / aten.addmm / aten.linear over int4 qdata + proj_down / proj_up / smooth_factor + per-group weight_scale
CUDA kernels: m16n8k64 int4 MMA GEMM + fused activation quantize over kitchen-native row-major tensors; fp32 accumulator (fp16 overflows at production Qwen scale ≈28 — see kernel header)
act_unsigned end-to-end for post-GELU fc2 layers: u4.s4 MMA variant + +0.171875 shift applied at the layer (kernel API stays shift-free)
Signed quantizer clamp to [-7, 7] (skip -8) to match nunchaku's absmax/7 dequant-symmetric contract
LoRA-up + bias applied in Python via bf16 addmm_ (faster than kernel fusion for R=96; see scaled_mm_svdquant_w4a4 docstring for the precision tradeoff and the cuBLASLt upgrade path if needed)

Tests

tests/test_svdquant_w4a4.py — 16 tests: quantizer clamp contract, signed/unsigned MMA dispatch, lora_x separation, cross-shape smoke.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/35 **Author:** [@HK416-TYPED](https://github.com/HK416-TYPED) **Created:** 4/21/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/svdquant-w4a4-kitchen-native` --- ### 📝 Commits (2) - [`82fdd71`](https://github.com/Comfy-Org/comfy-kitchen/commit/82fdd71fa872f363c043fbe0b4b648ecd0e911c9) Add SVDQuant W4A4 + AWQ W4A16 eager backends and public API - [`71419ed`](https://github.com/Comfy-Org/comfy-kitchen/commit/71419ed1c033e286e3bb24e3ac71ee99cd90ce4b) Add kitchen-native CUDA int4 MMA kernels for SVDQuant W4A4 ### 📊 Changes **14 files changed** (+2376 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/__init__.py` (+105 -0) 📝 `comfy_kitchen/backends/cuda/CMakeLists.txt` (+6 -0) 📝 `comfy_kitchen/backends/cuda/__init__.py` (+169 -0) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+124 -0) ➕ `comfy_kitchen/backends/cuda/ops/quantize_svdquant_w4a4.cu` (+186 -0) ➕ `comfy_kitchen/backends/cuda/ops/scaled_mm_svdquant_w4a4.cu` (+348 -0) ➕ `comfy_kitchen/backends/cuda/ops/svdquant_utils.cuh` (+177 -0) ➕ `comfy_kitchen/backends/cuda/ops/svdquant_w4a4_PLAN.md` (+196 -0) 📝 `comfy_kitchen/backends/eager/__init__.py` (+47 -0) ➕ `comfy_kitchen/backends/eager/awq.py` (+108 -0) ➕ `comfy_kitchen/backends/eager/svdquant.py` (+327 -0) 📝 `comfy_kitchen/tensor/__init__.py` (+3 -0) ➕ `comfy_kitchen/tensor/svdquant_w4a4.py` (+273 -0) ➕ `tests/test_svdquant_w4a4.py` (+307 -0) </details> ### 📄 Description Add SVDQuant W4A4 int4 quantization support for Qwen-Image-Edit and similar SVD-LoRA checkpoints. Built as a kitchen-native row-major backend . Targets sm_80+ (Ampere). Changes - comfy_kitchen.quantize_svdquant_w4a4 / scaled_mm_svdquant_w4a4 as torch.library.custom_op entry points with eager pure-PyTorch reference - TensorCoreSVDQuantW4A4Layout dispatching aten.t / aten.mm / aten.addmm / aten.linear over int4 qdata + proj_down / proj_up / smooth_factor + per-group weight_scale - CUDA kernels: m16n8k64 int4 MMA GEMM + fused activation quantize over kitchen-native row-major tensors; fp32 accumulator (fp16 overflows at production Qwen scale ≈28 — see kernel header) - act_unsigned end-to-end for post-GELU fc2 layers: u4.s4 MMA variant + +0.171875 shift applied at the layer (kernel API stays shift-free) - Signed quantizer clamp to [-7, 7] (skip -8) to match nunchaku's absmax/7 dequant-symmetric contract - LoRA-up + bias applied in Python via bf16 addmm_ (faster than kernel fusion for R=96; see scaled_mm_svdquant_w4a4 docstring for the precision tradeoff and the cuBLASLt upgrade path if needed) Tests tests/test_svdquant_w4a4.py — 16 tests: quantizer clamp contract, signed/unsigned MMA dispatch, lora_x separation, cross-shape smoke. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:18 +00:00