[PR #96] [MERGED] a4w8: use frozen codebook LUT + row-chunked quantize, improve LoRA application #141

New issue

Closed

opened 2026-10-26 22:20:47 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:47 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/96
Author: @kijai
Created: 8/6/2026
Status: ✅ Merged
Merged: 8/6/2026
Merged by: @comfyanonymous

Base: main ← Head: w4a8-int8-codebook

📝 Commits (2)

efff38b Use frozen codebook LUT + row-chunked quantize, improve LoRA application
27390bc w4a8: honor chunk budget for wide weights, pick codebook from whole tensor

📊 Changes

3 files changed (+161 additions, -10 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/__init__.py (+7 -4)
📝 comfy_kitchen/backends/eager/w4a8_int8.py (+144 -5)
📝 comfy_kitchen/tensor/w4a8_int8.py (+10 -1)

📄 Description

w4a8: frozen codebook LUT + row-chunked quantize

Improves quantization time and LoRA application cost. I also observe

Use a fixed Lloyd-Max LUT for ConvRot-Gaussianized weights
Rotate + pack in row chunks to cap peak memory (fixes LoRA requantize OOM), bit-identical to the whole-tensor path.
Reuse the stored codebook on requantize

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/96 **Author:** [@kijai](https://github.com/kijai) **Created:** 8/6/2026 **Status:** ✅ Merged **Merged:** 8/6/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `w4a8-int8-codebook` --- ### 📝 Commits (2) - [`efff38b`](https://github.com/Comfy-Org/comfy-kitchen/commit/efff38b5d10dc009463063c8342353c299d35878) Use frozen codebook LUT + row-chunked quantize, improve LoRA application - [`27390bc`](https://github.com/Comfy-Org/comfy-kitchen/commit/27390bc6551b2fa2b601bef00d0f5c049d1a5aa3) w4a8: honor chunk budget for wide weights, pick codebook from whole tensor ### 📊 Changes **3 files changed** (+161 additions, -10 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/__init__.py` (+7 -4) 📝 `comfy_kitchen/backends/eager/w4a8_int8.py` (+144 -5) 📝 `comfy_kitchen/tensor/w4a8_int8.py` (+10 -1) </details> ### 📄 Description w4a8: frozen codebook LUT + row-chunked quantize Improves quantization time and LoRA application cost. I also observe - Use a fixed Lloyd-Max LUT for ConvRot-Gaussianized weights - Rotate + pack in row chunks to cap peak memory (fixes LoRA requantize OOM), bit-identical to the whole-tensor path. - Reuse the stored codebook on requantize --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:47 +00:00