[PR #99] [MERGED] w4a8: stochastic rounding for LoRA merge + fused requant kernel #144

New issue

Closed

opened 2026-10-26 22:20:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:49 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/99
Author: @kijai
Created: 8/7/2026
Status: ✅ Merged
Merged: 8/7/2026
Merged by: @comfyanonymous

Base: main ← Head: w4a8-int8-codebook

📝 Commits (2)

2116e51 w4a8: stochastic rounding for LoRA merge + fused requant kernel
51cfd0e w4a8: row-chunk fused requant + fail loud on launch error

📊 Changes

5 files changed (+386 additions, -27 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/__init__.py (+74 -0)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+51 -0)
📝 comfy_kitchen/backends/cuda/ops/w4a8_gemm.cu (+176 -0)
📝 comfy_kitchen/backends/eager/w4a8_int8.py (+78 -26)
📝 comfy_kitchen/tensor/w4a8_int8.py (+7 -1)

📄 Description

w4a8: stochastic rounding for LoRA merge + fused requant kernel

Stochastic rounding on requantize
Fused CUDA requant kernel (rotate-fed): per-group codebook assign +
ALS + per-channel scale + SR + int4 pack in one launch. Requant
166ms -> 0.6ms/layer; falls back to eager for non-default configs.
Eager path also sped up (searchsorted codebook assign, ALS 3->2)
and made memory-frugal for the lowvram per-fault path.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/99 **Author:** [@kijai](https://github.com/kijai) **Created:** 8/7/2026 **Status:** ✅ Merged **Merged:** 8/7/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `w4a8-int8-codebook` --- ### 📝 Commits (2) - [`2116e51`](https://github.com/Comfy-Org/comfy-kitchen/commit/2116e51422556880b5848330161596fdef1c017a) w4a8: stochastic rounding for LoRA merge + fused requant kernel - [`51cfd0e`](https://github.com/Comfy-Org/comfy-kitchen/commit/51cfd0e20a5660578db38c9ef35dcc2f4b0c43de) w4a8: row-chunk fused requant + fail loud on launch error ### 📊 Changes **5 files changed** (+386 additions, -27 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/__init__.py` (+74 -0) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+51 -0) 📝 `comfy_kitchen/backends/cuda/ops/w4a8_gemm.cu` (+176 -0) 📝 `comfy_kitchen/backends/eager/w4a8_int8.py` (+78 -26) 📝 `comfy_kitchen/tensor/w4a8_int8.py` (+7 -1) </details> ### 📄 Description w4a8: stochastic rounding for LoRA merge + fused requant kernel - Stochastic rounding on requantize - Fused CUDA requant kernel (rotate-fed): per-group codebook assign + ALS + per-channel scale + SR + int4 pack in one launch. Requant 166ms -> 0.6ms/layer; falls back to eager for non-default configs. - Eager path also sped up (searchsorted codebook assign, ALS 3->2) and made memory-frugal for the lowvram per-fault path. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:49 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#144

No description provided.

Rows
Columns