[PR #3074] [CLOSED] Add TINT4 — INT4 Weight-Only Quantization via torchao #2992

New issue

Closed

opened 2026-10-28 21:14:21 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-28 21:14:21 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/ComfyUI-Manager/pull/3074
Author: @JWLHS
Created: 7/10/2026
Status: ❌ Closed

Base: main ← Head: add-tint4

📝 Commits (4)

b066a92 Add TINT4 — torchao INT4 weight-only quantization plugin
241d909 fix TINT4 repository URL
f4bf4c3 add TINT4 (INT4 quantization via torchao)
f44cb8c fix missing closing bracket for custom_nodes array

📊 Changes

1 file changed (+1940 additions, -82 deletions)

View changed files

📝 custom-node-list.json (+1940 -82)

📄 Description

TINT4 — torchao INT4 Weight-Only Quantization for ComfyUI

Repository: https://github.com/JWLHS/ComfyUI-TINT4

What it does

INT4 weight-only quantization (W4A16) for diffusion models via torchao. Activations remain FP16 — zero quantization quality loss.

Nodes

Node	Function
TINT4 Model Quantizer	Quantize FP16/BF16 models to INT4 safetensors
TINT4 Model Loader	Load quantized models
TINT4 LoRA Loader	Single LoRA injection (standard + LoKr, QKV fused/independent)
TINT4 LoRA Stack	Multi-LoRA stack (≤5)

Key features

Full LoRA support — forward-time injection, no kernel modification
3-layer acceleration — IS_CHANGED fingerprint skip (0s repeat), SHA256 JSON cache (<1s hit), GPU pre-hook bake-in
Multi-backend — Intel Arc XPU, NVIDIA CUDA, AMD ROCm
12+ model exclusion lists — Krea2 Turbo, Flux2 Klein 9B, Z-Image, Boogu, WAN, LTX2, Qwen, Ernie, HiDream, Chroma, Ideogram4

Performance (Arc A770, Krea2 Turbo)

LoRA	Layers	First load	Cache hit	IS_CHANGED
BreastSlider	104	0.05s	0.04s	0s
YFG GPT	256	0.63s	0.49s	0s
Onyx_V1	264	4.78s	<1s	0s

Requirements

# NVIDIA CUDA / AMD ROCm / CPU
pip install torchao>=0.17.0
# Intel XPU (Arc GPU)
pip install torchao --index-url https://download.pytorch.org/whl/xpu
Intel XPU users: after installing torchao, run fix_torchao_xpu.bat (included in the plugin) to patch missing submodules in the XPU fork.

---

<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/ComfyUI-Manager/pull/3074 **Author:** [@JWLHS](https://github.com/JWLHS) **Created:** 7/10/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `add-tint4` --- ### 📝 Commits (4) - [`b066a92`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/b066a920a131a6d9f1ee6deac12164601d8c115b) Add TINT4 — torchao INT4 weight-only quantization plugin - [`241d909`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/241d9094ee0eae2857325d2dc2e1a5ecbe969175) fix TINT4 repository URL - [`f4bf4c3`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/f4bf4c39b6f57879ec5f4ccbb31f8810b949102a) add TINT4 (INT4 quantization via torchao) - [`f44cb8c`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/f44cb8cfc6aa76166483522539f51ab117bde00a) fix missing closing bracket for custom_nodes array ### 📊 Changes **1 file changed** (+1940 additions, -82 deletions) <details> <summary>View changed files</summary> 📝 `custom-node-list.json` (+1940 -82) </details> ### 📄 Description ## TINT4 — torchao INT4 Weight-Only Quantization for ComfyUI **Repository**: https://github.com/JWLHS/ComfyUI-TINT4 ### What it does INT4 weight-only quantization (W4A16) for diffusion models via torchao. Activations remain FP16 — zero quantization quality loss. ### Nodes | Node | Function | |------|----------| | TINT4 Model Quantizer | Quantize FP16/BF16 models to INT4 safetensors | | TINT4 Model Loader | Load quantized models | | TINT4 LoRA Loader | Single LoRA injection (standard + LoKr, QKV fused/independent) | | TINT4 LoRA Stack | Multi-LoRA stack (≤5) | ### Key features - **Full LoRA support** — forward-time injection, no kernel modification - **3-layer acceleration** — IS_CHANGED fingerprint skip (0s repeat), SHA256 JSON cache (<1s hit), GPU pre-hook bake-in - **Multi-backend** — Intel Arc XPU, NVIDIA CUDA, AMD ROCm - **12+ model exclusion lists** — Krea2 Turbo, Flux2 Klein 9B, Z-Image, Boogu, WAN, LTX2, Qwen, Ernie, HiDream, Chroma, Ideogram4 ### Performance (Arc A770, Krea2 Turbo) | LoRA | Layers | First load | Cache hit | IS_CHANGED | |------|--------|-----------|-----------|:---:| | BreastSlider | 104 | 0.05s | 0.04s | 0s | | YFG GPT | 256 | 0.63s | 0.49s | 0s | | Onyx_V1 | 264 | 4.78s | <1s | 0s | ### Requirements ```bash # NVIDIA CUDA / AMD ROCm / CPU pip install torchao>=0.17.0 # Intel XPU (Arc GPU) pip install torchao --index-url https://download.pytorch.org/whl/xpu Intel XPU users: after installing torchao, run fix_torchao_xpu.bat (included in the plugin) to patch missing submodules in the XPU fork. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-28 21:14:21 +00:00