[PR #37] [MERGED] Add support for int8 quantization backend #94

New issue

Closed

opened 2026-10-26 22:20:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:19 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/37
Author: @silveroxides
Created: 5/7/2026
Status: ✅ Merged
Merged: 6/25/2026
Merged by: @comfyanonymous

Base: main ← Head: feature/int8-tensorwise

📝 Commits (10+)

954a4aa Add support for int8 tensorwise quantization with backend and layouts
60a3b2f Add functioning cuBLASLt backend for int8 that passes tests.
750cbd6 Add per-channel weight scaling support to TensorWiseINT8Layout via new per_channel flag. Adds dedicated triton per-row dequant kernel and fixes scale broadcasting in eager and CUDA backends.
1499ea4 Merge branch 'merge/upstream-int8-tensorwise' into feature/int8-tensorwise
5952e29 Merge pull request #5 from silveroxides/feature/int8-tensorwise
db14686 Merge pull request #6 from silveroxides/merge/upstream-int8-tensorwise
05166ef Merge branch 'merge/upstream-int8-tensorwise' into feature/int8-tensorwise
11a4aa4 Merge pull request #7 from silveroxides/feature/int8-tensorwise
519a902 Merge pull request #8 from silveroxides/merge/upstream-int8-tensorwise
adbd529 Add tests to cover int8 implementation

📊 Changes

21 files changed (+2739 additions, -5 deletions)

View changed files

📝 comfy_kitchen/__init__.py (+119 -0)
📝 comfy_kitchen/backends/cuda/CMakeLists.txt (+1 -0)
📝 comfy_kitchen/backends/cuda/__init__.py (+120 -0)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+59 -0)
➕ comfy_kitchen/backends/cuda/ops/cublas_gemm_int8.cu (+208 -0)
📝 comfy_kitchen/backends/eager/__init__.py (+48 -0)
📝 comfy_kitchen/backends/eager/quantization.py (+253 -0)
📝 comfy_kitchen/backends/triton/__init__.py (+32 -0)
📝 comfy_kitchen/backends/triton/quantization.py (+330 -0)
📝 comfy_kitchen/constraints.py (+2 -2)
📝 comfy_kitchen/tensor/__init__.py (+3 -0)
➕ comfy_kitchen/tensor/int8.py (+359 -0)
➕ samples/int8_model_patcher.py (+305 -0)
➕ test_cutlass.cu (+36 -0)
📝 tests/test_backends.py (+17 -0)
📝 tests/test_constraints.py (+43 -0)
📝 tests/test_dlpack.py (+122 -0)
➕ tests/test_int8.py (+408 -0)
📝 tests/test_integration.py (+9 -3)
📝 tests/test_qdq.py (+185 -0)

...and 1 more files

📄 Description

This PR adds support for int8 quantized models with full backend support and with optimized matmul kernels using triton.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/37 **Author:** [@silveroxides](https://github.com/silveroxides) **Created:** 5/7/2026 **Status:** ✅ Merged **Merged:** 6/25/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `feature/int8-tensorwise` --- ### 📝 Commits (10+) - [`954a4aa`](https://github.com/Comfy-Org/comfy-kitchen/commit/954a4aa3bc6f3280f1af78241537104ff2cdf653) Add support for int8 tensorwise quantization with backend and layouts - [`60a3b2f`](https://github.com/Comfy-Org/comfy-kitchen/commit/60a3b2f2d59f570be0e32f5334afc6d2cd8d4246) Add functioning cuBLASLt backend for int8 that passes tests. - [`750cbd6`](https://github.com/Comfy-Org/comfy-kitchen/commit/750cbd671ba5c53bb37ce69c9a58ebb775ac5a55) Add per-channel weight scaling support to TensorWiseINT8Layout via new per_channel flag. Adds dedicated triton per-row dequant kernel and fixes scale broadcasting in eager and CUDA backends. - [`1499ea4`](https://github.com/Comfy-Org/comfy-kitchen/commit/1499ea41e3f1ada681169df4b1935b475b478491) Merge branch 'merge/upstream-int8-tensorwise' into feature/int8-tensorwise - [`5952e29`](https://github.com/Comfy-Org/comfy-kitchen/commit/5952e2934dab769e2ae41cd7cfc93dd0ee04821d) Merge pull request #5 from silveroxides/feature/int8-tensorwise - [`db14686`](https://github.com/Comfy-Org/comfy-kitchen/commit/db14686034ed0c878e2278dbaac8ae531f8f7363) Merge pull request #6 from silveroxides/merge/upstream-int8-tensorwise - [`05166ef`](https://github.com/Comfy-Org/comfy-kitchen/commit/05166ef9496900abf7f9affb868c430c69079b8d) Merge branch 'merge/upstream-int8-tensorwise' into feature/int8-tensorwise - [`11a4aa4`](https://github.com/Comfy-Org/comfy-kitchen/commit/11a4aa48d0e1af1aa57efe79e6d55a2453f4cb1a) Merge pull request #7 from silveroxides/feature/int8-tensorwise - [`519a902`](https://github.com/Comfy-Org/comfy-kitchen/commit/519a9023a165950ec6007e5bb8fb8ea9f5ede0ad) Merge pull request #8 from silveroxides/merge/upstream-int8-tensorwise - [`adbd529`](https://github.com/Comfy-Org/comfy-kitchen/commit/adbd52997e03380d9da2c8dd8d04fffceaf5850b) Add tests to cover int8 implementation ### 📊 Changes **21 files changed** (+2739 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/__init__.py` (+119 -0) 📝 `comfy_kitchen/backends/cuda/CMakeLists.txt` (+1 -0) 📝 `comfy_kitchen/backends/cuda/__init__.py` (+120 -0) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+59 -0) ➕ `comfy_kitchen/backends/cuda/ops/cublas_gemm_int8.cu` (+208 -0) 📝 `comfy_kitchen/backends/eager/__init__.py` (+48 -0) 📝 `comfy_kitchen/backends/eager/quantization.py` (+253 -0) 📝 `comfy_kitchen/backends/triton/__init__.py` (+32 -0) 📝 `comfy_kitchen/backends/triton/quantization.py` (+330 -0) 📝 `comfy_kitchen/constraints.py` (+2 -2) 📝 `comfy_kitchen/tensor/__init__.py` (+3 -0) ➕ `comfy_kitchen/tensor/int8.py` (+359 -0) ➕ `samples/int8_model_patcher.py` (+305 -0) ➕ `test_cutlass.cu` (+36 -0) 📝 `tests/test_backends.py` (+17 -0) 📝 `tests/test_constraints.py` (+43 -0) 📝 `tests/test_dlpack.py` (+122 -0) ➕ `tests/test_int8.py` (+408 -0) 📝 `tests/test_integration.py` (+9 -3) 📝 `tests/test_qdq.py` (+185 -0) _...and 1 more files_ </details> ### 📄 Description This PR adds support for int8 quantized models with full backend support and with optimized matmul kernels using triton. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:19 +00:00