[PR #65] [MERGED] Basic convrot int4 support. #116

New issue

Closed

opened 2026-10-26 22:20:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:32 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/65
Author: @comfyanonymous
Created: 7/8/2026
Status: ✅ Merged
Merged: 7/9/2026
Merged by: @comfyanonymous

Base: main ← Head: dev/int4

📝 Commits (9)

d2a742f Basic convrot int4 support.
cc511a2 Overflow fix on int4.
5d5e849 Merge remote-tracking branch 'origin/main' into dev/int4
4e2518f Missing tests.
769f324 bring up int4 to same speed as int8 on blackwell
ac3d717 Support forcing the convrot int4 layers to run the matrix mult in int8.
93d19e1 Fix issues with large activations.
8cd18b9 Merge remote-tracking branch 'origin/main' into dev/int4
314f4bc Update function used to calculate shared mem usage.

📊 Changes

12 files changed (+5280 additions, -19 deletions)

View changed files

📝 comfy_kitchen/__init__.py (+8 -0)
📝 comfy_kitchen/backends/cuda/CMakeLists.txt (+1 -0)
📝 comfy_kitchen/backends/cuda/__init__.py (+652 -14)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+606 -0)
➕ comfy_kitchen/backends/cuda/ops/convrot_w4a4.cu (+3038 -0)
📝 comfy_kitchen/backends/cuda/ops/cutlass_gemm_int8.cu (+135 -3)
📝 comfy_kitchen/backends/cuda/ops/int8_linear.cu (+2 -2)
📝 comfy_kitchen/backends/eager/__init__.py (+47 -0)
➕ comfy_kitchen/backends/eager/convrot_w4a4.py (+199 -0)
📝 comfy_kitchen/tensor/__init__.py (+11 -0)
➕ comfy_kitchen/tensor/convrot_w4a4.py (+261 -0)
➕ tests/test_convrot_w4a4.py (+320 -0)

📄 Description

Should work on 30 and 40 series. 50 series doesn't have int4 ops so it will use int8 ops. 20 series needs special code for int4 and should fallback to int8 at the moment but is untested.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/65 **Author:** [@comfyanonymous](https://github.com/comfyanonymous) **Created:** 7/8/2026 **Status:** ✅ Merged **Merged:** 7/9/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `dev/int4` --- ### 📝 Commits (9) - [`d2a742f`](https://github.com/Comfy-Org/comfy-kitchen/commit/d2a742f711a7390fd368e9ea39e37104c43e2ac6) Basic convrot int4 support. - [`cc511a2`](https://github.com/Comfy-Org/comfy-kitchen/commit/cc511a2393132ceaf9cee9e7bc470c9bdbda59de) Overflow fix on int4. - [`5d5e849`](https://github.com/Comfy-Org/comfy-kitchen/commit/5d5e8491ec5de1156594abdae5ef7c4a0b09ca12) Merge remote-tracking branch 'origin/main' into dev/int4 - [`4e2518f`](https://github.com/Comfy-Org/comfy-kitchen/commit/4e2518f73919a594882b4e91a8d13913a7a30839) Missing tests. - [`769f324`](https://github.com/Comfy-Org/comfy-kitchen/commit/769f3249a9b4e8a13121ba6b2276d3c4323a5abc) bring up int4 to same speed as int8 on blackwell - [`ac3d717`](https://github.com/Comfy-Org/comfy-kitchen/commit/ac3d717c884646834b388f73d748387751dac999) Support forcing the convrot int4 layers to run the matrix mult in int8. - [`93d19e1`](https://github.com/Comfy-Org/comfy-kitchen/commit/93d19e17155c44b4940b8604b0ba24a9310f10e8) Fix issues with large activations. - [`8cd18b9`](https://github.com/Comfy-Org/comfy-kitchen/commit/8cd18b98ecde5ff2285f7684650b878ff8e4143d) Merge remote-tracking branch 'origin/main' into dev/int4 - [`314f4bc`](https://github.com/Comfy-Org/comfy-kitchen/commit/314f4bc494554e5e993df1cd361ab7a96d233798) Update function used to calculate shared mem usage. ### 📊 Changes **12 files changed** (+5280 additions, -19 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/__init__.py` (+8 -0) 📝 `comfy_kitchen/backends/cuda/CMakeLists.txt` (+1 -0) 📝 `comfy_kitchen/backends/cuda/__init__.py` (+652 -14) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+606 -0) ➕ `comfy_kitchen/backends/cuda/ops/convrot_w4a4.cu` (+3038 -0) 📝 `comfy_kitchen/backends/cuda/ops/cutlass_gemm_int8.cu` (+135 -3) 📝 `comfy_kitchen/backends/cuda/ops/int8_linear.cu` (+2 -2) 📝 `comfy_kitchen/backends/eager/__init__.py` (+47 -0) ➕ `comfy_kitchen/backends/eager/convrot_w4a4.py` (+199 -0) 📝 `comfy_kitchen/tensor/__init__.py` (+11 -0) ➕ `comfy_kitchen/tensor/convrot_w4a4.py` (+261 -0) ➕ `tests/test_convrot_w4a4.py` (+320 -0) </details> ### 📄 Description Should work on 30 and 40 series. 50 series doesn't have int4 ops so it will use int8 ops. 20 series needs special code for int4 and should fallback to int8 at the moment but is untested. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:32 +00:00