[PR #53] [MERGED] Add cutlass int8 kernel #108

New issue

Closed

opened 2026-10-26 22:20:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:28 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/53
Author: @kijai
Created: 6/25/2026
Status: ✅ Merged
Merged: 6/25/2026
Merged by: @comfyanonymous

Base: main ← Head: cutlass

📝 Commits (5)

bdc3a26 cutlass int8 kernel
eedf593 autotune
ad762bd satisfy ruff
da60ad1 Fix wheels not being built with cutlass.
7273ba0 Make ruff ignore third_party directory.

📊 Changes

10 files changed (+588 additions, -26 deletions)

View changed files

📝 .github/workflows/build-wheels.yml (+10 -0)
📝 .gitignore (+0 -1)
➕ .gitmodules (+3 -0)
📝 comfy_kitchen/backends/cuda/CMakeLists.txt (+15 -0)
📝 comfy_kitchen/backends/cuda/__init__.py (+97 -25)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+98 -0)
➕ comfy_kitchen/backends/cuda/ops/cutlass_gemm_int8.cu (+170 -0)
📝 comfy_kitchen/backends/cuda/ops/int8_linear.cu (+193 -0)
📝 pyproject.toml (+1 -0)
➕ third_party/cutlass (+1 -0)

📄 Description

Based on quick initial testing, this is faster than both the existing cublast and triton paths on my 5090 at least

Krea2, 1024x1024, 5090, Linux:

cublast 2.82it/s
triton 3.10it/s
cutlass 3.25it/s

Needs further testing on other hardware.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/53 **Author:** [@kijai](https://github.com/kijai) **Created:** 6/25/2026 **Status:** ✅ Merged **Merged:** 6/25/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `cutlass` --- ### 📝 Commits (5) - [`bdc3a26`](https://github.com/Comfy-Org/comfy-kitchen/commit/bdc3a2630e109f7840f3e1f0629ef99be460274d) cutlass int8 kernel - [`eedf593`](https://github.com/Comfy-Org/comfy-kitchen/commit/eedf5937f323137627ab37e0fa1039bb9f18f1a2) autotune - [`ad762bd`](https://github.com/Comfy-Org/comfy-kitchen/commit/ad762bdd11d24894ef994eac7afb07565bd86173) satisfy ruff - [`da60ad1`](https://github.com/Comfy-Org/comfy-kitchen/commit/da60ad1cb44759d424f6418ff2d561c36d72ea99) Fix wheels not being built with cutlass. - [`7273ba0`](https://github.com/Comfy-Org/comfy-kitchen/commit/7273ba0203395fef2341971a1a72729650d0c97d) Make ruff ignore third_party directory. ### 📊 Changes **10 files changed** (+588 additions, -26 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/build-wheels.yml` (+10 -0) 📝 `.gitignore` (+0 -1) ➕ `.gitmodules` (+3 -0) 📝 `comfy_kitchen/backends/cuda/CMakeLists.txt` (+15 -0) 📝 `comfy_kitchen/backends/cuda/__init__.py` (+97 -25) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+98 -0) ➕ `comfy_kitchen/backends/cuda/ops/cutlass_gemm_int8.cu` (+170 -0) 📝 `comfy_kitchen/backends/cuda/ops/int8_linear.cu` (+193 -0) 📝 `pyproject.toml` (+1 -0) ➕ `third_party/cutlass` (+1 -0) </details> ### 📄 Description Based on quick initial testing, this is faster than both the existing cublast and triton paths on my 5090 at least Krea2, 1024x1024, 5090, Linux: cublast 2.82it/s triton 3.10it/s cutlass 3.25it/s Needs further testing on other hardware. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:28 +00:00