[PR #103] [MERGED] Implement optimized int8 attention (CORE-74) #147

New issue

Closed

opened 2026-10-26 22:20:51 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:51 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/103
Author: @comfyanonymous
Created: 8/10/2026
Status: ✅ Merged
Merged: 8/11/2026
Merged by: @comfyanonymous

Base: main ← Head: dev/comfy_int8

📝 Commits (3)

9243fb3 Implement optimized int8 attention.
1a25155 Fix Nan on some models and other improvements.
215cc5a cleanups.

📊 Changes

18 files changed (+6762 additions, -2 deletions)

View changed files

📝 NOTICE (+11 -1)
📝 README.md (+1 -0)
📝 comfy_kitchen/__init__.py (+12 -0)
📝 comfy_kitchen/backends/cuda/CMakeLists.txt (+4 -0)
📝 comfy_kitchen/backends/cuda/dlpack_bindings.cpp (+663 -0)
📝 comfy_kitchen/backends/cuda/float_utils.cuh (+35 -1)
➕ comfy_kitchen/backends/cuda/sage_attention/attn_utils.cuh (+1247 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/cp_async.cuh (+151 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/math.cuh (+171 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/mma.cuh (+808 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/numeric_conversion.cuh (+159 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/permuted_smem.cuh (+214 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/qk_int_sv_i8_cuda.cuh (+834 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/quant_qk_int8.cu (+1014 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/quant_v_int8.cu (+211 -0)
➕ comfy_kitchen/backends/cuda/sage_attention/sage_attn_launcher.cu (+182 -0)
➕ comfy_kitchen/sage_attention.py (+617 -0)
➕ tests/test_int8_attention.py (+428 -0)

📄 Description

Original code is based on sage attention but this is modified to try to get slightly higher performance while keeping similar quality on consumer GPUs.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/103 **Author:** [@comfyanonymous](https://github.com/comfyanonymous) **Created:** 8/10/2026 **Status:** ✅ Merged **Merged:** 8/11/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `dev/comfy_int8` --- ### 📝 Commits (3) - [`9243fb3`](https://github.com/Comfy-Org/comfy-kitchen/commit/9243fb3621b9c02bcb81a939aa4e0ee6c5353ae6) Implement optimized int8 attention. - [`1a25155`](https://github.com/Comfy-Org/comfy-kitchen/commit/1a251552445c12f675ce314fa0b56f59543fa139) Fix Nan on some models and other improvements. - [`215cc5a`](https://github.com/Comfy-Org/comfy-kitchen/commit/215cc5a8e52f163bf8d4631030f452767c737ab5) cleanups. ### 📊 Changes **18 files changed** (+6762 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `NOTICE` (+11 -1) 📝 `README.md` (+1 -0) 📝 `comfy_kitchen/__init__.py` (+12 -0) 📝 `comfy_kitchen/backends/cuda/CMakeLists.txt` (+4 -0) 📝 `comfy_kitchen/backends/cuda/dlpack_bindings.cpp` (+663 -0) 📝 `comfy_kitchen/backends/cuda/float_utils.cuh` (+35 -1) ➕ `comfy_kitchen/backends/cuda/sage_attention/attn_utils.cuh` (+1247 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/cp_async.cuh` (+151 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/math.cuh` (+171 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/mma.cuh` (+808 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/numeric_conversion.cuh` (+159 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/permuted_smem.cuh` (+214 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/qk_int_sv_i8_cuda.cuh` (+834 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/quant_qk_int8.cu` (+1014 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/quant_v_int8.cu` (+211 -0) ➕ `comfy_kitchen/backends/cuda/sage_attention/sage_attn_launcher.cu` (+182 -0) ➕ `comfy_kitchen/sage_attention.py` (+617 -0) ➕ `tests/test_int8_attention.py` (+428 -0) </details> ### 📄 Description Original code is based on sage attention but this is modified to try to get slightly higher performance while keeping similar quality on consumer GPUs. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:51 +00:00