[PR #234] [MERGED] Optimize int8 attention for ampere #242

New issue

Closed

opened 2026-10-26 22:21:56 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:56 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/234
Author: @kijai
Created: 10/8/2026
Status: ✅ Merged
Merged: 10/16/2026
Merged by: @comfyanonymous

Base: main ← Head: ampere-attention

📝 Commits (2)

1f42d57 Optimize attention for ampere
a4a91c7 conversions off the XU pipe, 64-row tile up to 40k keys

📊 Changes

9 files changed (+194 additions, -69 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/__init__.py (+34 -16)
📝 comfy_kitchen/backends/cuda/sage_attention/attn_utils.cuh (+21 -17)
📝 comfy_kitchen/backends/cuda/sage_attention/math.cuh (+14 -0)
📝 comfy_kitchen/backends/cuda/sage_attention/mma.cuh (+32 -0)
📝 comfy_kitchen/backends/cuda/sage_attention/qk_int_sv_i8_cuda.cuh (+10 -8)
📝 comfy_kitchen/backends/cuda/sage_attention/sage_attn_launcher.cu (+37 -14)
📝 comfy_kitchen/backends/cuda/sage_attention/sol_attn_token.cu (+3 -2)
📝 comfy_kitchen/backends/cuda/sage_attention/sol_layout.cuh (+14 -12)
📝 tests/test_sol_attn.py (+29 -0)

📄 Description

Benchmarked around ~9% on dense attention and ~5% on sparse attention, end to end with Minimax-H3, on both a 3080 and 3090.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/234 **Author:** [@kijai](https://github.com/kijai) **Created:** 10/8/2026 **Status:** ✅ Merged **Merged:** 10/16/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `ampere-attention` --- ### 📝 Commits (2) - [`1f42d57`](https://github.com/Comfy-Org/comfy-kitchen/commit/1f42d573327a124b2838517b6627f544d7dedad6) Optimize attention for ampere - [`a4a91c7`](https://github.com/Comfy-Org/comfy-kitchen/commit/a4a91c7aaa0949b2857e2356208e04d6f57e0f29) conversions off the XU pipe, 64-row tile up to 40k keys ### 📊 Changes **9 files changed** (+194 additions, -69 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/__init__.py` (+34 -16) 📝 `comfy_kitchen/backends/cuda/sage_attention/attn_utils.cuh` (+21 -17) 📝 `comfy_kitchen/backends/cuda/sage_attention/math.cuh` (+14 -0) 📝 `comfy_kitchen/backends/cuda/sage_attention/mma.cuh` (+32 -0) 📝 `comfy_kitchen/backends/cuda/sage_attention/qk_int_sv_i8_cuda.cuh` (+10 -8) 📝 `comfy_kitchen/backends/cuda/sage_attention/sage_attn_launcher.cu` (+37 -14) 📝 `comfy_kitchen/backends/cuda/sage_attention/sol_attn_token.cu` (+3 -2) 📝 `comfy_kitchen/backends/cuda/sage_attention/sol_layout.cuh` (+14 -12) 📝 `tests/test_sol_attn.py` (+29 -0) </details> ### 📄 Description Benchmarked around ~9% on dense attention and ~5% on sparse attention, end to end with Minimax-H3, on both a 3080 and 3090. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:56 +00:00