Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

[PR #69] RDNA-aware autotune configs for the INT8 matmul kernels #120

New issue

Open

opened 2026-10-26 22:20:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/69
Author: @liminfei-amd
Created: 7/10/2026
Status: 🔄 Open

Base: main ← Head: rdna-int8-autotune

📝 Commits (6)

e9f2c72 RDNA-aware autotune configs for the INT8 matmul kernels
f004d58 Add waves_per_eu variants to the RDNA INT8 autotune pool
ce6e98d fix(triton): scope RDNA INT8 autotuning per device
a6dcb22 refactor(triton): share INT8 autotune kwargs
04ac357 fix(triton): improve RDNA INT8 autotune selection
a789909 test(triton): cover INT8 device index forwarding

📊 Changes

2 files changed (+295 additions, -21 deletions)

View changed files

📝 comfy_kitchen/backends/triton/quantization.py (+83 -21)
📝 tests/test_int8.py (+212 -0)

📄 Description

RDNA-aware autotune configs for the INT8 matmul kernels

What

This keeps the existing CUDA/CDNA behavior and selects a ten-candidate INT8 Triton pool for the RDNA generation serving each call. gfx11* gets a small-M 64x64x128, eight-warp tile; gfx12* retains its existing pool. For M <= 128, only the unstable 128x128x128 tile is removed.

device_index remains in the autotune key so heterogeneous devices tune separately, but is marked non-specializing in both kernels so the scalar value does not create duplicate code variants.

Why

The original six default configs use num_stages=3-4. Shallow num_stages=2 pools perform better on the tested consumer RDNA devices. Follow-up SDXL testing also showed that one pool should not be shared blindly across RDNA generations: the extra small-M tile helps gfx11, while replacing a gfx12 tile regresses that generation.

Validation

Rebased onto current main 5a997b17; cumulative diff is two files (+236/-21).
Focused config/device tests: 18/18 pass.
Complete tests/test_int8.py on gfx1100, gfx1151, and gfx1201: 80 passed / 33 skipped / the same 5 CUDA-extension-only failures as the before arm.
Fixed-config correctness/performance sweeps use both actual INT8 kernels and real SDXL@1024 projection shapes. Every output is finite and matches across configs.
Same-base, fresh-process weighted SDXL kernel estimate, before -> after:
- gfx1100: 67.20 -> 65.63 ms/step (-2.3%)
- gfx1151: 129.06 -> 122.98 ms/step (-4.7%)
- gfx1201: 39.24 -> 39.08 ms/step (-0.4%, effectively unchanged)
Per-device pruning returns ten RDNA candidates on every tested GPU; the 17-config decorator union is pruned before benchmarking.
Direct JIT-cache probe: changing device_index created one extra compiled entry before this update and zero afterward; the output remained finite.
Ruff, Python syntax, and whitespace checks pass.

Reporter follow-up

The exact 1.70 vs 1.79 s/it E2E run could not be reproduced without the reporter's private config/model setup. On gfx1151, their six-config pool was not slower than the one-config pool in steady-state kernel time; its first-use autotune cost was much larger. However, their feedback exposed a real selection gap in this PR's prior pool, and the generation-specific update above improves the matching SDXL shapes without regressing gfx1201.

Fixes #68.

AI usage disclosure: this update was prepared with AI assistance. The submitting human remains responsible for reviewing the changes and validation before publication.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/69 **Author:** [@liminfei-amd](https://github.com/liminfei-amd) **Created:** 7/10/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `rdna-int8-autotune` --- ### 📝 Commits (6) - [`e9f2c72`](https://github.com/Comfy-Org/comfy-kitchen/commit/e9f2c7214245239c83afb4576e824c34512c0c2a) RDNA-aware autotune configs for the INT8 matmul kernels - [`f004d58`](https://github.com/Comfy-Org/comfy-kitchen/commit/f004d5898832a673e4d41994a9c33766b3457e08) Add waves_per_eu variants to the RDNA INT8 autotune pool - [`ce6e98d`](https://github.com/Comfy-Org/comfy-kitchen/commit/ce6e98da39ee3ccd1c62cebcdc1f31c55f0057a5) fix(triton): scope RDNA INT8 autotuning per device - [`a6dcb22`](https://github.com/Comfy-Org/comfy-kitchen/commit/a6dcb2202f13309b0af562ef904ebf13438fbbcf) refactor(triton): share INT8 autotune kwargs - [`04ac357`](https://github.com/Comfy-Org/comfy-kitchen/commit/04ac357ae516cb1bf598ec9ded6e3519d47df5c5) fix(triton): improve RDNA INT8 autotune selection - [`a789909`](https://github.com/Comfy-Org/comfy-kitchen/commit/a78990909a19a8e56841270711e0ab03d2c60924) test(triton): cover INT8 device index forwarding ### 📊 Changes **2 files changed** (+295 additions, -21 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/triton/quantization.py` (+83 -21) 📝 `tests/test_int8.py` (+212 -0) </details> ### 📄 Description # RDNA-aware autotune configs for the INT8 matmul kernels ## What This keeps the existing CUDA/CDNA behavior and selects a ten-candidate INT8 Triton pool for the RDNA generation serving each call. `gfx11*` gets a small-M `64x64x128`, eight-warp tile; `gfx12*` retains its existing pool. For `M <= 128`, only the unstable `128x128x128` tile is removed. `device_index` remains in the autotune key so heterogeneous devices tune separately, but is marked non-specializing in both kernels so the scalar value does not create duplicate code variants. ## Why The original six default configs use `num_stages=3-4`. Shallow `num_stages=2` pools perform better on the tested consumer RDNA devices. Follow-up SDXL testing also showed that one pool should not be shared blindly across RDNA generations: the extra small-M tile helps `gfx11`, while replacing a `gfx12` tile regresses that generation. ## Validation - Rebased onto current `main` `5a997b17`; cumulative diff is two files (`+236/-21`). - Focused config/device tests: 18/18 pass. - Complete `tests/test_int8.py` on gfx1100, gfx1151, and gfx1201: 80 passed / 33 skipped / the same 5 CUDA-extension-only failures as the before arm. - Fixed-config correctness/performance sweeps use both actual INT8 kernels and real SDXL@1024 projection shapes. Every output is finite and matches across configs. - Same-base, fresh-process weighted SDXL kernel estimate, before -> after: - gfx1100: 67.20 -> 65.63 ms/step (-2.3%) - gfx1151: 129.06 -> 122.98 ms/step (-4.7%) - gfx1201: 39.24 -> 39.08 ms/step (-0.4%, effectively unchanged) - Per-device pruning returns ten RDNA candidates on every tested GPU; the 17-config decorator union is pruned before benchmarking. - Direct JIT-cache probe: changing `device_index` created one extra compiled entry before this update and zero afterward; the output remained finite. - Ruff, Python syntax, and whitespace checks pass. ## Reporter follow-up The exact 1.70 vs 1.79 s/it E2E run could not be reproduced without the reporter's private config/model setup. On gfx1151, their six-config pool was not slower than the one-config pool in steady-state kernel time; its first-use autotune cost was much larger. However, their feedback exposed a real selection gap in this PR's prior pool, and the generation-specific update above improves the matching SDXL shapes without regressing gfx1201. Fixes #68. AI usage disclosure: this update was prepared with AI assistance. The submitting human remains responsible for reviewing the changes and validation before publication. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 22:20:34 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#120

No description provided.

Rows
Columns