[PR #56] [CLOSED] eager: dequantize INT8 weights instead of torch._int_mm on AMD RDNA2 #110

New issue

Closed

opened 2026-10-26 22:20:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:28 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/56
Author: @0xDELUXA
Created: 6/26/2026
Status: ❌ Closed

Base: main ← Head: eager-int8-rdna-dequant

📝 Commits (1)

d8a1cc9 eager: dequantize INT8 weights instead of torch._int_mm on AMD RDNA2

📊 Changes

1 file changed (+45 additions, -0 deletions)

View changed files

📝 comfy_kitchen/backends/eager/quantization.py (+45 -0)

📄 Description

torch._int_mm has no kernel on RDNA2 (gfx103x) - rocBLAS ships no integer GEMM, hipBLASLt doesn't support the arch, and the HIP backend drops int8_linear from its registry there (it needs WMMA), so the call falls through to eager and raises.

On RDNA2 the eager int8_linear now dequantizes the weight and runs an ordinary float GEMM. The weight stays stored as INT8, so the VRAM saving is unaffected; only the int8 math speedup is lost, and that never existed on that arch.

Every other target is unchanged and keeps torch._int_mm. A device whose arch can't be read also keeps it, rather than being assumed to be the one arch without INT8 GEMM.

COMFY_KITCHEN_INT8_EAGER=int_mm|dequant forces either path; the default is auto (the arch check above), cached per device.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/56 **Author:** [@0xDELUXA](https://github.com/0xDELUXA) **Created:** 6/26/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `eager-int8-rdna-dequant` --- ### 📝 Commits (1) - [`d8a1cc9`](https://github.com/Comfy-Org/comfy-kitchen/commit/d8a1cc9dd5a8d132f4ea2f963fde056c5ab89d62) eager: dequantize INT8 weights instead of torch._int_mm on AMD RDNA2 ### 📊 Changes **1 file changed** (+45 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/eager/quantization.py` (+45 -0) </details> ### 📄 Description `torch._int_mm` has no kernel on RDNA2 (`gfx103x`) - rocBLAS ships no integer GEMM, hipBLASLt doesn't support the arch, and the HIP backend drops `int8_linear` from its registry there (it needs WMMA), so the call falls through to eager and raises. On RDNA2 the eager `int8_linear` now dequantizes the weight and runs an ordinary float GEMM. The weight stays stored as INT8, so the VRAM saving is unaffected; only the int8 math speedup is lost, and that never existed on that arch. Every other target is unchanged and keeps `torch._int_mm`. A device whose arch can't be read also keeps it, rather than being assumed to be the one arch without INT8 GEMM. `COMFY_KITCHEN_INT8_EAGER=int_mm|dequant` forces either path; the default is `auto` (the arch check above), cached per device. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:28 +00:00