[PR #205] Fall back to eager when a one-element INT8 scale adds leading dims #219

New issue

Open

opened 2026-10-26 22:21:41 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:41 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/205
Author: @arcusbuilds
Created: 9/26/2026
Status: 🔄 Open

Base: main ← Head: fix/int8-one-element-scale-broadcast-main

📝 Commits (1)

03847be Fall back to eager when a one-element INT8 scale adds leading dims

📊 Changes

4 files changed (+15 additions, -2 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/__init__.py (+1 -1)
📝 comfy_kitchen/backends/hip/__init__.py (+1 -1)
📝 tests/test_hip_wmma.py (+1 -0)
📝 tests/test_int8.py (+12 -0)

📄 Description

Fixes #204.

cuda and hip used scalar mode for any one-element scale. When that scale has more dims than q, eager broadcasting adds leading dims, so the GPU backends returned a different shape. Scalar mode now also requires scale.dim() <= q.dim(); other shapes go to the existing eager fallback.

Tests

cuda: new test_dequantize_one_element_scale_with_extra_dims_matches_eager in tests/test_int8.py. Fails without the fix ((3, 4, 32) vs eager (1, 3, 4, 32)), passes with it. pytest tests/test_int8.py -k dequantize: 8 passed (RTX 2000 Ada, sm_89).
hip: added a (1, 1, 1, 1) case to test_dequantize_int8_simple_dtype_matches_eager in tests/test_hip_wmma.py. Not run locally (no AMD GPU); the change is the same one-line guard as cuda.
ruff check clean on all changed files.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/205 **Author:** [@arcusbuilds](https://github.com/arcusbuilds) **Created:** 9/26/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/int8-one-element-scale-broadcast-main` --- ### 📝 Commits (1) - [`03847be`](https://github.com/Comfy-Org/comfy-kitchen/commit/03847be96e57fb2c676ef3e3b3a728199c87d48f) Fall back to eager when a one-element INT8 scale adds leading dims ### 📊 Changes **4 files changed** (+15 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/__init__.py` (+1 -1) 📝 `comfy_kitchen/backends/hip/__init__.py` (+1 -1) 📝 `tests/test_hip_wmma.py` (+1 -0) 📝 `tests/test_int8.py` (+12 -0) </details> ### 📄 Description Fixes #204. cuda and hip used scalar mode for any one-element scale. When that scale has more dims than `q`, eager broadcasting adds leading dims, so the GPU backends returned a different shape. Scalar mode now also requires `scale.dim() <= q.dim()`; other shapes go to the existing eager fallback. ## Tests - cuda: new `test_dequantize_one_element_scale_with_extra_dims_matches_eager` in `tests/test_int8.py`. Fails without the fix (`(3, 4, 32)` vs eager `(1, 3, 4, 32)`), passes with it. `pytest tests/test_int8.py -k dequantize`: 8 passed (RTX 2000 Ada, sm_89). - hip: added a `(1, 1, 1, 1)` case to `test_dequantize_int8_simple_dtype_matches_eager` in `tests/test_hip_wmma.py`. Not run locally (no AMD GPU); the change is the same one-line guard as cuda. - `ruff check` clean on all changed files. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 22:21:41 +00:00

bsakaguchi referenced this issue

2026-10-26 22:21:51 +00:00

[PR #219] Add prequantized INT8 GEMM with indexed BF16 gate and residual #234

bsakaguchi referenced this issue

2026-10-26 22:21:51 +00:00

[PR #223] Compose INT8 input quantization with indexed gate epilogue #235