[PR #153] [CLOSED] [Feat][Ascend] Add Ascend INT8 linear backend #179

New issue

Closed

opened 2026-10-26 22:21:16 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:16 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/153
Author: @Big2Wheel
Created: 9/4/2026
Status: ❌ Closed

Base: main ← Head: codex/ascend-int8-linear

📝 Commits (5)

5942fae feat(npu): add Ascend quantization backend
1ecb943 test(npu): include NPU in supported devices
0bccee7 refactor(ascend): clarify backend naming
6ab910a feat(ascend): add INT8 linear backend
91a5a3f test(ascend): handle optional quant matmul

📊 Changes

7 files changed (+674 additions, -41 deletions)

View changed files

📝 README.md (+56 -38)
📝 comfy_kitchen/__init__.py (+1 -0)
➕ comfy_kitchen/backends/ascend/__init__.py (+331 -0)
📝 comfy_kitchen/registry.py (+1 -1)
📝 tests/conftest.py (+2 -2)
➕ tests/test_ascend_backend.py (+268 -0)
📝 tests/test_backends.py (+15 -0)

📄 Description

What this PR does

Adds an Ascend implementation of Comfy Kitchen's int8_linear capability.

dynamically quantizes FP16/BF16 activations with torch_npu.npu_dynamic_quant
executes the linear layer with torch_npu.npu_quant_matmul
supports scalar and per-output-channel weight scales, optional bias, 2D and batched inputs
supports the existing ConvRot path and gelu_tanh/swiglu input activations
validates the call contract before backend selection, so unsupported shapes, dtypes, activations, and ConvRot group sizes fall back through the normal registry path
only advertises int8_linear when the installed torch-npu exposes npu_quant_matmul

This avoids the eager aten::_int_mm path, which falls back to CPU on the tested Ascend stack.

Relationship to #151

This is a stacked follow-up to #151 and intentionally builds on its three commits.

#151 introduces the optional ascend backend, its registration/availability handling, the shared INT8 quantize/dequantize capabilities, and NPU device discovery. This PR adds the first compute-heavy consumer of that foundation: Ascend-native INT8 linear/ConvRot execution.

Please review and merge #151 first. Until #151 lands, GitHub shows those base commits in this PR as well; after #151 is merged, this branch can be rebased/updated so the remaining diff contains only the INT8-linear-specific commits.

User-facing change

On supported Ascend NPU environments, Comfy Kitchen can automatically select the ascend backend for compatible INT8 linear layers. Other devices and unsupported Ascend calls retain the existing fallback behavior.

How this was tested

Environment:

Ascend 910B3, restricted to one device
Python 3.12
PyTorch 2.10.0
torch-npu 2.10.0.post4

Unit/regression tests:

pytest -q tests/test_ascend_backend.py tests/test_backends.py tests/test_constraints.py tests/test_int8.py
110 passed, 68 skipped

python -m ruff check comfy_kitchen/backends/ascend/__init__.py tests/test_ascend_backend.py tests/test_backends.py
All checks passed

End-to-end validation:

Z-Image Turbo INT8 ConvRot checkpoint
1920x1088, 8 sampling steps, fixed prompt and seed
1,360 automatic ascend backend selections for int8_linear per run
no aten::_int_mm or CPU fallback
five generated outputs had the same MD5: 428c244502d03e9c7f99431e2d1070f1
previous hot-run times: 13.26 s, 13.16 s, and 12.77 s (mean 13.06 s)
post-rebase fresh-process validation completed in 16.14 s, including model loading/warm-up effects

The eager baseline was also checked: it explicitly fell back from aten::_int_mm to CPU and processed only about 10 INT8 linear calls in roughly 55 seconds, so it was stopped rather than presented as a misleading full-workflow comparison.

Representative real-layer operator measurements (M=8352):

Layer	Dequantized BF16	Ascend INT8	Latency reduction
Attention output	1.088 ms	0.938 ms	13.8%
QKV projection	3.139 ms	2.102 ms	33.0%
FFN w1	2.766 ms	1.929 ms	30.3%
FFN w2	2.870 ms	1.990 ms	30.7%

Numerical comparison against a dequantized reference produced cosine similarity around 0.999958 and relative MAE around 0.89%. These benchmark figures document the tested setup and are not intended as a universal performance guarantee.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/153 **Author:** [@Big2Wheel](https://github.com/Big2Wheel) **Created:** 9/4/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `codex/ascend-int8-linear` --- ### 📝 Commits (5) - [`5942fae`](https://github.com/Comfy-Org/comfy-kitchen/commit/5942fae7a80bc3c5df640b661c2fbd1d2b5fb092) feat(npu): add Ascend quantization backend - [`1ecb943`](https://github.com/Comfy-Org/comfy-kitchen/commit/1ecb9434690dc771152d108038da8612ff9f1ff4) test(npu): include NPU in supported devices - [`0bccee7`](https://github.com/Comfy-Org/comfy-kitchen/commit/0bccee7393739704ae3c05ab233908820df51eef) refactor(ascend): clarify backend naming - [`6ab910a`](https://github.com/Comfy-Org/comfy-kitchen/commit/6ab910aad706e9ee28d140f36e964649165e909c) feat(ascend): add INT8 linear backend - [`91a5a3f`](https://github.com/Comfy-Org/comfy-kitchen/commit/91a5a3fed39a8b88858aaa0b9e214a59ccc32e90) test(ascend): handle optional quant matmul ### 📊 Changes **7 files changed** (+674 additions, -41 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+56 -38) 📝 `comfy_kitchen/__init__.py` (+1 -0) ➕ `comfy_kitchen/backends/ascend/__init__.py` (+331 -0) 📝 `comfy_kitchen/registry.py` (+1 -1) 📝 `tests/conftest.py` (+2 -2) ➕ `tests/test_ascend_backend.py` (+268 -0) 📝 `tests/test_backends.py` (+15 -0) </details> ### 📄 Description ## What this PR does Adds an Ascend implementation of Comfy Kitchen's `int8_linear` capability. - dynamically quantizes FP16/BF16 activations with `torch_npu.npu_dynamic_quant` - executes the linear layer with `torch_npu.npu_quant_matmul` - supports scalar and per-output-channel weight scales, optional bias, 2D and batched inputs - supports the existing ConvRot path and `gelu_tanh`/`swiglu` input activations - validates the call contract before backend selection, so unsupported shapes, dtypes, activations, and ConvRot group sizes fall back through the normal registry path - only advertises `int8_linear` when the installed torch-npu exposes `npu_quant_matmul` This avoids the eager `aten::_int_mm` path, which falls back to CPU on the tested Ascend stack. ## Relationship to #151 This is a stacked follow-up to #151 and intentionally builds on its three commits. #151 introduces the optional `ascend` backend, its registration/availability handling, the shared INT8 quantize/dequantize capabilities, and NPU device discovery. This PR adds the first compute-heavy consumer of that foundation: Ascend-native INT8 linear/ConvRot execution. Please review and merge #151 first. Until #151 lands, GitHub shows those base commits in this PR as well; after #151 is merged, this branch can be rebased/updated so the remaining diff contains only the INT8-linear-specific commits. ## User-facing change On supported Ascend NPU environments, Comfy Kitchen can automatically select the `ascend` backend for compatible INT8 linear layers. Other devices and unsupported Ascend calls retain the existing fallback behavior. ## How this was tested Environment: - Ascend 910B3, restricted to one device - Python 3.12 - PyTorch 2.10.0 - torch-npu 2.10.0.post4 Unit/regression tests: ```text pytest -q tests/test_ascend_backend.py tests/test_backends.py tests/test_constraints.py tests/test_int8.py 110 passed, 68 skipped ``` ```text python -m ruff check comfy_kitchen/backends/ascend/__init__.py tests/test_ascend_backend.py tests/test_backends.py All checks passed ``` End-to-end validation: - Z-Image Turbo INT8 ConvRot checkpoint - 1920x1088, 8 sampling steps, fixed prompt and seed - 1,360 automatic `ascend` backend selections for `int8_linear` per run - no `aten::_int_mm` or CPU fallback - five generated outputs had the same MD5: `428c244502d03e9c7f99431e2d1070f1` - previous hot-run times: 13.26 s, 13.16 s, and 12.77 s (mean 13.06 s) - post-rebase fresh-process validation completed in 16.14 s, including model loading/warm-up effects The eager baseline was also checked: it explicitly fell back from `aten::_int_mm` to CPU and processed only about 10 INT8 linear calls in roughly 55 seconds, so it was stopped rather than presented as a misleading full-workflow comparison. Representative real-layer operator measurements (`M=8352`): | Layer | Dequantized BF16 | Ascend INT8 | Latency reduction | | --- | ---: | ---: | ---: | | Attention output | 1.088 ms | 0.938 ms | 13.8% | | QKV projection | 3.139 ms | 2.102 ms | 33.0% | | FFN w1 | 2.766 ms | 1.929 ms | 30.3% | | FFN w2 | 2.870 ms | 1.990 ms | 30.7% | Numerical comparison against a dequantized reference produced cosine similarity around 0.999958 and relative MAE around 0.89%. These benchmark figures document the tested setup and are not intended as a universal performance guarantee. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:16 +00:00