[PR #151] [MERGED] [Feat][Ascend] Add Ascend INT8 quantization backend #180

New issue

Closed

opened 2026-10-26 22:21:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/151
Author: @Big2Wheel
Created: 9/3/2026
Status: ✅ Merged
Merged: 9/24/2026
Merged by: @comfyanonymous

Base: main ← Head: codex/ascend-backend-base

📝 Commits (3)

4cffd74 feat(npu): add Ascend quantization backend
4a1442f test(npu): include NPU in supported devices
3bab240 refactor(ascend): clarify backend naming

📊 Changes

7 files changed (+394 additions, -41 deletions)

View changed files

📝 README.md (+56 -38)
📝 comfy_kitchen/__init__.py (+1 -0)
➕ comfy_kitchen/backends/ascend/__init__.py (+187 -0)
📝 comfy_kitchen/registry.py (+1 -1)
📝 tests/conftest.py (+2 -2)
➕ tests/test_ascend_backend.py (+137 -0)
📝 tests/test_backends.py (+10 -0)

📄 Description

What this PR does

Adds an optional ascend backend for Huawei Ascend NPU hardware to comfy-kitchen and registers the first INT8 quantization primitives:

quantize_int8_rowwise using torch_npu.npu_dynamic_quant
quantize_int8_tensorwise using Ascend reductions and torch_npu.npu_quantize
device-side dequantize_int8_simple
device-side dequantize_int8_simple_dtype

The backend is registered only when torch-npu and an Ascend device are available. CPU, CUDA, HIP, and XPU installations do not gain a torch-npu dependency. The project-facing backend name is ascend; the PyTorch device type remains npu because that is the API defined by torch-npu.

Constraint-based dispatch limits the Ascend implementation to supported devices and dtypes. FP32 inputs and stochastic rounding continue through the existing device-side eager implementation rather than copying tensors to CPU.

This is the backend-foundation part of Ascend support. INT8 Linear and additional fused operators will be proposed separately to keep review scope focused.

Related to Comfy-Org/ComfyUI#11909.

Why

Without an Ascend backend, comfy-kitchen cannot select torch-npu quantization operators. This change establishes the backend integration and validates the Q/DQ contract before adding Ascend INT8 matrix multiplication.

Testing

Tested on one Huawei Ascend 910B3 device with torch 2.10.0 and torch-npu 2.10.0.post4.

Renamed backend, Ascend-specific, and Q/DQ regression tests: 62 passed, 4 skipped
Full post-rebase stacked suite (validated through #155): 660 passed, 1603 skipped, 0 failed
ruff check: passed
git diff --check: passed

The skipped tests require unavailable CUDA/HIP hardware.

Ascend profiler traces confirmed:

npu::npu_dynamic_quant / aclnnDynamicQuantV2
npu::npu_quantize / aclnnQuantize
no H2D, D2H, or memcpy events in the tested Q/DQ path

For a BF16 tensor with shape [8352, 3840] (10 warmups, 30 measured iterations):

Operation	Eager mean	Ascend mean	Improvement
Row-wise INT8 quantization	1.119 ms	0.195 ms	82.5%
Tensor-wise INT8 quantization	1.018 ms	0.343 ms	66.3%

Tests cover FP16/BF16, all-zero rows, supplied/recalculated scales, output dtypes, automatic backend selection, unsupported-call rejection, and device-side eager fallback.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/151 **Author:** [@Big2Wheel](https://github.com/Big2Wheel) **Created:** 9/3/2026 **Status:** ✅ Merged **Merged:** 9/24/2026 **Merged by:** [@comfyanonymous](https://github.com/comfyanonymous) **Base:** `main` ← **Head:** `codex/ascend-backend-base` --- ### 📝 Commits (3) - [`4cffd74`](https://github.com/Comfy-Org/comfy-kitchen/commit/4cffd745c5c27bdff2190e87ec3b228e06ac8e6b) feat(npu): add Ascend quantization backend - [`4a1442f`](https://github.com/Comfy-Org/comfy-kitchen/commit/4a1442f98a2f11ac29c2012e739ad1a03246052d) test(npu): include NPU in supported devices - [`3bab240`](https://github.com/Comfy-Org/comfy-kitchen/commit/3bab2407a92d5b96d044708ca05961d15cc1c6bd) refactor(ascend): clarify backend naming ### 📊 Changes **7 files changed** (+394 additions, -41 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+56 -38) 📝 `comfy_kitchen/__init__.py` (+1 -0) ➕ `comfy_kitchen/backends/ascend/__init__.py` (+187 -0) 📝 `comfy_kitchen/registry.py` (+1 -1) 📝 `tests/conftest.py` (+2 -2) ➕ `tests/test_ascend_backend.py` (+137 -0) 📝 `tests/test_backends.py` (+10 -0) </details> ### 📄 Description ## What this PR does Adds an optional `ascend` backend for Huawei Ascend NPU hardware to comfy-kitchen and registers the first INT8 quantization primitives: - `quantize_int8_rowwise` using `torch_npu.npu_dynamic_quant` - `quantize_int8_tensorwise` using Ascend reductions and `torch_npu.npu_quantize` - device-side `dequantize_int8_simple` - device-side `dequantize_int8_simple_dtype` The backend is registered only when torch-npu and an Ascend device are available. CPU, CUDA, HIP, and XPU installations do not gain a torch-npu dependency. The project-facing backend name is `ascend`; the PyTorch device type remains `npu` because that is the API defined by torch-npu. Constraint-based dispatch limits the Ascend implementation to supported devices and dtypes. FP32 inputs and stochastic rounding continue through the existing device-side eager implementation rather than copying tensors to CPU. This is the backend-foundation part of Ascend support. INT8 Linear and additional fused operators will be proposed separately to keep review scope focused. Related to Comfy-Org/ComfyUI#11909. ## Why Without an Ascend backend, comfy-kitchen cannot select torch-npu quantization operators. This change establishes the backend integration and validates the Q/DQ contract before adding Ascend INT8 matrix multiplication. ## Testing Tested on one Huawei Ascend 910B3 device with torch 2.10.0 and torch-npu 2.10.0.post4. - Renamed backend, Ascend-specific, and Q/DQ regression tests: 62 passed, 4 skipped - Full post-rebase stacked suite (validated through #155): 660 passed, 1603 skipped, 0 failed - `ruff check`: passed - `git diff --check`: passed The skipped tests require unavailable CUDA/HIP hardware. Ascend profiler traces confirmed: - `npu::npu_dynamic_quant` / `aclnnDynamicQuantV2` - `npu::npu_quantize` / `aclnnQuantize` - no H2D, D2H, or memcpy events in the tested Q/DQ path For a BF16 tensor with shape `[8352, 3840]` (10 warmups, 30 measured iterations): | Operation | Eager mean | Ascend mean | Improvement | |---|---:|---:|---:| | Row-wise INT8 quantization | 1.119 ms | 0.195 ms | 82.5% | | Tensor-wise INT8 quantization | 1.018 ms | 0.343 ms | 66.3% | Tests cover FP16/BF16, all-zero rows, supplied/recalculated scales, output dtypes, automatic backend selection, unsupported-call rejection, and device-side eager fallback. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:21:17 +00:00