[PR #58] [CLOSED] Fix INT8 quantization Triton kernel for HIP/ROCm #112

New issue

Closed

opened 2026-10-26 22:20:30 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:30 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/58
Author: @jnolck
Created: 6/27/2026
Status: ❌ Closed

Base: main ← Head: fix-triton-hip-int8-rounding

📝 Commits (5)

fe31d4b Fix INT8 quantization Triton kernel for HIP/ROCm
2a700f3 fix: triton hip int8 rounding
c86c859 Use round-half-to-even to exactly match libdevice.rint behavior
f9161aa Fix negative half-integer rounding with sign-safe parity check
7ded4d8 Brain blank, thought this was going to be a five minute thing, fixed that last snafu

📊 Changes

1 file changed (+19 additions, -2 deletions)

View changed files

📝 comfy_kitchen/backends/triton/quantization.py (+19 -2)

📄 Description

Replace libdevice.rint() with manual sign-aware rounding since Triton 3.6.0 HIP backend lacks rint, math.round, and int clamp. All 33 INT8 tests pass on gfx1100 (RDNA 3 / RX 7900 XT).

⬢ [jnolck@toolbx tests]$ python -m pytest test_int8.py -v
========================================================== test session starts ==========================================================
platform linux -- Python 3.12.13, pytest-7.4.4, pluggy-1.6.0 -- /home/jnolck/Documents/src/ai/.venv/bin/python
cachedir: .pytest_cache
rootdir: /var/home/jnolck/Documents/src/ai/comfy-kitchen
configfile: pytest.ini
plugins: anyio-4.13.0, dash-4.2.0, hydra-core-1.3.3
collected 33 items                                                                                                                      

test_int8.py::test_cuda_int8_cublas_turing_n_alignment PASSED                                                                     [  3%]
test_int8.py::test_eager_int8_matmul_turing_n_alignment PASSED                                                                    [  6%]
test_int8.py::TestTensorWiseINT8Layout::test_weight_quantize_shape_dtype PASSED                                                   [  9%]
test_int8.py::TestTensorWiseINT8Layout::test_activation_quantize_shape_dtype PASSED                                               [ 12%]
test_int8.py::TestTensorWiseINT8Layout::test_weight_dequantize_dtype PASSED                                                       [ 15%]
test_int8.py::TestTensorWiseINT8Layout::test_weight_roundtrip_error PASSED                                                        [ 18%]
test_int8.py::TestTensorWiseINT8Layout::test_state_dict_tensors_keys PASSED                                                       [ 21%]
test_int8.py::TestTensorWiseINT8Layout::test_supports_fast_matmul PASSED                                                          [ 24%]
test_int8.py::TestTensorWiseINT8Layout::test_linear_dispatch PASSED                                                               [ 27%]
test_int8.py::TestTensorWiseINT8Layout::test_linear_dispatch_uses_activation_dtype PASSED                                         [ 30%]
test_int8.py::TestTensorWiseINT8Layout::test_mm_dispatch PASSED                                                                   [ 33%]
test_int8.py::TestTensorWiseINT8Layout::test_mm_dispatch_uses_activation_dtype PASSED                                             [ 36%]
test_int8.py::TestTensorWiseINT8Layout::test_addmm_dispatch PASSED                                                                [ 39%]
test_int8.py::TestTensorWiseINT8Layout::test_addmm_dispatch_uses_activation_dtype PASSED                                          [ 42%]
test_int8.py::TestTensorWiseINT8Layout::test_int8_linear_correctness[triton] PASSED                                               [ 45%]
test_int8.py::TestTensorWiseINT8Layout::test_int8_linear_correctness[eager] PASSED                                                [ 48%]
test_int8.py::TestTensorWiseINT8Layout::test_public_api_quantize_tensorwise PASSED                                                [ 51%]
test_int8.py::TestTensorWiseINT8Layout::test_public_api_quantize_rowwise PASSED                                                   [ 54%]
test_int8.py::TestTensorWiseINT8Layout::test_public_api_dequantize_simple PASSED                                                  [ 57%]
test_int8.py::TestTensorWiseINT8Layout::test_public_api_int8_linear PASSED                                                        [ 60%]
test_int8.py::TestTensorWiseINT8Layout::test_convrot_hadamard_properties PASSED                                                   [ 63%]
test_int8.py::TestTensorWiseINT8Layout::test_convrot_param_validation PASSED                                                      [ 66%]
test_int8.py::TestTensorWiseINT8Layout::test_convrot_weight_roundtrip PASSED                                                      [ 69%]
test_int8.py::TestTensorWiseINT8Layout::test_convrot_divisibility PASSED                                                          [ 72%]
test_int8.py::TestTensorWiseINT8Layout::test_convrot_linear_mm_addmm_dispatch PASSED                                              [ 75%]
test_int8.py::TestTensorWiseINT8Layout::test_convrot_triton_fused_correctness PASSED                                              [ 78%]
test_int8.py::TestTensorWisePublicAPI::test_public_api_quantize_tensorwise PASSED                                                 [ 81%]
test_int8.py::TestTensorWisePublicAPI::test_public_api_quantize_rowwise PASSED                                                    [ 84%]
test_int8.py::TestTensorWisePublicAPI::test_public_api_dequantize_simple PASSED                                                   [ 87%]
test_int8.py::TestTensorWisePublicAPI::test_public_api_int8_linear PASSED                                                         [ 90%]
test_int8.py::TestTensorWisePublicAPI::test_eager_int8_linear_single_row PASSED                                                   [ 93%]
test_int8.py::TestTensorWisePublicAPI::test_eager_int8_linear_pads_k_to_int8_mm_tile PASSED                                       [ 96%]
test_int8.py::TestTensorWisePublicAPI::test_eager_int8_linear_pads_n_to_int8_mm_tile PASSED                                       [100%]

========================================================== 33 passed in 13.94s ==========================================================

A lot of them were failing before and I was getting seg faults when trying to run workflows and started comfy with --enable-triton-backend.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/58 **Author:** [@jnolck](https://github.com/jnolck) **Created:** 6/27/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix-triton-hip-int8-rounding` --- ### 📝 Commits (5) - [`fe31d4b`](https://github.com/Comfy-Org/comfy-kitchen/commit/fe31d4b923d19693ffd7ca3450a52175a7cf44ad) Fix INT8 quantization Triton kernel for HIP/ROCm - [`2a700f3`](https://github.com/Comfy-Org/comfy-kitchen/commit/2a700f322a9c5e52317c633e1eddb01ece545526) fix: triton hip int8 rounding - [`c86c859`](https://github.com/Comfy-Org/comfy-kitchen/commit/c86c859723ece158206f7bc02a0f2d431670d66c) Use round-half-to-even to exactly match libdevice.rint behavior - [`f9161aa`](https://github.com/Comfy-Org/comfy-kitchen/commit/f9161aa85b02fcf8a71d1d9e5b5e6f080445f30f) Fix negative half-integer rounding with sign-safe parity check - [`7ded4d8`](https://github.com/Comfy-Org/comfy-kitchen/commit/7ded4d8cf8d9a5d03eb9fd0ad17a000b72aa445d) Brain blank, thought this was going to be a five minute thing, fixed that last snafu ### 📊 Changes **1 file changed** (+19 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/triton/quantization.py` (+19 -2) </details> ### 📄 Description Replace libdevice.rint() with manual sign-aware rounding since Triton 3.6.0 HIP backend lacks rint, math.round, and int clamp. All 33 INT8 tests pass on gfx1100 (RDNA 3 / RX 7900 XT). ``` ⬢ [jnolck@toolbx tests]$ python -m pytest test_int8.py -v ========================================================== test session starts ========================================================== platform linux -- Python 3.12.13, pytest-7.4.4, pluggy-1.6.0 -- /home/jnolck/Documents/src/ai/.venv/bin/python cachedir: .pytest_cache rootdir: /var/home/jnolck/Documents/src/ai/comfy-kitchen configfile: pytest.ini plugins: anyio-4.13.0, dash-4.2.0, hydra-core-1.3.3 collected 33 items test_int8.py::test_cuda_int8_cublas_turing_n_alignment PASSED [ 3%] test_int8.py::test_eager_int8_matmul_turing_n_alignment PASSED [ 6%] test_int8.py::TestTensorWiseINT8Layout::test_weight_quantize_shape_dtype PASSED [ 9%] test_int8.py::TestTensorWiseINT8Layout::test_activation_quantize_shape_dtype PASSED [ 12%] test_int8.py::TestTensorWiseINT8Layout::test_weight_dequantize_dtype PASSED [ 15%] test_int8.py::TestTensorWiseINT8Layout::test_weight_roundtrip_error PASSED [ 18%] test_int8.py::TestTensorWiseINT8Layout::test_state_dict_tensors_keys PASSED [ 21%] test_int8.py::TestTensorWiseINT8Layout::test_supports_fast_matmul PASSED [ 24%] test_int8.py::TestTensorWiseINT8Layout::test_linear_dispatch PASSED [ 27%] test_int8.py::TestTensorWiseINT8Layout::test_linear_dispatch_uses_activation_dtype PASSED [ 30%] test_int8.py::TestTensorWiseINT8Layout::test_mm_dispatch PASSED [ 33%] test_int8.py::TestTensorWiseINT8Layout::test_mm_dispatch_uses_activation_dtype PASSED [ 36%] test_int8.py::TestTensorWiseINT8Layout::test_addmm_dispatch PASSED [ 39%] test_int8.py::TestTensorWiseINT8Layout::test_addmm_dispatch_uses_activation_dtype PASSED [ 42%] test_int8.py::TestTensorWiseINT8Layout::test_int8_linear_correctness[triton] PASSED [ 45%] test_int8.py::TestTensorWiseINT8Layout::test_int8_linear_correctness[eager] PASSED [ 48%] test_int8.py::TestTensorWiseINT8Layout::test_public_api_quantize_tensorwise PASSED [ 51%] test_int8.py::TestTensorWiseINT8Layout::test_public_api_quantize_rowwise PASSED [ 54%] test_int8.py::TestTensorWiseINT8Layout::test_public_api_dequantize_simple PASSED [ 57%] test_int8.py::TestTensorWiseINT8Layout::test_public_api_int8_linear PASSED [ 60%] test_int8.py::TestTensorWiseINT8Layout::test_convrot_hadamard_properties PASSED [ 63%] test_int8.py::TestTensorWiseINT8Layout::test_convrot_param_validation PASSED [ 66%] test_int8.py::TestTensorWiseINT8Layout::test_convrot_weight_roundtrip PASSED [ 69%] test_int8.py::TestTensorWiseINT8Layout::test_convrot_divisibility PASSED [ 72%] test_int8.py::TestTensorWiseINT8Layout::test_convrot_linear_mm_addmm_dispatch PASSED [ 75%] test_int8.py::TestTensorWiseINT8Layout::test_convrot_triton_fused_correctness PASSED [ 78%] test_int8.py::TestTensorWisePublicAPI::test_public_api_quantize_tensorwise PASSED [ 81%] test_int8.py::TestTensorWisePublicAPI::test_public_api_quantize_rowwise PASSED [ 84%] test_int8.py::TestTensorWisePublicAPI::test_public_api_dequantize_simple PASSED [ 87%] test_int8.py::TestTensorWisePublicAPI::test_public_api_int8_linear PASSED [ 90%] test_int8.py::TestTensorWisePublicAPI::test_eager_int8_linear_single_row PASSED [ 93%] test_int8.py::TestTensorWisePublicAPI::test_eager_int8_linear_pads_k_to_int8_mm_tile PASSED [ 96%] test_int8.py::TestTensorWisePublicAPI::test_eager_int8_linear_pads_n_to_int8_mm_tile PASSED [100%] ========================================================== 33 passed in 13.94s ========================================================== ``` A lot of them were failing before and I was getting seg faults when trying to run workflows and started comfy with --enable-triton-backend. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:20:30 +00:00