[PR #231] Tune SM120 V quantization for large D128 inputs #240

New issue

Open

opened 2026-10-26 22:21:54 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:21:54 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/231
Author: @Tokha233
Created: 10/1/2026
Status: 🔄 Open

Base: main ← Head: perf/sm120-v-quant-long

📝 Commits (3)

6dd6f95 Tune SM120 V quantization for large D128 inputs
14c13e0 Clarify V quantization benchmark reproduction
98f7fb9 Pin complete shared-object build provenance

📊 Changes

5 files changed (+2389 additions, -0 deletions)

View changed files

➕ benchmarks/bench_v_quant.py (+82 -0)
📝 comfy_kitchen/backends/cuda/sage_attention/quant_v_int8.cu (+38 -0)
➕ docs/benchmarks/sm120-v-quant.json (+2089 -0)
➕ docs/benchmarks/sm120-v-quant.md (+80 -0)
➕ tests/test_int8_v_quant_schedule.py (+100 -0)

📄 Description

Large D128 V inputs spend unnecessary time in the fixed 512-thread quantization block on SM120. Use 1024 threads for a bounded FP16/BF16, shape and layout range; retain the existing kernel math, output permutation, padding and all other schedules. This is independent of the D128 attention main-loop (#218) and D64 tile (#227) changes.

RTX 5090 D v2, Torch 2.12.0+cu130, CUDA 13.0.88:

BF16 H3 interleaved QKV, N=14,850: V quantization 0.852880 -> 0.521000 ms (-38.91%); complete attention 13.428368 -> 13.069504 ms (-2.67%).
Full Larry v4 INT8 8-step sampler: 15.688698 -> 15.537895 s (-0.9612%), four formal requests per arm in independent warmed AB/BA processes. All eight video/audio latent hashes match; peak Torch allocation is unchanged.
Separate profiling confirms 400 calls to the 1024-thread V kernel per sampler request.
38 targeted cases pass, one two-device case skipped. INT8 codes and FP32 scale bits match one-head old-schedule calls, including boundary shapes, special values, streams and CUDA Graphs.
Full attention regression has identical results on both arms: 457 passed, 595 skipped, two inherited zero/negative-scale failures already tracked by #224. No new failures.

Protocol and raw measurements. Main Python and attention/V sources are pinned to 12389a3; other native objects were reused identically from the existing combined test build, with only the V object replaced. Sampler timing excludes condition/VAE/export. This is not extra acceleration over the fully fused R85 QKV preparation path.

Prepared with AI assistance; tests and measurements ran on the stated hardware. DCO signed off.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/231 **Author:** [@Tokha233](https://github.com/Tokha233) **Created:** 10/1/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `perf/sm120-v-quant-long` --- ### 📝 Commits (3) - [`6dd6f95`](https://github.com/Comfy-Org/comfy-kitchen/commit/6dd6f95a7d4b8fd9ea1d6b72a695b513c2ac045c) Tune SM120 V quantization for large D128 inputs - [`14c13e0`](https://github.com/Comfy-Org/comfy-kitchen/commit/14c13e0b07c7e87b61fe408ae476202a077f7ea8) Clarify V quantization benchmark reproduction - [`98f7fb9`](https://github.com/Comfy-Org/comfy-kitchen/commit/98f7fb96aed08aada468d9daf49f2757a99178bf) Pin complete shared-object build provenance ### 📊 Changes **5 files changed** (+2389 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `benchmarks/bench_v_quant.py` (+82 -0) 📝 `comfy_kitchen/backends/cuda/sage_attention/quant_v_int8.cu` (+38 -0) ➕ `docs/benchmarks/sm120-v-quant.json` (+2089 -0) ➕ `docs/benchmarks/sm120-v-quant.md` (+80 -0) ➕ `tests/test_int8_v_quant_schedule.py` (+100 -0) </details> ### 📄 Description Large D128 V inputs spend unnecessary time in the fixed 512-thread quantization block on SM120. Use 1024 threads for a bounded FP16/BF16, shape and layout range; retain the existing kernel math, output permutation, padding and all other schedules. This is independent of the D128 attention main-loop (#218) and D64 tile (#227) changes. RTX 5090 D v2, Torch 2.12.0+cu130, CUDA 13.0.88: - BF16 H3 interleaved QKV, N=14,850: V quantization **0.852880 -> 0.521000 ms (-38.91%)**; complete attention **13.428368 -> 13.069504 ms (-2.67%)**. - Full Larry v4 INT8 8-step sampler: **15.688698 -> 15.537895 s (-0.9612%)**, four formal requests per arm in independent warmed AB/BA processes. All eight video/audio latent hashes match; peak Torch allocation is unchanged. - Separate profiling confirms 400 calls to the 1024-thread V kernel per sampler request. - 38 targeted cases pass, one two-device case skipped. INT8 codes and FP32 scale bits match one-head old-schedule calls, including boundary shapes, special values, streams and CUDA Graphs. - Full attention regression has identical results on both arms: 457 passed, 595 skipped, two inherited zero/negative-scale failures already tracked by #224. No new failures. [Protocol and raw measurements](https://github.com/Tokha233/comfy-kitchen/blob/perf/sm120-v-quant-long/docs/benchmarks/sm120-v-quant.md). Main Python and attention/V sources are pinned to 12389a3; other native objects were reused identically from the existing combined test build, with only the V object replaced. Sampler timing excludes condition/VAE/export. This is not extra acceleration over the fully fused R85 QKV preparation path. Prepared with AI assistance; tests and measurements ran on the stated hardware. DCO signed off. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 22:21:54 +00:00

bsakaguchi referenced this issue

2026-10-26 22:21:59 +00:00

[PR #240] Add quad remesh kernels (CORE-607) #248