mirror of
https://github.com/Comfy-Org/comfy-kitchen.git
synced 2026-11-04 23:37:42 +00:00
[PR #223] Compose INT8 input quantization with indexed gate epilogue #235
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Comfy-Org/comfy-kitchen#235
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
📋 Pull Request Information
Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/223
Author: @Tokha233
Created: 9/30/2026
Status: 🔄 Open
Base:
main← Head:cuda/indexed-linear-gate📝 Commits (7)
dc7c739Add prequantized INT8 GEMM with indexed BF16 gate and residual5f7290dFix indexed gate fallback layout and wheel licenseba876f2Compose ConvRot INT8 input quantization with indexed residual gate6bf0cb0Guard indexed linear ConvRot kernel limits23ba820Merge main and retain indexed linear exports3ac922fAvoid unused indexed-gate output allocation on fallback4c363e7Sync indexed gate exports with Kitchen 0.2.40📊 Changes
17 files changed (+1351 additions, -1 deletions)
View changed files
📝
NOTICE(+7 -0)📝
comfy_kitchen/__init__.py(+3 -0)📝
comfy_kitchen/backends/cuda/CMakeLists.txt(+1 -0)📝
comfy_kitchen/backends/cuda/dlpack_bindings.cpp(+42 -0)➕
comfy_kitchen/backends/cuda/ops/cutlass_gemm_indexed_gate.cu(+223 -0)➕
comfy_kitchen/backends/cuda/ops/indexed_gate.cuh(+155 -0)➕
comfy_kitchen/indexed_gate.py(+199 -0)➕
docs/benchmarks/gate-api-benchmark.json(+210 -0)➕
docs/benchmarks/indexed-gate-wheel.json(+11 -0)➕
docs/benchmarks/indexed-linear-sampler.json(+148 -0)➕
docs/benchmarks/indexed-linear.md(+12 -0)➕
docs/benchmarks/int8-indexed-gate.md(+52 -0)➕
docs/benchmarks/source-sha256.json(+11 -0)📝
pyproject.toml(+1 -1)➕
tests/benchmark_int8_indexed_gate.py(+72 -0)➕
tests/test_indexed_linear.py(+46 -0)➕
tests/test_int8_indexed_gate.py(+158 -0)📄 Description
Follow-up to #219: expose BF16 input activation quantization plus the indexed gate GEMM through
int8_linear_indexed_gate. This lets a consumer fold SwiGLU/ConvRot quantization and FC2's compact modulation gate without calling private CUDA bindings. No new native kernel is added.CPU/other-device fallback uses the existing
int8_linearoutput before addcmul, preserving its backend rounding. CUDA uses the existing ConvRot256 quantizer plus #219. The operation and fake contract support real Inductor.Validation: 33 tests pass on RTX 5090 D v2 / Torch 2.12.0+cu130 (indexed GEMM and composed CPU/CUDA, including wide-K, shared-memory fallback and Inductor regressions). Current-ComfyUI FC2-only sampler experiment, four alternating formal pairs, Larry v4 INT8 / 8 steps / 768×512 / 124 frames: 15.802266 → 15.583501 s (−1.3844%), identical video/audio latent hashes in all eight formal requests. Peak Torch memory +153 KiB. Raw records and scope in
docs/benchmarks/indexed-linear.md.Stacked on #219, which must merge first. Ready for review of the composed API. The incremental comparison is https://github.com/Tokha233/comfy-kitchen/compare/5f7290d1301e6bc7e2109c7c3d35a3d14f7f3de9...cuda/indexed-linear-gate . It includes the wide-K/shared-memory fallback fix; inherited epilogue commits belong to #219. A companion ComfyUI consumer uses normal weight casting/offloading and preserves training/hooked paths. No additive speedup or service-throughput claim.
Prepared with AI assistance and executed GPU tests; DCO signed off.
Latest combined validation and per-PR fixes: https://github.com/Tokha233/ComfyUI-H3-SpeedKit/blob/main/docs/UPSTREAM-RECOVERY-TESTS-20261001.zh-CN.md . Combined gains overlap and must not be added to the single-operation timings above.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.