Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

[PR #124] Round fused SwiGLU to the storage dtype before ConvRot. #159

New issue

Open

opened 2026-10-26 22:20:59 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:20:59 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-kitchen/pull/124
Author: @niehen6174
Created: 8/20/2026
Status: 🔄 Open

Base: main ← Head: fix/int8-swiglu-match-eager

📝 Commits (1)

860255d Round fused SwiGLU to the storage dtype before ConvRot.

📊 Changes

3 files changed (+33 additions, -7 deletions)

View changed files

📝 comfy_kitchen/backends/cuda/ops/int8_linear.cu (+7 -2)
📝 comfy_kitchen/backends/hip/hadamard.h (+14 -2)
📝 tests/test_int8_input_act.py (+12 -3)

📄 Description

Summary

int8_linear(..., input_act="swiglu") is supposed to equal int8_linear(silu(gate)*up, ...).
The fused CUDA/HIP quantizer left silu*up in fp32, so ConvRot's per-row absmax
diverged from the eager chain (which materializes BF16/FP16). One LSB of scale
rewrites the whole INT8 row.

This PR rounds silu and the multiply to the storage dtype before the rotation.

Accuracy

Kitchen unit shapes + MiniMax-H3 fc2 (N=5376, K=14336), isolated build of this patch,
F.silu(gate)*up then int8_linear vs fused:

	bit-neq	max\|Δ\|
M=4096	0 / 22,020,096	0
M=32700 (H3 1344×768 packed rows)	0 / 175,795,200	0

Same H3 t2va, 50 NFE, seed=42, INT8+FA, old fused kernel (fp32 product) vs
unfused INT8 (eager SwiGLU then quantize):

	video PSNR mean / min	audio PSNR
old vs unfused INT8	22.0 / 19.8	49.8

That is a different diffusion sample, not a slightly noisier tensor.
After this PR, fc2 is bit-exact, so that pair should be identical (PSNR 99).

Speed (why keep the fuse)

H3 fc2, M=32700, RTX 4090 D, CUDA event, 50 DiT blocks/step:

	ms / block
`silu*mul` + `int8_linear`	16.44
fused `input_act=swiglu` (this PR)	12.47
delta	−3.97

20 NFE exclusive 4090 D (SGLang, earlier run with the old fuse; wall-clock
should match because the extra rounding is register-only):

	denoise	e2e	step
INT8+FA, no fuse	276.1 s	294.7 s	13.80 s
+ fused SwiGLU	271.7 s	290.3 s	13.58 s
delta	−4.4 s	−4.4 s	−0.22 s

Test plan

pytest tests/test_int8_input_act.py -k swiglu

CUDA+bf16 int8_linear(x, input_act="swiglu") must torch.equal the eager chain.
FP16/BF16 fused INT8 codes must equal quantize(silu(gate)*up).

Files

comfy_kitchen/backends/cuda/ops/int8_linear.cu — CUDA quantizer
comfy_kitchen/backends/hip/hadamard.h — HIP mirror
tests/test_int8_input_act.py — require bit-exact on the fused CUDA path

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-kitchen/pull/124 **Author:** [@niehen6174](https://github.com/niehen6174) **Created:** 8/20/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/int8-swiglu-match-eager` --- ### 📝 Commits (1) - [`860255d`](https://github.com/Comfy-Org/comfy-kitchen/commit/860255d24e3711115f4f44c6dae70fc9faa7f95d) Round fused SwiGLU to the storage dtype before ConvRot. ### 📊 Changes **3 files changed** (+33 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `comfy_kitchen/backends/cuda/ops/int8_linear.cu` (+7 -2) 📝 `comfy_kitchen/backends/hip/hadamard.h` (+14 -2) 📝 `tests/test_int8_input_act.py` (+12 -3) </details> ### 📄 Description ## Summary `int8_linear(..., input_act="swiglu")` is supposed to equal `int8_linear(silu(gate)*up, ...)`. The fused CUDA/HIP quantizer left `silu*up` in fp32, so ConvRot's per-row absmax diverged from the eager chain (which materializes BF16/FP16). One LSB of scale rewrites the whole INT8 row. This PR rounds silu and the multiply to the storage dtype before the rotation. ## Accuracy Kitchen unit shapes + MiniMax-H3 fc2 (`N=5376`, `K=14336`), isolated build of this patch, `F.silu(gate)*up` then `int8_linear` vs fused: | | bit-neq | max\|Δ\| | |---|---|---| | M=4096 | **0 / 22,020,096** | 0 | | M=32700 (H3 1344×768 packed rows) | **0 / 175,795,200** | 0 | Same H3 t2va, 50 NFE, seed=42, INT8+FA, **old fused kernel** (fp32 product) vs unfused INT8 (eager SwiGLU then quantize): | | video PSNR mean / min | audio PSNR | |---|---|---| | old vs unfused INT8 | **22.0 / 19.8** | 49.8 | That is a different diffusion sample, not a slightly noisier tensor. After this PR, fc2 is **bit-exact**, so that pair should be identical (**PSNR 99**). ## Speed (why keep the fuse) H3 fc2, M=32700, RTX 4090 D, CUDA event, 50 DiT blocks/step: | | ms / block | |---|---| | `silu*mul` + `int8_linear` | 16.44 | | fused `input_act=swiglu` (this PR) | **12.47** | | delta | **−3.97** | 20 NFE exclusive 4090 D (SGLang, earlier run with the old fuse; wall-clock should match because the extra rounding is register-only): | | denoise | e2e | step | |---|---|---|---| | INT8+FA, no fuse | 276.1 s | 294.7 s | 13.80 s | | + fused SwiGLU | 271.7 s | 290.3 s | 13.58 s | | delta | **−4.4 s** | **−4.4 s** | **−0.22 s** | ## Test plan ```bash pytest tests/test_int8_input_act.py -k swiglu ``` CUDA+bf16 `int8_linear(x, input_act="swiglu")` must `torch.equal` the eager chain. FP16/BF16 fused INT8 codes must equal `quantize(silu(gate)*up)`. ## Files - `comfy_kitchen/backends/cuda/ops/int8_linear.cu` — CUDA quantizer - `comfy_kitchen/backends/hip/hadamard.h` — HIP mirror - `tests/test_int8_input_act.py` — require bit-exact on the fused CUDA path --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-26 22:20:59 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#159

No description provided.

Rows
Columns