[GH-ISSUE #78] ROCm/gfx1201 INT8 ConvRot: eager fallback is very slow, global Triton crashes, selective Triton int8_linear works #27

New issue

Closed

opened 2026-10-26 22:19:18 +00:00 by bsakaguchi · 10 comments

bsakaguchi commented 2026-10-26 22:19:18 +00:00

Owner

Copy link

Originally created by @flexusjan on GitHub (Jul 18, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/78

ROCm/gfx1201 INT8 ConvRot: eager fallback is very slow, global Triton crashes, selective Triton int8_linear works

Summary

On an AMD Radeon RX 9070 XT (gfx1201) with ROCm 7.2, INT8 ConvRot models work only through the safe eager fallback, but the fallback is much slower than FP8. Enabling the global comfy-kitchen Triton backend crashes the process with a ROCm GPU memory access fault.

However, a narrow workaround that keeps comfy-kitchen cuda and triton disabled globally, but routes only ConvRot int8_linear from eager to comfy_kitchen.backends.triton.quantization.int8_linear, is stable and much faster on this machine.

This looks like the ROCm INT8 ConvRot path needs more granular backend selection: Triton int8_linear appears usable, while global Triton/RoPE and the CUDA/nanobind backend paths are not safe on this setup.

I searched for related reports first. The closest ComfyUI issues I found are related but not exact:

• https://github.com/Comfy-Org/ComfyUI/issues/14722
• https://github.com/Comfy-Org/ComfyUI/issues/13730

Environment

• OS: Arch Linux
• Kernel: Linux archlinux 7.1.3-arch2-2 #1 SMP PREEMPT_DYNAMIC Thu, 16 Jul 2026 17:41:28 +0000 x86_64 GNU/Linux
• GPU: AMD Radeon RX 9070 XT
• GPU arch: gfx1201
• rocm-smi driver version: 7.1.3-arch2-2
• VRAM: 16 GB
• ComfyUI: 0.28.0
• ComfyUI commit: 4800e78518ebb1f2a9443ea5418edbff6c3935f9
• Python: 3.13.14
• PyTorch: 2.12.1+rocm7.2
• ROCm/HIP: 7.2.53211
• torch.version.cuda: None
• Triton ROCm: 3.7.1
• comfy-kitchen: 0.2.22
• Launch args:

main.py --use-pytorch-cross-attention --reserve-vram 1.5 --vram-headroom 0 --disable-pinned-memory --enable-manager

ComfyUI reports the device as:

cuda:0 AMD Radeon RX 9070 XT : native

Model / workflow tested

The repro workflow is a Krea2 1 MP portrait workflow using:

• UNET/diffusion model: Krea2_Turbo_convrot_int8mixed.safetensors
• CLIP/text encoder: qwen3vl_4b_uncensored_int8_convrot.safetensors
• VAE: qwen_image_vae.safetensors
• Sampler: euler, simple
• Steps: 8
• CFG: 1
• Denoise: 1
• Resolution selector: 2:3 (Portrait Photo), 1 MP

The full ComfyUI workflow uses a few custom nodes, but the backend failures below can also be reproduced with small standalone comfy-kitchen tensor calls. The custom nodes do not appear to be the root cause.

Expected behavior

On ROCm/gfx1201, comfy-kitchen should be able to use the fast INT8 ConvRot linear path without enabling unsafe global Triton/RoPE paths and without attempting the CUDA/nanobind backend path that currently fails on ROCm.

Ideally the backend dispatcher would support a narrower capability selection such as:

• keep cuda backend disabled when torch.version.cuda is None
• keep global Triton disabled or conservative on ROCm
• allow Triton only for int8_linear when:
  • torch.version.hip is not None
  • arch is a supported matrix-core AMD target, e.g. gfx11xx/gfx12xx
  • input is ConvRot INT8 linear

Minimal standalone repros

CUDA backend on ROCm: PyCapsule/nanobind argument error

This reproduces the CUDA backend binding problem without a ComfyUI workflow:

import torch
import comfy_kitchen as ck
from comfy_kitchen.registry import registry

print("torch", torch.__version__, "hip", torch.version.hip, "cuda", torch.version.cuda)
print(torch.cuda.get_device_name(0), torch.cuda.get_device_properties(0).gcnArchName)
print(ck.list_backends())

w = torch.randn((128, 256), device="cuda", dtype=torch.bfloat16)
impl = registry.get_implementation(
    "quantize_int8_convrot_weight",
    kwargs={"weight": w, "group_size": 256, "stochastic_rounding": 0},
)
print("impl", impl.__module__, impl.__name__)
q, s = impl(w, 256, stochastic_rounding=0)
torch.cuda.synchronize()

On this system that selects:

impl comfy_kitchen.backends.cuda quantize_int8_convrot_weight

and fails with:

TypeError: quantize_int8_rowwise_convrot64(): incompatible function arguments.
...
Invoked with types: PyCapsule, PyCapsule, PyCapsule, int, bool, int, int

Selective Triton int8_linear works if weights are quantized through eager

This isolates the working fast path:

import time
import torch
from comfy_kitchen.backends.eager.quantization import quantize_int8_convrot_weight
from comfy_kitchen.backends.triton.quantization import int8_linear

torch.manual_seed(2)

for m, k, n in [(1, 4096, 4096), (256, 4096, 4096), (1024, 4096, 4096)]:
    print("shape", (m, k, n), flush=True)
    x = torch.randn((m, k), device="cuda", dtype=torch.bfloat16)
    w = torch.randn((n, k), device="cuda", dtype=torch.bfloat16)
    q, s = quantize_int8_convrot_weight(w, 256, 0)
    torch.cuda.synchronize()

    y = int8_linear(x, q, s, None, torch.bfloat16, True, 256)
    torch.cuda.synchronize()

    t0 = time.perf_counter()
    for _ in range(5):
        y = int8_linear(x, q, s, None, torch.bfloat16, True, 256)
    torch.cuda.synchronize()
    print("avg_ms", (time.perf_counter() - t0) * 1000 / 5, "out", y.shape, y.dtype)

Observed output:

shape (1, 4096, 4096)
avg_ms 0.1248 out torch.Size([1, 4096]) torch.bfloat16

shape (256, 4096, 4096)
avg_ms 0.1718 out torch.Size([256, 4096]) torch.bfloat16

shape (1024, 4096, 4096)
avg_ms 0.2732 out torch.Size([1024, 4096]) torch.bfloat16

Actual behavior

1. Clean safe fallback is stable but too slow

With the default safe behavior where comfy-kitchen cuda and triton backends are disabled on ROCm, the ConvRot model is stable but very slow:

Prompt executed in ~65-80 seconds

For comparison, an FP8 Krea2 model in the same workflow is around:

Prompt executed in ~12.7-13.0 seconds

2. Enabling the global Triton backend crashes

Starting ComfyUI with global Triton enabled for comfy-kitchen causes a process crash during the INT8 ConvRot workflow:

Memory access fault by GPU node-1 ...
Fatal Python error: Aborted

This is not just a Python exception; the Python process aborts.

3. Enabling/forcing the comfy-kitchen CUDA backend on ROCm hits nanobind/PyCapsule errors

If the comfy-kitchen CUDA backend is allowed/forced on ROCm, backend functions fail with argument binding errors like:

TypeError: quantize_int8_rowwise_convrot64(): incompatible function arguments.
The following argument types are supported:
    1. quantize_int8_rowwise_convrot64(input: ndarray[shape=(*, *), device='cuda'], output: ndarray[dtype=int8, shape=(*, *), device='cuda'], scales: ndarray[dtype=float32, shape=(*, *), device='cuda'], group_size: int, stochastic: bool, seed: int, stream_ptr: int) -> None

Invoked with types: PyCapsule, PyCapsule, PyCapsule, int, bool, int, int

Related failures were also seen in other CUDA backend functions:

dequantize_int8_convrot_weight(): incompatible function arguments ... Invoked with types: PyCapsule ...
quantize_int8_rowwise_convrot64(): incompatible function arguments ...
apply_rope(): incompatible function arguments ...
stochastic_round_fp8(): incompatible function arguments ...

Workaround found

The following narrow patch was tested locally in comfy_kitchen/backends/eager/quantization.py.

It leaves comfy-kitchen cuda and triton disabled globally, but for ROCm ConvRot int8_linear only, it directly calls the Triton int8_linear implementation:

def int8_linear(
    x: torch.Tensor,
    weight: torch.Tensor,
    weight_scale: torch.Tensor,
    bias: torch.Tensor | None = None,
    out_dtype: torch.dtype = torch.bfloat16,
    convrot: bool = False,
    convrot_groupsize: int = 256,
) -> torch.Tensor:
    if convrot and x.is_cuda and torch.version.hip is not None:
        try:
            from comfy_kitchen.backends.triton.quantization import int8_linear as triton_int8_linear
        except ImportError:
            triton_int8_linear = None
        if triton_int8_linear is not None:
            return triton_int8_linear(
                x,
                weight,
                weight_scale,
                bias=bias,
                out_dtype=out_dtype,
                convrot=convrot,
                convrot_groupsize=convrot_groupsize,
            )

    ...

This is not meant as a final upstream patch because it bypasses the backend registry, but it demonstrates that the Triton INT8 linear path itself can work on this ROCm/gfx1201 system if selected narrowly.

Results with workaround

Same workflow, same model, same settings:

First/warmup run: Prompt executed in 30.52 seconds
Hot run 1:        Prompt executed in 10.49 seconds
Hot run 2:        Prompt executed in 10.54 seconds

Sampler progress for hot runs was around 7.9s; the rest appears to be model/VAE memory movement under --reserve-vram 1.5.

So the selective Triton int8_linear workaround is:

• stable in repeated workflow runs
• faster than FP8 on this machine
• much faster than the eager ConvRot fallback
• avoids the crash seen with global Triton
• avoids the PyCapsule/nanobind errors from the CUDA backend on ROCm

Notes

This may overlap with native INT8 ConvRot support work in ComfyUI, but this report is specifically about current comfy-kitchen backend behavior on ROCm/gfx1201:

• CUDA backend registers but its nanobind functions do not accept the wrapped arguments on ROCm.
• Global Triton can crash the process.
• The Triton int8_linear implementation itself appears usable and fast when called narrowly.

Originally created by @flexusjan on GitHub (Jul 18, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/78 # ROCm/gfx1201 INT8 ConvRot: eager fallback is very slow, global Triton crashes, selective Triton `int8_linear` works ## Summary On an AMD Radeon RX 9070 XT (`gfx1201`) with ROCm 7.2, INT8 ConvRot models work only through the safe eager fallback, but the fallback is much slower than FP8. Enabling the global comfy-kitchen Triton backend crashes the process with a ROCm GPU memory access fault. However, a narrow workaround that keeps comfy-kitchen `cuda` and `triton` disabled globally, but routes only ConvRot `int8_linear` from eager to `comfy_kitchen.backends.triton.quantization.int8_linear`, is stable and much faster on this machine. This looks like the ROCm INT8 ConvRot path needs more granular backend selection: Triton `int8_linear` appears usable, while global Triton/RoPE and the CUDA/nanobind backend paths are not safe on this setup. I searched for related reports first. The closest ComfyUI issues I found are related but not exact: - https://github.com/Comfy-Org/ComfyUI/issues/14722 - https://github.com/Comfy-Org/ComfyUI/issues/13730 ## Environment - OS: Arch Linux - Kernel: `Linux archlinux 7.1.3-arch2-2 #1 SMP PREEMPT_DYNAMIC Thu, 16 Jul 2026 17:41:28 +0000 x86_64 GNU/Linux` - GPU: AMD Radeon RX 9070 XT - GPU arch: `gfx1201` - rocm-smi driver version: `7.1.3-arch2-2` - VRAM: 16 GB - ComfyUI: `0.28.0` - ComfyUI commit: `4800e78518ebb1f2a9443ea5418edbff6c3935f9` - Python: `3.13.14` - PyTorch: `2.12.1+rocm7.2` - ROCm/HIP: `7.2.53211` - `torch.version.cuda`: `None` - Triton ROCm: `3.7.1` - comfy-kitchen: `0.2.22` - Launch args: ```text main.py --use-pytorch-cross-attention --reserve-vram 1.5 --vram-headroom 0 --disable-pinned-memory --enable-manager ``` ComfyUI reports the device as: ```text cuda:0 AMD Radeon RX 9070 XT : native ``` ## Model / workflow tested The repro workflow is a Krea2 1 MP portrait workflow using: - UNET/diffusion model: `Krea2_Turbo_convrot_int8mixed.safetensors` - CLIP/text encoder: `qwen3vl_4b_uncensored_int8_convrot.safetensors` - VAE: `qwen_image_vae.safetensors` - Sampler: `euler`, `simple` - Steps: `8` - CFG: `1` - Denoise: `1` - Resolution selector: `2:3 (Portrait Photo)`, `1 MP` The full ComfyUI workflow uses a few custom nodes, but the backend failures below can also be reproduced with small standalone comfy-kitchen tensor calls. The custom nodes do not appear to be the root cause. ## Expected behavior On ROCm/gfx1201, comfy-kitchen should be able to use the fast INT8 ConvRot linear path without enabling unsafe global Triton/RoPE paths and without attempting the CUDA/nanobind backend path that currently fails on ROCm. Ideally the backend dispatcher would support a narrower capability selection such as: - keep `cuda` backend disabled when `torch.version.cuda is None` - keep global Triton disabled or conservative on ROCm - allow Triton only for `int8_linear` when: - `torch.version.hip is not None` - arch is a supported matrix-core AMD target, e.g. `gfx11xx`/`gfx12xx` - input is ConvRot INT8 linear ## Minimal standalone repros ### CUDA backend on ROCm: PyCapsule/nanobind argument error This reproduces the CUDA backend binding problem without a ComfyUI workflow: ```python import torch import comfy_kitchen as ck from comfy_kitchen.registry import registry print("torch", torch.__version__, "hip", torch.version.hip, "cuda", torch.version.cuda) print(torch.cuda.get_device_name(0), torch.cuda.get_device_properties(0).gcnArchName) print(ck.list_backends()) w = torch.randn((128, 256), device="cuda", dtype=torch.bfloat16) impl = registry.get_implementation( "quantize_int8_convrot_weight", kwargs={"weight": w, "group_size": 256, "stochastic_rounding": 0}, ) print("impl", impl.__module__, impl.__name__) q, s = impl(w, 256, stochastic_rounding=0) torch.cuda.synchronize() ``` On this system that selects: ```text impl comfy_kitchen.backends.cuda quantize_int8_convrot_weight ``` and fails with: ```text TypeError: quantize_int8_rowwise_convrot64(): incompatible function arguments. ... Invoked with types: PyCapsule, PyCapsule, PyCapsule, int, bool, int, int ``` ### Selective Triton `int8_linear` works if weights are quantized through eager This isolates the working fast path: ```python import time import torch from comfy_kitchen.backends.eager.quantization import quantize_int8_convrot_weight from comfy_kitchen.backends.triton.quantization import int8_linear torch.manual_seed(2) for m, k, n in [(1, 4096, 4096), (256, 4096, 4096), (1024, 4096, 4096)]: print("shape", (m, k, n), flush=True) x = torch.randn((m, k), device="cuda", dtype=torch.bfloat16) w = torch.randn((n, k), device="cuda", dtype=torch.bfloat16) q, s = quantize_int8_convrot_weight(w, 256, 0) torch.cuda.synchronize() y = int8_linear(x, q, s, None, torch.bfloat16, True, 256) torch.cuda.synchronize() t0 = time.perf_counter() for _ in range(5): y = int8_linear(x, q, s, None, torch.bfloat16, True, 256) torch.cuda.synchronize() print("avg_ms", (time.perf_counter() - t0) * 1000 / 5, "out", y.shape, y.dtype) ``` Observed output: ```text shape (1, 4096, 4096) avg_ms 0.1248 out torch.Size([1, 4096]) torch.bfloat16 shape (256, 4096, 4096) avg_ms 0.1718 out torch.Size([256, 4096]) torch.bfloat16 shape (1024, 4096, 4096) avg_ms 0.2732 out torch.Size([1024, 4096]) torch.bfloat16 ``` ## Actual behavior ### 1. Clean safe fallback is stable but too slow With the default safe behavior where comfy-kitchen `cuda` and `triton` backends are disabled on ROCm, the ConvRot model is stable but very slow: ```text Prompt executed in ~65-80 seconds ``` For comparison, an FP8 Krea2 model in the same workflow is around: ```text Prompt executed in ~12.7-13.0 seconds ``` ### 2. Enabling the global Triton backend crashes Starting ComfyUI with global Triton enabled for comfy-kitchen causes a process crash during the INT8 ConvRot workflow: ```text Memory access fault by GPU node-1 ... Fatal Python error: Aborted ``` This is not just a Python exception; the Python process aborts. ### 3. Enabling/forcing the comfy-kitchen CUDA backend on ROCm hits nanobind/PyCapsule errors If the comfy-kitchen CUDA backend is allowed/forced on ROCm, backend functions fail with argument binding errors like: ```text TypeError: quantize_int8_rowwise_convrot64(): incompatible function arguments. The following argument types are supported: 1. quantize_int8_rowwise_convrot64(input: ndarray[shape=(*, *), device='cuda'], output: ndarray[dtype=int8, shape=(*, *), device='cuda'], scales: ndarray[dtype=float32, shape=(*, *), device='cuda'], group_size: int, stochastic: bool, seed: int, stream_ptr: int) -> None Invoked with types: PyCapsule, PyCapsule, PyCapsule, int, bool, int, int ``` Related failures were also seen in other CUDA backend functions: ```text dequantize_int8_convrot_weight(): incompatible function arguments ... Invoked with types: PyCapsule ... quantize_int8_rowwise_convrot64(): incompatible function arguments ... apply_rope(): incompatible function arguments ... stochastic_round_fp8(): incompatible function arguments ... ``` ## Workaround found The following narrow patch was tested locally in `comfy_kitchen/backends/eager/quantization.py`. It leaves comfy-kitchen `cuda` and `triton` disabled globally, but for ROCm ConvRot `int8_linear` only, it directly calls the Triton `int8_linear` implementation: ```python def int8_linear( x: torch.Tensor, weight: torch.Tensor, weight_scale: torch.Tensor, bias: torch.Tensor | None = None, out_dtype: torch.dtype = torch.bfloat16, convrot: bool = False, convrot_groupsize: int = 256, ) -> torch.Tensor: if convrot and x.is_cuda and torch.version.hip is not None: try: from comfy_kitchen.backends.triton.quantization import int8_linear as triton_int8_linear except ImportError: triton_int8_linear = None if triton_int8_linear is not None: return triton_int8_linear( x, weight, weight_scale, bias=bias, out_dtype=out_dtype, convrot=convrot, convrot_groupsize=convrot_groupsize, ) ... ``` This is not meant as a final upstream patch because it bypasses the backend registry, but it demonstrates that the Triton INT8 linear path itself can work on this ROCm/gfx1201 system if selected narrowly. ## Results with workaround Same workflow, same model, same settings: ```text First/warmup run: Prompt executed in 30.52 seconds Hot run 1: Prompt executed in 10.49 seconds Hot run 2: Prompt executed in 10.54 seconds ``` Sampler progress for hot runs was around `7.9s`; the rest appears to be model/VAE memory movement under `--reserve-vram 1.5`. So the selective Triton `int8_linear` workaround is: - stable in repeated workflow runs - faster than FP8 on this machine - much faster than the eager ConvRot fallback - avoids the crash seen with global Triton - avoids the PyCapsule/nanobind errors from the CUDA backend on ROCm ## Notes This may overlap with native INT8 ConvRot support work in ComfyUI, but this report is specifically about current comfy-kitchen backend behavior on ROCm/gfx1201: - CUDA backend registers but its nanobind functions do not accept the wrapped arguments on ROCm. - Global Triton can crash the process. - The Triton `int8_linear` implementation itself appears usable and fast when called narrowly.

bsakaguchi closed this issue 2026-10-26 22:19:18 +00:00

bsakaguchi commented 2026-10-26 22:19:19 +00:00

Author

Owner

Copy link

@coderabbitai[bot] commented on GitHub (Jul 18, 2026):

⚠️ Possible Duplicate Issue(s)

• https://github.com/Comfy-Org/comfy-kitchen/issues/64

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

<!-- gh-comment-id:5010868484 --> @coderabbitai[bot] commented on GitHub (Jul 18, 2026): <!-- This is an auto-generated issue plan by CodeRabbit --> <details> <summary>⚠️ Possible Duplicate Issue(s)</summary> - https://github.com/Comfy-Org/comfy-kitchen/issues/64 </details> --- <details> <summary> 🧪 Issue enrichment is currently in open beta.</summary> You can configure auto-planning by selecting labels in the issue_enrichment configuration. To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`: ```yaml issue_enrichment: auto_enrich: enabled: false ``` </details> 💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit)!

bsakaguchi commented 2026-10-26 22:19:19 +00:00

Author

Owner

Copy link

@0xDELUXA commented on GitHub (Jul 18, 2026):

I have a genuine question. Why are you using --use-pytorch-cross-attention on RDNA4? We have SageAttention v1, FlashAttention 2 via Triton, and even SageAttention v2 implemented with WMMA. They all perform significantly better.

<!-- gh-comment-id:5011027707 --> @0xDELUXA commented on GitHub (Jul 18, 2026): I have a genuine question. Why are you using `--use-pytorch-cross-attention` on RDNA4? We have SageAttention v1, FlashAttention 2 via Triton, and even SageAttention v2 implemented with WMMA. They all perform significantly better.

bsakaguchi commented 2026-10-26 22:19:19 +00:00

Author

Owner

Copy link

@Jules-A commented on GitHub (Jul 18, 2026):

We have SageAttention v1, FlashAttention 2 via Triton, and even SageAttention v2 implemented with WMMA.

I didn't have much luck with SageAttention v1 and found better results with FlashAttention v2 (at least in the ROCm fork), how do you get SageAttention v2 working and if so how come that's not available by default in that fork?

<!-- gh-comment-id:5012830190 --> @Jules-A commented on GitHub (Jul 18, 2026): > We have SageAttention v1, FlashAttention 2 via Triton, and even SageAttention v2 implemented with WMMA. I didn't have much luck with SageAttention v1 and found better results with FlashAttention v2 (at least in the ROCm fork), how do you get SageAttention v2 working and if so how come that's not available by default in that fork?

bsakaguchi commented 2026-10-26 22:19:20 +00:00

Author

Owner

Copy link

@0xDELUXA commented on GitHub (Jul 18, 2026):

Jam opened a proper Sage2 PR with WMMA kernels for RDNA4: https://github.com/thu-ml/SageAttention/pull/368.
There are also different Sage2 forks made by various community members that only have Triton backends. Based on my experience, they're not really worth using on RDNA4.
I'm not sure what you mean by "... and if so how come that's not available by default in that fork?" Which one are you referring to, ROCm/flash-attention? They're different projects.

<!-- gh-comment-id:5012897096 --> @0xDELUXA commented on GitHub (Jul 18, 2026): Jam opened a proper Sage2 PR with WMMA kernels for RDNA4: https://github.com/thu-ml/SageAttention/pull/368. There are also different Sage2 forks made by various community members that only have Triton backends. Based on my experience, they're not really worth using on RDNA4. I'm not sure what you mean by "... and if so how come that's not available by default in that fork?" Which one are you referring to, [ROCm/flash-attention](https://github.com/ROCm/flash-attention)? They're different projects.

bsakaguchi commented 2026-10-26 22:19:21 +00:00

Author

Owner

Copy link

@Jules-A commented on GitHub (Jul 18, 2026):

I'm not sure what you mean by "... and if so how come that's not available by default in that fork?"

This fork of ComfyUI: https://github.com/patientx-cfz/comfyui-rocm

<!-- gh-comment-id:5013097285 --> @Jules-A commented on GitHub (Jul 18, 2026): > I'm not sure what you mean by "... and if so how come that's not available by default in that fork?" This fork of ComfyUI: https://github.com/patientx-cfz/comfyui-rocm

bsakaguchi commented 2026-10-26 22:19:21 +00:00

Author

Owner

Copy link

@0xDELUXA commented on GitHub (Jul 18, 2026):

Sage2 has to be built from source, and we don't plan to include things like this in comfyui-rocm.

<!-- gh-comment-id:5013279301 --> @0xDELUXA commented on GitHub (Jul 18, 2026): Sage2 has to be built from source, and we don't plan to include things like this in `comfyui-rocm`.

bsakaguchi commented 2026-10-26 22:19:21 +00:00

Author

Owner

Copy link

@flexusjan commented on GitHub (Jul 19, 2026):

@0xDELUXA thanks for pointing this out. I re-tested the attention backends on the same Krea2 Turbo INT8 ConvRot workflow instead of assuming --use-pytorch-cross-attention was the best choice.

Setup:

• GPU: RX 9070 XT / gfx1201
• ROCm / PyTorch: torch 2.12.1+rocm7.2
• ComfyUI: 0.28.0
• comfy-kitchen: 0.2.22
• Model: Krea2_Turbo_convrot_int8mixed.safetensors
• Shared ComfyUI args: --reserve-vram 1.5 --disable-pinned-memory --enable-manager
• Each mode: 1 warmup + 6 measured hot runs
• Metrics sampled via rocm-smi

Results:

Attention mode	Startup flag / env	API wall avg	Comfy prompt avg	GPU avg	VRAM avg	VRAM max
SageAttention v2 / gfx12 WMMA	--use-sage-attention	10.20s	9.92s	93.5%	14.99 GB	15.74 GB
FlashAttention 2 via Triton AMD	FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE --use-flash-attention	10.53s	10.23s	94.3%	15.07 GB	15.93 GB
PyTorch cross attention	--use-pytorch-cross-attention	10.89s	10.60s	92.6%	15.04 GB	16.00 GB

So yes, that was a useful pointer for RDNA4. In this workflow, SageAttention v2 built from the gfx12/WMMA PR is the fastest option I tested, and FlashAttention 2 is also faster than PyTorch cross attention.

The original comfy-kitchen INT8 ConvRot issue still remains separate from the attention backend choice: without the local ConvRot workaround, this workflow was much slower on ROCm. With the workaround plus SageAttention v2, INT8 ConvRot is now faster than my previous FP8 baseline.

<!-- gh-comment-id:5013518702 --> @flexusjan commented on GitHub (Jul 19, 2026): @0xDELUXA thanks for pointing this out. I re-tested the attention backends on the same Krea2 Turbo INT8 ConvRot workflow instead of assuming `--use-pytorch-cross-attention` was the best choice. Setup: - GPU: RX 9070 XT / gfx1201 - ROCm / PyTorch: `torch 2.12.1+rocm7.2` - ComfyUI: `0.28.0` - comfy-kitchen: `0.2.22` - Model: `Krea2_Turbo_convrot_int8mixed.safetensors` - Shared ComfyUI args: `--reserve-vram 1.5 --disable-pinned-memory --enable-manager` - Each mode: 1 warmup + 6 measured hot runs - Metrics sampled via `rocm-smi` Results: | Attention mode | Startup flag / env | API wall avg | Comfy prompt avg | GPU avg | VRAM avg | VRAM max | |---|---|---:|---:|---:|---:|---:| | SageAttention v2 / gfx12 WMMA | `--use-sage-attention` | 10.20s | 9.92s | 93.5% | 14.99 GB | 15.74 GB | | FlashAttention 2 via Triton AMD | `FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE --use-flash-attention` | 10.53s | 10.23s | 94.3% | 15.07 GB | 15.93 GB | | PyTorch cross attention | `--use-pytorch-cross-attention` | 10.89s | 10.60s | 92.6% | 15.04 GB | 16.00 GB | So yes, that was a useful pointer for RDNA4. In this workflow, SageAttention v2 built from the gfx12/WMMA PR is the fastest option I tested, and FlashAttention 2 is also faster than PyTorch cross attention. The original comfy-kitchen INT8 ConvRot issue still remains separate from the attention backend choice: without the local ConvRot workaround, this workflow was much slower on ROCm. With the workaround plus SageAttention v2, INT8 ConvRot is now faster than my previous FP8 baseline.

bsakaguchi commented 2026-10-26 22:19:21 +00:00

Author

Owner

Copy link

@flexusjan commented on GitHub (Aug 3, 2026):

Closing this as resolved by the official HIP backend work that has since landed in comfy-kitchen.

I re-tested the original ROCm/gfx1201 INT8 ConvRot scenario on comfy-kitchen 0.2.26 with ComfyUI 0.30.0 and torch 2.12.1+rocm7.2. With the ROCm visible device set restricted to the RX 9070 XT, the HIP backend advertises the relevant native capabilities including int8_linear and ConvRot support, and the Krea2 INT8 ConvRot workflow is back in the expected performance range:

• cold: ~28.2s Comfy prompt time / ~28.7s API wall time
• warm: ~10.5s Comfy prompt time / ~11.0s API wall time

The slow path I reproduced after updating was caused by the process exposing both gfx1201 and gfx1036. In that mixed visible-device configuration, comfy-kitchen correctly registers HIP as elementwise-only because not every visible device has WMMA, so int8_linear can fall back to eager when Triton/CUDA are disabled by ComfyUI. Restricting ROCm visibility fixes that local setup issue, for example:

export HIP_VISIBLE_DEVICES=0
export ROCR_VISIBLE_DEVICES=0

I also closed #81 because it was only a pre-HIP-backend workaround. If there is still a desired upstream improvement here, I think it should be a separate issue around mixed ROCm visible-device handling or clearer diagnostics, not the old eager-to-Triton workaround.

<!-- gh-comment-id:5164841371 --> @flexusjan commented on GitHub (Aug 3, 2026): Closing this as resolved by the official HIP backend work that has since landed in comfy-kitchen. I re-tested the original ROCm/gfx1201 INT8 ConvRot scenario on comfy-kitchen 0.2.26 with ComfyUI 0.30.0 and torch 2.12.1+rocm7.2. With the ROCm visible device set restricted to the RX 9070 XT, the HIP backend advertises the relevant native capabilities including `int8_linear` and ConvRot support, and the Krea2 INT8 ConvRot workflow is back in the expected performance range: - cold: ~28.2s Comfy prompt time / ~28.7s API wall time - warm: ~10.5s Comfy prompt time / ~11.0s API wall time The slow path I reproduced after updating was caused by the process exposing both `gfx1201` and `gfx1036`. In that mixed visible-device configuration, comfy-kitchen correctly registers HIP as elementwise-only because not every visible device has WMMA, so `int8_linear` can fall back to eager when Triton/CUDA are disabled by ComfyUI. Restricting ROCm visibility fixes that local setup issue, for example: ```bash export HIP_VISIBLE_DEVICES=0 export ROCR_VISIBLE_DEVICES=0 ``` I also closed #81 because it was only a pre-HIP-backend workaround. If there is still a desired upstream improvement here, I think it should be a separate issue around mixed ROCm visible-device handling or clearer diagnostics, not the old eager-to-Triton workaround.

bsakaguchi commented 2026-10-26 22:19:21 +00:00

Author

Owner

Copy link

@alecz20 commented on GitHub (Aug 9, 2026):

I haven't seen this comparison here, so here it goes:

My anecdotal evidence shows 100% speed increase in int8_convort Sampling speed between comfy-kitchen 0.2.24 and 0.2.28 with Krea2_turbo. (RX 7900XT, 1024x1024 latent):

PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True" python main.py --enable-manager --disable-smart-memory

comfy-kitchen 0.2.24: 3.76 s/it
comfy-kitchen 0.2.28: 1.84 s/it

this is only slightly faster than fp8 which takes about 1.90s/it.

Good work!

<!-- gh-comment-id:5233170241 --> @alecz20 commented on GitHub (Aug 9, 2026): I haven't seen this comparison here, so here it goes: My anecdotal evidence shows **100%** speed increase in int8_convort Sampling speed between comfy-kitchen **0.2.24** and **0.2.28** with Krea2_turbo. (RX 7900XT, 1024x1024 latent): ``` PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True" python main.py --enable-manager --disable-smart-memory ``` comfy-kitchen 0.2.24: **3.76 s/it** comfy-kitchen 0.2.28: **1.84 s/it** this is only slightly faster than fp8 which takes about 1.90s/it. Good work!

bsakaguchi commented 2026-10-26 22:19:21 +00:00

Author

Owner

Copy link

@0xDELUXA commented on GitHub (Aug 9, 2026):

I haven't seen this comparison here, so here it goes:

My anecdotal evidence shows 100% speed increase in int8_convort Sampling speed between comfy-kitchen 0.2.24 and 0.2.28 with Krea2_turbo. (RX 7900XT, 1024x1024 latent):

PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True" python main.py --enable-manager --disable-smart-memory

comfy-kitchen 0.2.24: 3.76 s/it comfy-kitchen 0.2.28: 1.84 s/it

this is only slightly faster than fp8 which takes about 1.90s/it.

Good work!

Thanks! A lot has improved between 0.2.24 and 0.2.28, including the new HIP backend, so it’s not all down to that. Either way, great to see AMD getting better in ComfyUI! 🚀

<!-- gh-comment-id:5233596153 --> @0xDELUXA commented on GitHub (Aug 9, 2026): > I haven't seen this comparison here, so here it goes: > > My anecdotal evidence shows **100%** speed increase in int8_convort Sampling speed between comfy-kitchen **0.2.24** and **0.2.28** with Krea2_turbo. (RX 7900XT, 1024x1024 latent): > > ``` > PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True" python main.py --enable-manager --disable-smart-memory > ``` > > comfy-kitchen 0.2.24: **3.76 s/it** comfy-kitchen 0.2.28: **1.84 s/it** > > this is only slightly faster than fp8 which takes about 1.90s/it. > > Good work! Thanks! A lot has improved between 0.2.24 and 0.2.28, including the new HIP backend, so it’s not all down to that. Either way, great to see AMD getting better in ComfyUI! 🚀

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

dev/adaptive-w3w5

dev/w3a8-gateup

dev/relayout-combine

dev/hip-residual-inplace

dev/qwen-adaptive-tree-pr

dev/h3-attn-layout-ci-ccache

dev/h3-attn-layout

dev/h3-quantizer-perf

dev/amd-resident-linear-fusions

dev/ada-unified-tree

dev/flash-decode-review

dev/qwen-decode-series

dev/bf16-delta-state

dev/int8-kv-decode

dev/qwen38-w6a8

matt/be-3580-agents-md

chore/node-runtime-refresh-2026-09-28

dev/cache-registry

feature/147-sageattention-wheels

fix/cla-api-failure-handling

docs/torch-min-version-2.7

alexisrolland-patch-2

alexisrolland-patch-1

dev/optim_int4

add-cla-workflow

dev/hip1

refactor-1

v0.2.40

v0.2.39

v0.2.38

v0.2.37

v0.2.36

v0.2.35

v0.2.34

v0.2.33

v0.2.32

v0.2.31

v0.2.30

v0.2.29

v0.2.28

v0.2.27

v0.2.26

v0.2.25

v0.2.24

v0.2.23

v0.2.22

v0.2.21

v0.2.20

v0.2.19

v0.2.18

v0.2.17

v0.2.16

v0.2.15

v0.2.14

v0.2.13

v0.2.12

v0.2.11

v0.2.10

v0.2.9

v0.2.8

v0.2.7

v0.2.6

v0.2.5

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.6

v0.1.5

v0.1.4

v0.1.3

v0.1.2

v0.1.1

v0.1.0

Labels

Clear labels   

pull-request

Mirrored from GitHub Pull Request

No labels

pull-request

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#27

No description provided.