[GH-ISSUE #55] INT8 Krea2 models broken on Tesla T4 (SM75): CUDA invalid argument or corrupted outputs

bsakaguchi commented

2026-10-26 22:19:12 +00:00

Owner

Originally created by @PhoenixCreation on GitHub (Jun 26, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/55

Summary

Native INT8 Krea2 models do not work correctly on Tesla T4 (SM75).

Two different failure modes occur:

Krea2_Turbo_convrot_int8mixed.safetensors

crashes with:

RuntimeError: CUDA INT8 rowwise quantization failed: invalid argument

or

RuntimeError: CUDA INT8 linear dequantization failed: invalid argument

Krea2_Turbo_int8mixed.safetensors
- does not consistently crash but generates deterministic tiled psychedelic noise instead of a valid image.

FP8, NF4 and GGUF versions of Krea2 work correctly on the same system.

Environment

GPU:

Tesla T4
Compute capability: SM75

PyTorch:

2.12.1+cu130

ComfyUI:

latest (github.com/Comfy-Org/ComfyUI@7cb784e0f4)

Comfy Kitchen:

0.2.12

CUDA architectures:

>>> torch.cuda.get_arch_list()
['sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120']

Models tested

Models from https://huggingface.co/Winnougan/Krea-2-Base-Turbo-NVFP4-FP8-INT8

Fails with CUDA error:

Krea2_Turbo_convrot_int8mixed.safetensors

Produces corrupted tiled output:

Krea2_Turbo_int8mixed.safetensors

Works correctly:

Krea2_Turbo_fp8.safetensors

Workflow settings

Sampler:

sampler: er_sde
scheduler: simple
steps: 8
cfg: 0
denoise: 1.0

Resolution tested:

1280x720

These settings follow the recommended Krea2 Turbo settings.

Additional tests performed

Disabled MultiGPU:

no change

Enabled Triton backend:

comfy launch -- --enable-triton-backend

no change

Disabled CUTLASS:

export COMFY_KITCHEN_DISABLE_CUTLASS=1

no change

Expected behavior

INT8 Krea2 models should either:

Generate correct images on Tesla T4 (SM75), or
Explicitly report that SM75 is unsupported.

Actual behavior

ConvRot INT8 model crashes.
Standard INT8 model produces corrupted tiled outputs.

This might be an SM75-specific issue in the native INT8 backend.

Originally created by @PhoenixCreation on GitHub (Jun 26, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/55 ## Summary Native INT8 Krea2 models do not work correctly on Tesla T4 (SM75). Two different failure modes occur: 1. `Krea2_Turbo_convrot_int8mixed.safetensors` * crashes with: ``` RuntimeError: CUDA INT8 rowwise quantization failed: invalid argument ``` or ``` RuntimeError: CUDA INT8 linear dequantization failed: invalid argument ``` 2. `Krea2_Turbo_int8mixed.safetensors` * does not consistently crash but generates deterministic tiled psychedelic noise instead of a valid image. FP8, NF4 and GGUF versions of Krea2 work correctly on the same system. --- ## Environment GPU: * Tesla T4 * Compute capability: SM75 PyTorch: * 2.12.1+cu130 ComfyUI: * latest (https://github.com/Comfy-Org/ComfyUI/commit/7cb784e0f48784bb6ed588912e186e5ee1e9ee68) Comfy Kitchen: * 0.2.12 CUDA architectures: ```python >>> torch.cuda.get_arch_list() ['sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120'] ``` --- ## Models tested Models from https://huggingface.co/Winnougan/Krea-2-Base-Turbo-NVFP4-FP8-INT8 Fails with CUDA error: * `Krea2_Turbo_convrot_int8mixed.safetensors` Produces corrupted tiled output: * `Krea2_Turbo_int8mixed.safetensors` Works correctly: * `Krea2_Turbo_fp8.safetensors` --- ## Workflow settings Sampler: * sampler: `er_sde` * scheduler: `simple` * steps: `8` * cfg: `0` * denoise: `1.0` Resolution tested: * 1280x720 These settings follow the recommended Krea2 Turbo settings. --- ## Additional tests performed Disabled MultiGPU: * no change Enabled Triton backend: ```bash comfy launch -- --enable-triton-backend ``` * no change Disabled CUTLASS: ```bash export COMFY_KITCHEN_DISABLE_CUTLASS=1 ``` * no change --- ## Expected behavior INT8 Krea2 models should either: 1. Generate correct images on Tesla T4 (SM75), or 2. Explicitly report that SM75 is unsupported. --- ## Actual behavior * ConvRot INT8 model crashes. * Standard INT8 model produces corrupted tiled outputs. This might be an SM75-specific issue in the native INT8 backend.

bsakaguchi closed this issue

2026-10-26 22:19:12 +00:00

bsakaguchi commented

2026-10-26 22:19:13 +00:00

Author

Owner

@coderabbitai[bot] commented on GitHub (Jun 26, 2026):

🔗 Related PRs

Comfy-Org/comfy-kitchen#53 - Add cutlass int8 kernel [merged]

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

@coderabbitai[bot] commented on GitHub (Jun 26, 2026):  <details> <summary>🔗 Related PRs</summary> Comfy-Org/comfy-kitchen#53 - Add cutlass int8 kernel [merged] </details> --- <details> <summary> 🧪 Issue enrichment is currently in open beta.</summary> You can configure auto-planning by selecting labels in the issue_enrichment configuration. To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`: ```yaml issue_enrichment: auto_enrich: enabled: false ``` </details> 💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit)!

bsakaguchi commented

2026-10-26 22:19:13 +00:00

Author

Owner

@sashasubbbb commented on GitHub (Jun 27, 2026):

Can confirm i'm also getting same error on RTX 2070 (SM75)

error.txt

@sashasubbbb commented on GitHub (Jun 27, 2026): Can confirm i'm also getting same error on RTX 2070 (SM75) [error.txt](https://github.com/user-attachments/files/29402534/error.txt)

bsakaguchi commented

2026-10-26 22:19:13 +00:00

Author

Owner

@comfyanonymous commented on GitHub (Jun 27, 2026):

Can you try these wheels and let me know if they work? https://github.com/Comfy-Org/comfy-kitchen/actions/runs/28279131999

@comfyanonymous commented on GitHub (Jun 27, 2026): Can you try these wheels and let me know if they work? https://github.com/Comfy-Org/comfy-kitchen/actions/runs/28279131999

bsakaguchi commented

2026-10-26 22:19:13 +00:00

Author

Owner

@PhoenixCreation commented on GitHub (Jun 27, 2026):

Hi @comfyanonymous, will try them in some time and will let you know

@PhoenixCreation commented on GitHub (Jun 27, 2026): Hi @comfyanonymous, will try them in some time and will let you know

bsakaguchi commented

2026-10-26 22:19:13 +00:00

Author

Owner

@PhoenixCreation commented on GitHub (Jun 27, 2026):

Hi @comfyanonymous, it is working with those wheels(tested with linux cp312 cuda wheel). Although two observation:

According to https://huggingface.co/Winnougan/Krea-2-Base-Turbo-NVFP4-FP8-INT8, CFG is supposed to 0 for turbo model but I needed to set it to 1, setting it 0 gave me garbage images. Should it be 1 or 0?
Speed comparision for 720x1280 image:

krea2_turbo_fp8.safetensors - 4.7 sec/it
Krea2_Turbo_convrot_int8mixed.safetensors - 3.7 sec/it
Krea2_Turbo_int8mixed.safetensors - 3.6 sec/it
These are not the speed gains that one might expect from int8 but they are there so no complaints.

Thanks for fixing this 🙇

@PhoenixCreation commented on GitHub (Jun 27, 2026): Hi @comfyanonymous, it is working with those wheels(tested with linux cp312 cuda wheel). Although two observation: 1. According to https://huggingface.co/Winnougan/Krea-2-Base-Turbo-NVFP4-FP8-INT8, CFG is supposed to 0 for turbo model but I needed to set it to 1, setting it 0 gave me garbage images. Should it be 1 or 0? 2. Speed comparision for 720x1280 image: - krea2_turbo_fp8.safetensors - 4.7 sec/it - Krea2_Turbo_convrot_int8mixed.safetensors - 3.7 sec/it - Krea2_Turbo_int8mixed.safetensors - 3.6 sec/it These are not the speed gains that one might expect from int8 but they are there so no complaints. Thanks for fixing this 🙇

bsakaguchi commented

2026-10-26 22:19:14 +00:00

Author

Owner

@Ph0rk0z commented on GitHub (Jun 27, 2026):

I am testing this right now.. not for krea but for klein. Triton is gated for sm_80. That's a huge regression from int8-fast. Half a second is lost and I have trouble disabling eager to check the cuda backend.

I removed the sm80 gate and the triton kernel ran successfully.. albeit a little slower than the previous node. 6.3s vs 5.9s. So why is the gate there?

@Ph0rk0z commented on GitHub (Jun 27, 2026): I am testing this right now.. not for krea but for klein. Triton is gated for sm_80. That's a huge regression from int8-fast. Half a second is lost and I have trouble disabling eager to check the cuda backend. **I removed the sm80 gate and the triton kernel ran successfully.. albeit a little slower than the previous node. 6.3s vs 5.9s. So why is the gate there?**

bsakaguchi commented

2026-10-26 22:19:14 +00:00

Author

Owner

@PhoenixCreation commented on GitHub (Jun 27, 2026):

Another update, tried the latest wheel with ideogram and it is working fine:

Standard comfy installation with int8 ideogram gives just random noise image
Standard comfy installation with fp8 ideogram was running at 60-70 seconds/it
With updated wheel, it is now running at 20 seconds/it with int8 models

I also tried the int8 from https://huggingface.co/silveroxides/ideogram4-dequant-and-int8-quant/tree/main and it is also working fine with new wheel.

Again, Thanks @comfyanonymous!

@PhoenixCreation commented on GitHub (Jun 27, 2026): Another update, tried the latest wheel with ideogram and it is working fine: - Standard comfy installation with int8 ideogram gives just random noise image - Standard comfy installation with fp8 ideogram was running at 60-70 seconds/it - With updated wheel, it is now running at 20 seconds/it with int8 models I also tried the int8 from https://huggingface.co/silveroxides/ideogram4-dequant-and-int8-quant/tree/main and it is also working fine with new wheel. Again, Thanks @comfyanonymous!

bsakaguchi commented

2026-10-26 22:19:14 +00:00

Author

Owner

@Ph0rk0z commented on GitHub (Jun 27, 2026):

5.9s is back after I add triton.Config({'block_m': 128, 'block_n': 128, 'block_k': 64, 'group_size_m': 8}, num_stages=2, num_warps=4),

[INFO] Native ops: int8_tensorwise, hybrid_mxfp8, float8_e4m3fn_rowwise, float8_e4m3fn_blockwise, int8_blockwise , emulated ops: float8_e4m3fn, nvfp4, mxfp8, float8_e5m2
[INFO] model weight dtype torch.float16, manual cast: torch.float16
[INFO] model_type FLUX
[INFO] gguf qtypes: Q6_K (37), F32 (145), Q4_K (217)
[MINIMAL] Dequantizing token_embd.weight to prevent runtime OOM.
[MultiGPU Core Patching] text_encoder_device_patched returning device: cuda:0 (current_text_encoder_device=cuda:0)
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load Flux2TEModel_
[INFO] loaded completely; 21002.17 MB usable, 6829.34 MB loaded, full load: True
[INFO] Requested to load Flux2
[INFO] loaded completely; 14564.04 MB usable, 8996.02 MB loaded, full load: True
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:07<00:00,  1.94s/it]
[INFO] Requested to load TAESD
[INFO] loaded completely; 4994.88 MB usable, 10.21 MB loaded, full load: True
[INFO] Prompt executed in 27.51 seconds
[INFO] got prompt
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:05<00:00,  1.43s/it]
[INFO] Prompt executed in 5.91 seconds

so once again

            default_devices=triton_devices,
            #min_compute_capability=(8, 0),  # Required for Triton INT8 dot

Gate is done for no reason. I guess now to check the cublas_LT kernel for speed.

update: the cublas_LT kernel is slower than cuda, eager and triton and it NaN because all I get is sprinkles.

@Ph0rk0z commented on GitHub (Jun 27, 2026): 5.9s is back after I add ` triton.Config({'block_m': 128, 'block_n': 128, 'block_k': 64, 'group_size_m': 8}, num_stages=2, num_warps=4),` ``` [INFO] Native ops: int8_tensorwise, hybrid_mxfp8, float8_e4m3fn_rowwise, float8_e4m3fn_blockwise, int8_blockwise , emulated ops: float8_e4m3fn, nvfp4, mxfp8, float8_e5m2 [INFO] model weight dtype torch.float16, manual cast: torch.float16 [INFO] model_type FLUX [INFO] gguf qtypes: Q6_K (37), F32 (145), Q4_K (217) [MINIMAL] Dequantizing token_embd.weight to prevent runtime OOM. [MultiGPU Core Patching] text_encoder_device_patched returning device: cuda:0 (current_text_encoder_device=cuda:0) [INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16 [INFO] Requested to load Flux2TEModel_ [INFO] loaded completely; 21002.17 MB usable, 6829.34 MB loaded, full load: True [INFO] Requested to load Flux2 [INFO] loaded completely; 14564.04 MB usable, 8996.02 MB loaded, full load: True 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:07<00:00, 1.94s/it] [INFO] Requested to load TAESD [INFO] loaded completely; 4994.88 MB usable, 10.21 MB loaded, full load: True [INFO] Prompt executed in 27.51 seconds [INFO] got prompt 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:05<00:00, 1.43s/it] [INFO] Prompt executed in 5.91 seconds ``` so once again ``` default_devices=triton_devices, #min_compute_capability=(8, 0), # Required for Triton INT8 dot ``` Gate is done for no reason. I guess now to check the cublas_LT kernel for speed. update: the cublas_LT kernel is slower than cuda, eager and triton and it NaN because all I get is sprinkles.

bsakaguchi commented

2026-10-26 22:19:14 +00:00

Author

Owner

@comfyanonymous commented on GitHub (Jun 30, 2026):

If you use latest comfyui turing will work with int8.

@comfyanonymous commented on GitHub (Jun 30, 2026): If you use latest comfyui turing will work with int8.

Rows
Columns

[GH-ISSUE #55] INT8 Krea2 models broken on Tesla T4 (SM75): CUDA invalid argument or corrupted outputs #19

Summary

Environment

Models tested

Workflow settings

Additional tests performed

Expected behavior

Actual behavior