Comfy-Org/comfy-kitchen

Fork 0

mirror of https://github.com/Comfy-Org/comfy-kitchen.git synced 2026-11-04 23:37:42 +00:00

[GH-ISSUE #196] INT8 ConvRot stalls on model re-entry on Windows/NVIDIA — 100% GPU at ~80W, high VRAM, sometimes OOM in int8_linear #58

New issue

Open

opened 2026-10-26 22:19:49 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 22:19:49 +00:00

Owner

Originally created by @Epervios on GitHub (Sep 24, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/196

video_wan_animate2.zip

Summary

INT8 ConvRot models on Windows/NVIDIA reproducibly enter a bad state during model re-entry / reload: the first inference segment completes normally, but a subsequent segment can stall indefinitely at Model Initializing with GPU utilization reported at 100%, clocks high, power collapsing to ~80 W, and VRAM remaining near saturation (~15.6–15.9 GB on a 16 GB card). In some runs the same path fails with an explicit OOM inside comfy_kitchen.backends.cuda.int8_linear while trying to allocate temporary buffers.

The issue reproduces with both:

wan_animate_2_int8_convrot.safetensors
wan_animate_2_distill_int8_convrot.safetensors

A non-ConvRot FP8 model does not reproduce the ~80 W stall pattern under the same ComfyUI environment, although it is not functionally equivalent to Wan Animate 2 and therefore is only a control for the execution path.

Environment

OS: Windows 11
GPU: NVIDIA GeForce RTX 4070 Ti SUPER, 16 GB VRAM
System RAM: 64 GB
ComfyUI: 0.37.0
Python: 3.13.14
PyTorch: 2.13.0+cu130
comfy-kitchen: 0.2.35
comfy-aimdo: 0.5.5
Device allocator: cudaMallocAsync
DynamicVRAM: enabled unless otherwise noted
Async weight offload: tested both enabled and disabled
Attention: PyTorch attention

Reproduction pattern

Using the Wan Animate 2 workflow with multiple sequential motion-transfer segments:

Start ComfyUI normally.
Load wan_animate_2_int8_convrot.safetensors or wan_animate_2_distill_int8_convrot.safetensors.
First segment completes normally.
VAE work runs and the WAN model is prepared again for the next segment.
Second/later segment may remain at:
0/6 [..., Model Initializing ...]
GPU telemetry transitions from normal compute (~250–265 W) to a persistent bad state around ~80–87 W while utilization remains 100%.

Example with distilled model:

[INFO] Model storage policy: fast_disk=True paths=['...wan_animate_2_distill_int8_convrot.safetensors']
[INFO] Requested to load WAN_Animate2
[INFO] Model WAN_Animate2 prepared for dynamic VRAM loading. 15878MB Staged. 1059 patches attached. Force pre-loaded 202 weights: 2008 KB.
100%|...| 6/6 [02:00<00:00, 20.06s/it]
...
[INFO] Model WAN_Animate2 prepared for dynamic VRAM loading. 15878MB Staged. 1059 patches attached. Force pre-loaded 202 weights: 2008 KB.
  0%|...| 0/6 [00:00<?, ?it/s, Model Initializing ...]

GPU telemetry signature

Immediately before the bad state:

23:51:40  ~263 W, ~15.7 GB VRAM
23:51:45  ~254 W, ~15.7 GB VRAM
23:51:46   86.99 W, 100% GPU, 2820 MHz, 15710 MiB
23:51:47   85.18 W, 100% GPU, 2820 MHz, 15718 MiB
23:51:48   84.72 W, 100% GPU, 2820 MHz, 15718 MiB
...

The GPU can remain in this state for a long time without sampler progress.

Explicit OOM evidence in comfy-kitchen int8_linear

A separate run failed directly inside the CUDA INT8 path:

File "...comfy_kitchen\backends\cuda\__init__.py", line 2064, in int8_linear
    x_qdata = torch.empty((m, k), dtype=torch.int8, device=x.device)

torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory.
Currently allocated     : 11.55 GiB
Requested               : 440.20 MiB
Device limit            : 15.99 GiB
Free (according to CUDA): 0 bytes

Another controlled run with DynamicVRAM disabled failed in the same general path:

File "...comfy_kitchen\backends\cuda\__init__.py", line 2120, in int8_linear
    out = torch.empty((m, n), dtype=out_dtype, device=x.device)

torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory.
Currently allocated     : 12.24 GiB
Requested               : 880.40 MiB
Device limit            : 15.99 GiB
Free (according to CUDA): 0 bytes

A/B tests already performed

Async offload

Enabled: bad ~80 W stall reproduced
Disabled: same bad ~80 W stall reproduced
Conclusion: async offload is not the root cause

DynamicVRAM

Enabled: model can run, but later re-entry can stall
Disabled: first sampler can hard-OOM immediately in int8_linear requesting ~880 MiB
Conclusion: DynamicVRAM is needed to fit this workload on 16 GB, but does not prevent the re-entry failure

VRAM reserve

--reserve-vram 2: stall reproduced
--reserve-vram 1: stall reproduced
Conclusion: a simple reserve does not eliminate the failure

Platform

Reproduced previously under Docker/WSL
Reproduced again on native Windows Portable
Conclusion: not specific to WSL/GPU-PV

Quant/model control

Wan Animate 2 INT8 ConvRot: reproduces
Wan Animate 2 Distill INT8 ConvRot: reproduces
Non-ConvRot FP8 control model: no ~80 W stall observed across repeated model re-entry
Caveat: the FP8 control is a different Wan model family and is not a quality-equivalent replacement

The reproducible common factor across the two affected Wan Animate 2 variants is INT8 ConvRot. The explicit OOM traces land in comfy_kitchen int8_linear, where temporary INT8/output buffers are allocated while CUDA reports 0 free bytes.

The bad-state telemetry also looks different from ordinary compute: utilization remains at 100%, clocks remain high, but power collapses from ~250–265 W to ~80 W and progress stops.

This may involve a lifecycle interaction between:

INT8 ConvRot scratch/workspace allocation
DynamicVRAM / AIMDO residency
Windows WDDM memory accounting
model warm re-entry / reuse

I am not claiming which component is definitively responsible, only that the failure is reproducible and consistently associated with this execution path.

Potentially related reports:

ComfyUI #16470 — Qwen 2.1 INT8 ConvRot much slower than Q8 GGUF
https://github.com/Comfy-Org/ComfyUI/issues/16470
ComfyUI #16437 — Qwen-Image 2.1 DynamicVRAM corruption after model reload
https://github.com/Comfy-Org/ComfyUI/issues/16437
ComfyUI #15452 — Dynamic VRAM warm/reused quantized model produces invalid output
https://github.com/Comfy-Org/ComfyUI/issues/15452
ComfyUI #15352 — MiniMax H3 failure after unload/reload on Windows with AIMDO DynamicVRAM
https://github.com/Comfy-Org/ComfyUI/issues/15352
comfy-kitchen #78 — INT8 ConvRot backend behavior / int8_linear routing differences
https://github.com/Comfy-Org/comfy-kitchen/issues/78
comfy-kitchen #55 — INT8 ConvRot CUDA failures/corruption on another NVIDIA platform
https://github.com/Comfy-Org/comfy-kitchen/issues/55

What would be useful to know

Is int8_linear expected to require temporary allocations of this size during every re-entry?
Is there a known scratch-buffer reuse / workspace lifecycle issue for ConvRot under DynamicVRAM/AIMDO on Windows?
Is there a supported way to force a different backend specifically for int8_linear on NVIDIA for comparison?
Would a minimal standalone reproducer outside ComfyUI help isolate the issue further?

I can provide full ComfyUI logs and 1-second nvidia-smi telemetry if useful.

Originally created by @Epervios on GitHub (Sep 24, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-kitchen/issues/196 [video_wan_animate2.zip](https://github.com/user-attachments/files/32758080/video_wan_animate2.zip) ## Summary INT8 ConvRot models on Windows/NVIDIA reproducibly enter a bad state during model re-entry / reload: the first inference segment completes normally, but a subsequent segment can stall indefinitely at `Model Initializing` with GPU utilization reported at 100%, clocks high, power collapsing to ~80 W, and VRAM remaining near saturation (~15.6–15.9 GB on a 16 GB card). In some runs the same path fails with an explicit OOM inside `comfy_kitchen.backends.cuda.int8_linear` while trying to allocate temporary buffers. The issue reproduces with both: - `wan_animate_2_int8_convrot.safetensors` - `wan_animate_2_distill_int8_convrot.safetensors` A non-ConvRot FP8 model does not reproduce the ~80 W stall pattern under the same ComfyUI environment, although it is not functionally equivalent to Wan Animate 2 and therefore is only a control for the execution path. ## Environment - OS: Windows 11 - GPU: NVIDIA GeForce RTX 4070 Ti SUPER, 16 GB VRAM - System RAM: 64 GB - ComfyUI: 0.37.0 - Python: 3.13.14 - PyTorch: 2.13.0+cu130 - comfy-kitchen: 0.2.35 - comfy-aimdo: 0.5.5 - Device allocator: cudaMallocAsync - DynamicVRAM: enabled unless otherwise noted - Async weight offload: tested both enabled and disabled - Attention: PyTorch attention ## Reproduction pattern Using the Wan Animate 2 workflow with multiple sequential motion-transfer segments: 1. Start ComfyUI normally. 2. Load `wan_animate_2_int8_convrot.safetensors` or `wan_animate_2_distill_int8_convrot.safetensors`. 3. First segment completes normally. 4. VAE work runs and the WAN model is prepared again for the next segment. 5. Second/later segment may remain at: `0/6 [..., Model Initializing ...]` 6. GPU telemetry transitions from normal compute (~250–265 W) to a persistent bad state around ~80–87 W while utilization remains 100%. Example with distilled model: ``` [INFO] Model storage policy: fast_disk=True paths=['...wan_animate_2_distill_int8_convrot.safetensors'] [INFO] Requested to load WAN_Animate2 [INFO] Model WAN_Animate2 prepared for dynamic VRAM loading. 15878MB Staged. 1059 patches attached. Force pre-loaded 202 weights: 2008 KB. 100%|...| 6/6 [02:00<00:00, 20.06s/it] ... [INFO] Model WAN_Animate2 prepared for dynamic VRAM loading. 15878MB Staged. 1059 patches attached. Force pre-loaded 202 weights: 2008 KB. 0%|...| 0/6 [00:00<?, ?it/s, Model Initializing ...] ``` ## GPU telemetry signature Immediately before the bad state: ``` 23:51:40 ~263 W, ~15.7 GB VRAM 23:51:45 ~254 W, ~15.7 GB VRAM 23:51:46 86.99 W, 100% GPU, 2820 MHz, 15710 MiB 23:51:47 85.18 W, 100% GPU, 2820 MHz, 15718 MiB 23:51:48 84.72 W, 100% GPU, 2820 MHz, 15718 MiB ... ``` The GPU can remain in this state for a long time without sampler progress. ## Explicit OOM evidence in comfy-kitchen int8_linear A separate run failed directly inside the CUDA INT8 path: ``` File "...comfy_kitchen\backends\cuda\__init__.py", line 2064, in int8_linear x_qdata = torch.empty((m, k), dtype=torch.int8, device=x.device) torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. Currently allocated : 11.55 GiB Requested : 440.20 MiB Device limit : 15.99 GiB Free (according to CUDA): 0 bytes ``` Another controlled run with DynamicVRAM disabled failed in the same general path: ``` File "...comfy_kitchen\backends\cuda\__init__.py", line 2120, in int8_linear out = torch.empty((m, n), dtype=out_dtype, device=x.device) torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. Currently allocated : 12.24 GiB Requested : 880.40 MiB Device limit : 15.99 GiB Free (according to CUDA): 0 bytes ``` ## A/B tests already performed ### Async offload - Enabled: bad ~80 W stall reproduced - Disabled: same bad ~80 W stall reproduced - Conclusion: async offload is not the root cause ### DynamicVRAM - Enabled: model can run, but later re-entry can stall - Disabled: first sampler can hard-OOM immediately in `int8_linear` requesting ~880 MiB - Conclusion: DynamicVRAM is needed to fit this workload on 16 GB, but does not prevent the re-entry failure ### VRAM reserve - `--reserve-vram 2`: stall reproduced - `--reserve-vram 1`: stall reproduced - Conclusion: a simple reserve does not eliminate the failure ### Platform - Reproduced previously under Docker/WSL - Reproduced again on native Windows Portable - Conclusion: not specific to WSL/GPU-PV ### Quant/model control - Wan Animate 2 INT8 ConvRot: reproduces - Wan Animate 2 Distill INT8 ConvRot: reproduces - Non-ConvRot FP8 control model: no ~80 W stall observed across repeated model re-entry - Caveat: the FP8 control is a different Wan model family and is not a quality-equivalent replacement ## Why this seems related to the INT8 ConvRot / int8_linear path The reproducible common factor across the two affected Wan Animate 2 variants is INT8 ConvRot. The explicit OOM traces land in `comfy_kitchen int8_linear`, where temporary INT8/output buffers are allocated while CUDA reports 0 free bytes. The bad-state telemetry also looks different from ordinary compute: utilization remains at 100%, clocks remain high, but power collapses from ~250–265 W to ~80 W and progress stops. This may involve a lifecycle interaction between: - INT8 ConvRot scratch/workspace allocation - DynamicVRAM / AIMDO residency - Windows WDDM memory accounting - model warm re-entry / reuse I am not claiming which component is definitively responsible, only that the failure is reproducible and consistently associated with this execution path. ## Related issues Potentially related reports: - ComfyUI #16470 — Qwen 2.1 INT8 ConvRot much slower than Q8 GGUF https://github.com/Comfy-Org/ComfyUI/issues/16470 - ComfyUI #16437 — Qwen-Image 2.1 DynamicVRAM corruption after model reload https://github.com/Comfy-Org/ComfyUI/issues/16437 - ComfyUI #15452 — Dynamic VRAM warm/reused quantized model produces invalid output https://github.com/Comfy-Org/ComfyUI/issues/15452 - ComfyUI #15352 — MiniMax H3 failure after unload/reload on Windows with AIMDO DynamicVRAM https://github.com/Comfy-Org/ComfyUI/issues/15352 - comfy-kitchen #78 — INT8 ConvRot backend behavior / int8_linear routing differences https://github.com/Comfy-Org/comfy-kitchen/issues/78 - comfy-kitchen #55 — INT8 ConvRot CUDA failures/corruption on another NVIDIA platform https://github.com/Comfy-Org/comfy-kitchen/issues/55 ## What would be useful to know 1. Is `int8_linear` expected to require temporary allocations of this size during every re-entry? 2. Is there a known scratch-buffer reuse / workspace lifecycle issue for ConvRot under DynamicVRAM/AIMDO on Windows? 3. Is there a supported way to force a different backend specifically for `int8_linear` on NVIDIA for comparison? 4. Would a minimal standalone reproducer outside ComfyUI help isolate the issue further? I can provide full ComfyUI logs and 1-second `nvidia-smi` telemetry if useful.

bsakaguchi commented

2026-10-26 22:19:49 +00:00

Author

Owner

@michaelmanly commented on GitHub (Sep 26, 2026):

@Epervios
looks like the first Wan Animate segment runs successfully, then the same workflow can hang at model re-entry or OOM on the next segment.

can you share the workflow JSON? i'll give you the exact Badgr Run command for the same workflow.

@michaelmanly commented on GitHub (Sep 26, 2026): @Epervios looks like the first Wan Animate segment runs successfully, then the same workflow can hang at model re-entry or OOM on the next segment. can you share the workflow JSON? i'll give you the exact Badgr Run command for the same workflow.

bsakaguchi commented

2026-10-26 22:19:50 +00:00

Author

Owner

@Epervios commented on GitHub (Sep 28, 2026):

@michaelmanly Thanks for looking into this. I have the original video_wan_animate2.json from my first test. It contains two chained Wan Animate 2 Motion Transfer subgraphs using the INT8 ConvRot model. The second subgraph is bypassed in the saved workflow and needs to be enabled to reproduce the model re-entry issue. I'll add the JSON file separately so you can provide the exact Badgr Run command.

@Epervios commented on GitHub (Sep 28, 2026): @michaelmanly Thanks for looking into this. I have the original `video_wan_animate2.json` from my first test. It contains two chained Wan Animate 2 Motion Transfer subgraphs using the INT8 ConvRot model. The second subgraph is bypassed in the saved workflow and needs to be enabled to reproduce the model re-entry issue. I'll add the JSON file separately so you can provide the exact Badgr Run command.

bsakaguchi referenced this issue

2026-10-26 22:20:30 +00:00

[PR #58] [CLOSED] Fix INT8 quantization Triton kernel for HIP/ROCm #112

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-kitchen#58

No description provided.

Rows
Columns