Comfy-Org/comfy-aimdo

Fork 0

mirror of https://github.com/Comfy-Org/comfy-aimdo.git synced 2026-11-04 23:37:21 +00:00

GGUF); GGUF hangs before sampling #25

New issue

Open

opened 2026-10-26 22:23:47 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 22:23:47 +00:00

Owner

Originally created by @tutggadfasdfgsd-web on GitHub (Aug 5, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/82

Environment

GPU: AMD Radeon RX 7900 XT (20GB), Windows 11
torch: 2.9.1+rocm7.2.1 / ROCm 7.2.1
ComfyUI: 0.30.0 (requirements.txt pins comfy-aimdo==0.4.11)
comfy-aimdo: tested 0.4.11 and 0.4.13 (latest)

Summary

With --enable-dynamic-vram on ROCm/Windows, DynamicVRAM initializes successfully
(log: comfy-aimdo inited for GPU: AMD Radeon RX 7900 XT (VRAM: 20464 MB)), but
quantized models produce wrong output or hang:

1) Quantized safetensors model (MiniMaxH3, INT4 `convrot_w4a4`, MixedPrecisionOps)

→ output is a completely gray image (constant latent).

0.4.11: sampler 10 steps in <1s (10.35 it/s) — no real compute at all.
0.4.13: sampler 10 steps in ~5s (1.74 it/s) — real compute now, but output still gray.
No errors/exceptions in either version (0.4.13 added native log routing; nothing logged).

2) GGUF model (same MiniMaxH3 arch, qtypes F32/BF16/Q4_0/Q4_K, sd.cpp compat mode)

→ sampler never starts step 1: dedicated VRAM fills to ~19.4GB, GPU compute ~0%,
ComfyUI UI freezes for 5+ minutes (OS itself is responsive).

This model loads via the legacy path (loaded partially; 16569 MB loaded),
i.e. it does NOT use ModelPatcherDynamic — but aimdo's CUDA detour hooks
(6 hooks) are still installed at the torch level.

Diagnostics performed (inside a running ComfyUI process)

Minimal hostbuf → vbar → device roundtrip test with aimdo already initialized:

HostBuffer write/read: consistent ✅
vbar_fault: OK ✅
copy pattern bytes to vbar device tensor, read back: identical ✅ (roundtrip_equal: true)

So the low-level data path (HostBuffer → VBAR → VRAM) is correct on ROCm
for plain byte copies.

Suspected root cause

Issue appears specific to quantized weights interacting with dynamic loading:

In ModelPatcherDynamic.load() (model_patcher.py), for QuantizedTensor weights the
geometry passed to vbar allocation is the weight object itself, whereas non-quantized
weights get a TensorGeometry — possible incorrect size/layout accounting.
GGUF case: aimdo's CUDA detour hooks may interfere with the legacy (non-dynamic)
load/compute path even though the model never uses ModelPatcherDynamic.

Questions

Is ROCm officially supported for quantized weights with DynamicVRAM?
Any known workaround for quantized models on ROCm?
Happy to provide full logs or run additional diagnostics on request.

Originally created by @tutggadfasdfgsd-web on GitHub (Aug 5, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/82 ## Environment - GPU: AMD Radeon RX 7900 XT (20GB), Windows 11 - torch: 2.9.1+rocm7.2.1 / ROCm 7.2.1 - ComfyUI: 0.30.0 (requirements.txt pins comfy-aimdo==0.4.11) - comfy-aimdo: tested **0.4.11** and **0.4.13** (latest) ## Summary With `--enable-dynamic-vram` on ROCm/Windows, DynamicVRAM initializes successfully (log: `comfy-aimdo inited for GPU: AMD Radeon RX 7900 XT (VRAM: 20464 MB)`), but **quantized models produce wrong output or hang**: ### 1) Quantized safetensors model (MiniMaxH3, INT4 `convrot_w4a4`, MixedPrecisionOps) → output is a **completely gray image** (constant latent). - 0.4.11: sampler 10 steps in <1s (10.35 it/s) — no real compute at all. - 0.4.13: sampler 10 steps in ~5s (1.74 it/s) — real compute now, but output **still gray**. - No errors/exceptions in either version (0.4.13 added native log routing; nothing logged). ### 2) GGUF model (same MiniMaxH3 arch, qtypes F32/BF16/Q4_0/Q4_K, sd.cpp compat mode) → sampler **never starts step 1**: dedicated VRAM fills to ~19.4GB, GPU compute ~0%, ComfyUI UI freezes for 5+ minutes (OS itself is responsive). - This model loads via the **legacy path** (`loaded partially; 16569 MB loaded`), i.e. it does NOT use ModelPatcherDynamic — but aimdo's CUDA detour hooks (6 hooks) are still installed at the torch level. ## Diagnostics performed (inside a running ComfyUI process) Minimal hostbuf → vbar → device roundtrip test with aimdo already initialized: - HostBuffer write/read: consistent ✅ - `vbar_fault`: OK ✅ - copy pattern bytes to vbar device tensor, read back: **identical** ✅ (`roundtrip_equal: true`) So the low-level data path (HostBuffer → VBAR → VRAM) is **correct on ROCm** for plain byte copies. ## Suspected root cause Issue appears specific to **quantized weights** interacting with dynamic loading: - In `ModelPatcherDynamic.load()` (`model_patcher.py`), for `QuantizedTensor` weights the geometry passed to vbar allocation is the weight object itself, whereas non-quantized weights get a `TensorGeometry` — possible incorrect size/layout accounting. - GGUF case: aimdo's CUDA detour hooks may interfere with the **legacy** (non-dynamic) load/compute path even though the model never uses ModelPatcherDynamic. ## Questions 1. Is ROCm officially supported for quantized weights with DynamicVRAM? 2. Any known workaround for quantized models on ROCm? Happy to provide full logs or run additional diagnostics on request.

bsakaguchi commented

2026-10-26 22:23:48 +00:00

Author

Owner

@coderabbitai[bot] commented on GitHub (Aug 5, 2026):

🔗 Related PRs

Comfy-Org/comfy-aimdo#78 - Implement NVML pressure and async pre-free clamping [merged]

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

@coderabbitai[bot] commented on GitHub (Aug 5, 2026):  <details> <summary>🔗 Related PRs</summary> Comfy-Org/comfy-aimdo#78 - Implement NVML pressure and async pre-free clamping [merged] </details> --- <details> <summary> 🧪 Issue enrichment is currently in open beta.</summary> You can configure auto-planning by selecting labels in the issue_enrichment configuration. To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`: ```yaml issue_enrichment: auto_enrich: enabled: false ``` </details> 💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit)!

bsakaguchi commented

2026-10-26 22:23:49 +00:00

Author

Owner

@0xDELUXA commented on GitHub (Aug 12, 2026):

I haven't encountered this on RDNA4/Windows ROCm 10.1 and Torch 2.14. Could you try testing it again on your system? Also, GGUF or W4A4 isn't the best quantization option. W4A8 or int8 convrot would be a better choice.

@0xDELUXA commented on GitHub (Aug 12, 2026): I haven't encountered this on RDNA4/Windows ROCm 10.1 and Torch 2.14. Could you try testing it again on your system? Also, GGUF or W4A4 isn't the best quantization option. W4A8 or int8 convrot would be a better choice.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/comfy-aimdo#25

No description provided.

Rows
Columns

[GH-ISSUE #82] [ROCm/Windows] DynamicVRAM: gray image output with quantized models (INT4/GGUF); GGUF hangs before sampling #25

Environment

Summary

1) Quantized safetensors model (MiniMaxH3, INT4 convrot_w4a4, MixedPrecisionOps)

2) GGUF model (same MiniMaxH3 arch, qtypes F32/BF16/Q4_0/Q4_K, sd.cpp compat mode)

Diagnostics performed (inside a running ComfyUI process)

Suspected root cause

Questions

1) Quantized safetensors model (MiniMaxH3, INT4 `convrot_w4a4`, MixedPrecisionOps)