[GH-ISSUE #125] malloc_graph_destroy() stack-overflows on deep allocation graphs (Windows 1 MB stack), cleanup aborts, next run crashes with 0xC0000005 #47

New issue

Open

opened 2026-10-26 22:24:03 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:24:03 +00:00

Owner

Originally created by @shellddd on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/125

Summary

On Windows, malloc_graph_destroy() recurses once per recorded allocation (free_events() and
materialize() are both recursive). For large graphs — e.g. a 104k-token attention in a video
model — the event chain is deep enough to exhaust the default 1 MB thread stack
(STATUS_STACK_OVERFLOW, 0xC00000FD).

Because the call is made from MallocGraph.__del__, CPython swallows the failure
(Exception ignored in: <function MallocGraph.__del__>), so the graph is left half destroyed:
its physical_page_unref() / virtual_range_unref() calls never run. The next large model run
then crashes with 0xC0000005 inside aimdo.dll.

So the observable failure is: one prompt finishes "successfully" with a swallowed stack
overflow at teardown, and the next prompt dies with an access violation.

Environment

OS: Windows 11 26200 (x64)
Python: 3.13.11 (embedded)
PyTorch: 2.9.1+cu130
NVIDIA driver: 610.88 (CUDA 13.x)
GPU: RTX 3080 20 GB
comfy-aimdo: 0.5.5 (also checked v0.5.6 source: unchanged)
ComfyUI: 0.38.2, dynamic VRAM enabled (default)

Reproduction

Workflow: MiniMax H3 video model, 832x480, dynamic VRAM on, --fast-disk,
ComfyUI's model compiler enabled (default).

Run a prompt whose attention is ~90k tokens (≈31 s of video, ~700 frames). It may succeed,
but the log shows at teardown:

Windows fatal exception: stack overflow

Stack (most recent call first):
  File "...\comfy_aimdo\malloc_graph.py", line 74 in __del__
  File "...\comfy\model_prefetch.py", line 103 in cleanup_malloc_graph
  File "...\comfy\model_prefetch.py", line 157 in cleanup_prefetch_queues
  File "...\comfy\execution.py", line 552 in execute
  ...

Exception ignored in: <function MallocGraph.__del__ at 0x...>
Traceback (most recent call last):
  File "...\comfy_aimdo\malloc_graph.py", line 74, in __del__
    control.lib.malloc_graph_destroy(handle)
OSError: exception: stack overflow

Run another prompt of the same size. It crashes:

[INFO] BlockSparseAttention: dense (92164 tokens): sigma 1 ...
[INFO] BlockSparseAttention: dense (92164 tokens): sigma 0.983 ...
[Exited, code -1073741819 (0xC0000005)]

Windows event log:

Faulting application name: python.exe
Faulting module name: aimdo.dll
Exception code: 0xc0000005
Fault offset: 0x0000000000004cac   (earlier crashes: 0x4c72)

Observed 3 times across different sessions, always with the same shape:
a swallowed stack overflow at teardown, then 0xC0000005 on the next ≥90k-token run.
Runs below ~65k tokens never crash, even when repeated dozens of times in one process.

Root cause (from source)

src/malloc-graph.c:

static Event *event(MallocGraph *g, EventType type, size_t value, size_t bytes) {
    Event *event = calloc(1, sizeof(*event));
    ...
    append_event(g, g->state->cursor, event);   // each event becomes a child of the previous one
    g->state->cursor = event;
    return event;
}

This builds a linear chain of events, one per recorded allocation, so the tree depth equals the
number of allocations recorded for the run.

Both teardown paths then recurse along that chain:

static void free_events(Event **events, size_t count) {
    for (size_t i = 0; i < count; i++) {
        Event *event = events[i];
        free_events(event->next, event->next_count);        // recursion depth == chain length
        ...
    }
}

static bool materialize(MallocGraph *g, Event *event) {
    if (event->snapshot) { ... }
    return materialize(g, event->previous) && dry_apply(g, event);   // same
}

scrub_rogue_snapshots() recurses over the same tree as well.

malloc_graph_destroy() runs abort_graph() (which calls materialize()), then
free_events(). On Windows the thread stack (the prompt worker thread created by
threading.Thread) is 1 MB. A large video/attention graph records tens of thousands of
allocations, so the recursion blows the stack.

When that happens inside __del__, the OSError: exception: stack overflow is swallowed by
CPython, so:

free_events() only freed the events below the failing frame,
the physical_page_unref() loop over mapped_pages / physical_pages / small_* never runs,
virtual_range_unref(g->base) / virtual_range_unref(g->small_base) never run
(each graph reserves MG_PAGES = 8192 × 8 MiB = 64 GiB of VA plus 8 GiB for small pages).

The leaked/partially torn-down graph then poisons the next large allocation, producing the
0xC0000005 inside aimdo.dll.

Suggested fix

Convert the three recursive walkers to explicit-stack loops. Sketch for free_events():

typedef struct { Event **items; size_t count; size_t index; } FreeFrame;

static void free_events(Event **events, size_t count) {
    // explicit stack of (child-array, index) frames, no C recursion
    ...
}

materialize(g, event) needs the chain collected into an array first (walk
event->previous to the snapshot/root), then applied top-down — again no recursion.
scrub_rogue_snapshots() is the same tree walk with an explicit stack.

Even simpler defensive measure: if the graph's event count is huge, do the destroy work on a
worker thread with a large stack, or bail out of destroy() before touching the tree and hand
the VRAM back through the existing rogue-candidate path.

Workarounds we use on our side

--disable-comfy-compiler (no graph recorded at all), or
raising the prompt-worker thread stack (threading.stack_size(64 << 20)), which multiplied the
usable depth ~60×, or
capping shots below ~87k tokens (30 s at 832x480) so the chain stays shallow.

Attachments

Happy to attach full comfyui.log + launcher diagnostic bundles for all three crashes
(python 3.13.11, torch 2.9.1+cu130, driver 610.88) if useful.

Related but different: #48 is 0xC00000FD at startup on 0.2.14; this one is a runtime
stack overflow during graph teardown on large graphs, only after ≥90k-token runs.

Originally created by @shellddd on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/125 ## Summary On Windows, `malloc_graph_destroy()` recurses once per recorded allocation (`free_events()` and `materialize()` are both recursive). For large graphs — e.g. a 104k-token attention in a video model — the event chain is deep enough to exhaust the default 1 MB thread stack (`STATUS_STACK_OVERFLOW`, 0xC00000FD). Because the call is made from `MallocGraph.__del__`, CPython swallows the failure (`Exception ignored in: <function MallocGraph.__del__>`), so the graph is left half destroyed: its `physical_page_unref()` / `virtual_range_unref()` calls never run. The next large model run then crashes with `0xC0000005` inside `aimdo.dll`. So the observable failure is: **one prompt finishes "successfully" with a swallowed stack overflow at teardown, and the next prompt dies with an access violation.** ## Environment - OS: Windows 11 26200 (x64) - Python: 3.13.11 (embedded) - PyTorch: 2.9.1+cu130 - NVIDIA driver: 610.88 (CUDA 13.x) - GPU: RTX 3080 20 GB - comfy-aimdo: 0.5.5 (also checked v0.5.6 source: unchanged) - ComfyUI: 0.38.2, dynamic VRAM enabled (default) ## Reproduction Workflow: MiniMax H3 video model, 832x480, dynamic VRAM on, `--fast-disk`, ComfyUI's model compiler enabled (default). 1. Run a prompt whose attention is ~90k tokens (≈31 s of video, ~700 frames). It may succeed, but the log shows at teardown: ``` Windows fatal exception: stack overflow Stack (most recent call first): File "...\comfy_aimdo\malloc_graph.py", line 74 in __del__ File "...\comfy\model_prefetch.py", line 103 in cleanup_malloc_graph File "...\comfy\model_prefetch.py", line 157 in cleanup_prefetch_queues File "...\comfy\execution.py", line 552 in execute ... Exception ignored in: <function MallocGraph.__del__ at 0x...> Traceback (most recent call last): File "...\comfy_aimdo\malloc_graph.py", line 74, in __del__ control.lib.malloc_graph_destroy(handle) OSError: exception: stack overflow ``` 2. Run another prompt of the same size. It crashes: ``` [INFO] BlockSparseAttention: dense (92164 tokens): sigma 1 ... [INFO] BlockSparseAttention: dense (92164 tokens): sigma 0.983 ... [Exited, code -1073741819 (0xC0000005)] ``` Windows event log: ``` Faulting application name: python.exe Faulting module name: aimdo.dll Exception code: 0xc0000005 Fault offset: 0x0000000000004cac (earlier crashes: 0x4c72) ``` Observed 3 times across different sessions, always with the same shape: a swallowed `stack overflow` at teardown, then `0xC0000005` on the next ≥90k-token run. Runs below ~65k tokens never crash, even when repeated dozens of times in one process. ## Root cause (from source) `src/malloc-graph.c`: ```c static Event *event(MallocGraph *g, EventType type, size_t value, size_t bytes) { Event *event = calloc(1, sizeof(*event)); ... append_event(g, g->state->cursor, event); // each event becomes a child of the previous one g->state->cursor = event; return event; } ``` This builds a *linear chain* of events, one per recorded allocation, so the tree depth equals the number of allocations recorded for the run. Both teardown paths then recurse along that chain: ```c static void free_events(Event **events, size_t count) { for (size_t i = 0; i < count; i++) { Event *event = events[i]; free_events(event->next, event->next_count); // recursion depth == chain length ... } } static bool materialize(MallocGraph *g, Event *event) { if (event->snapshot) { ... } return materialize(g, event->previous) && dry_apply(g, event); // same } ``` `scrub_rogue_snapshots()` recurses over the same tree as well. `malloc_graph_destroy()` runs `abort_graph()` (which calls `materialize()`), then `free_events()`. On Windows the thread stack (the prompt worker thread created by `threading.Thread`) is 1 MB. A large video/attention graph records tens of thousands of allocations, so the recursion blows the stack. When that happens inside `__del__`, the `OSError: exception: stack overflow` is swallowed by CPython, so: - `free_events()` only freed the events below the failing frame, - the `physical_page_unref()` loop over `mapped_pages` / `physical_pages` / `small_*` never runs, - `virtual_range_unref(g->base)` / `virtual_range_unref(g->small_base)` never run (each graph reserves `MG_PAGES = 8192` × 8 MiB = **64 GiB** of VA plus 8 GiB for small pages). The leaked/partially torn-down graph then poisons the next large allocation, producing the `0xC0000005` inside `aimdo.dll`. ## Suggested fix Convert the three recursive walkers to explicit-stack loops. Sketch for `free_events()`: ```c typedef struct { Event **items; size_t count; size_t index; } FreeFrame; static void free_events(Event **events, size_t count) { // explicit stack of (child-array, index) frames, no C recursion ... } ``` `materialize(g, event)` needs the chain collected into an array first (walk `event->previous` to the snapshot/root), then applied top-down — again no recursion. `scrub_rogue_snapshots()` is the same tree walk with an explicit stack. Even simpler defensive measure: if the graph's event count is huge, do the destroy work on a worker thread with a large stack, or bail out of `destroy()` before touching the tree and hand the VRAM back through the existing rogue-candidate path. ## Workarounds we use on our side - `--disable-comfy-compiler` (no graph recorded at all), or - raising the prompt-worker thread stack (`threading.stack_size(64 << 20)`), which multiplied the usable depth ~60×, or - capping shots below ~87k tokens (30 s at 832x480) so the chain stays shallow. ## Attachments Happy to attach full `comfyui.log` + launcher diagnostic bundles for all three crashes (python 3.13.11, torch 2.9.1+cu130, driver 610.88) if useful. Related but different: #48 is `0xC00000FD` at startup on 0.2.14; this one is a *runtime* stack overflow during graph teardown on large graphs, only after ≥90k-token runs.

bsakaguchi referenced this issue

2026-10-26 22:24:24 +00:00

[PR #47] [MERGED] Handle Linux integrated GPU memory budgeting (CORE-128) #82