[GH-ISSUE #128] Redundant full-deficit eviction on every allocation causes multi-hour VRAM thrash under sustained oversubscription #50

New issue

Open

opened 2026-10-26 22:24:04 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:24:04 +00:00

Owner

Originally created by @sugarkrill on GitHub (Oct 22, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/128

Title: Redundant full-deficit eviction on every allocation causes multi-hour VRAM thrash under sustained oversubscription

Summary

Under sustained VRAM oversubscription on Windows, several allocation call sites each unconditionally re-run a full vbars_free(budget_deficit(X)) against the current reported deficit before attempting their own allocation — even though budget_deficit()'s underlying data source is itself explicitly throttled and only refreshes every ~2s. On a hot allocation path (cuMemAllocAsync, invoked on every PyTorch allocation during model compute), this means hundreds of calls per second can each redundantly re-trigger a multi-GB evict-and-reload cycle against a deficit value that hasn't changed since the last check. The result is a GPU that reports ~100% utilization at a small fraction of its rated power draw (observed: ~59W vs. 150-220W under real compute), with VRAM usage pinned near capacity, for anywhere from 20 minutes to 2+ hours, before either completing or hitting an OOM.

Environment where this was reproduced

Windows 11, RTX 4070 Super (12,281 MB VRAM), v0.5.6 (commit 08b37eec3)
DynamicVRAM enabled, workload intentionally oversubscribed (~38.5GB combined model footprint)
Reproduced consistently with and without --disable-dynamic-vram's equivalent legacy path involved (the thrash is in this shared allocation code, not specific to any one caller)

Root cause

Five call sites call vbars_free(budget_deficit(X)) directly, with no debounce:

src/model-vbar.c, vbar_fault() — the per-fault "stopgap" call
src/malloc-graph.c, create_page()
src/vrambuf.c, vrambuf_grow()
src/pyt-cu-plug-alloc-async.c, aimdo_cuda_malloc() and aimdo_cuda_malloc_async()

budget_deficit() itself is cheap to call, but its WDDM-backed component (poll_budget_deficit() in src-win/shmem-detect.c) only actually refreshes its view of GPU memory usage once every 2000ms (the non-Windows cuda_budget_deficit() in src/control.c has the identical 2000ms throttle). Between refreshes, every call to any of the five sites above sees the same deficit value and re-runs a full vbars_free() eviction for it — even though a very recent call already acted on that exact number. On Windows specifically, this is compounded by WDDM's reported "usage" bundling pinned host memory (used for the demand-paged weight-streaming staging buffer) in with VRAM under one combined budget, so the deficit this loop is reacting to is frequently much larger than the true in-use VRAM — but that inflation is a separate issue from the redundant-call bug itself, which would waste cycles even against an accurate deficit number.

Proposed fix

Replace the five raw calls with a single throttled helper that shares one per-device timestamp, gated to the same 2000ms cadence the deficit data itself refreshes on. When it does run, it still evicts the full deficit it computes — this only suppresses the immediate repeat calls that would otherwise act on stale data. Each call site's existing OOM-retry fallback (the unthrottled, bounded vbars_free() call used when the first allocation attempt still fails) is untouched, so correctness in the genuine-OOM case is unaffected.

Diff against v0.5.6 (08b37eec3):

diff --git a/src/control.c b/src/control.c
index c7e2809..9757d5d 100644
--- a/src/control.c
+++ b/src/control.c
@@ -180,6 +180,16 @@ bool cuda_budget_deficit(const char **prevailing_deficit_method) {
     return true;
 }
 
+void stopgap_vbars_free(size_t size) {
+    uint64_t now = GET_TICK();
+
+    if (now - stopgap_evict_last_check < 2000) {
+        return;
+    }
+    stopgap_evict_last_check = now;
+    vbars_free(budget_deficit(size));
+}
+
 SHARED_EXPORT
 void aimdo_analyze(void *devctx) {
     size_t free_bytes = 0, total_bytes = 0;
diff --git a/src/control.h b/src/control.h
index 9c024c1..fc39531 100644
--- a/src/control.h
+++ b/src/control.h
@@ -39,6 +39,7 @@ typedef struct AimdoContext {
     uint64_t _total_vram_last_check;
     ssize_t _deficit_sync;
     uint64_t _control_timestamp_last_check;
+    uint64_t _stopgap_evict_last_check;
     void *_highest_priority; /* ModelVBAR * */
     void *_lowest_priority; /* ModelVBAR * */
     bool _vbars_dirty;
@@ -99,6 +100,7 @@ bool set_devctx_for_current_cuda_device(void);
 #define wddm_timestamp_last_check   (g_devctx->_wddm_timestamp_last_check)
 #endif
 #define control_timestamp_last_check (g_devctx->_control_timestamp_last_check)
+#define stopgap_evict_last_check    (g_devctx->_stopgap_evict_last_check)
 
 #define highest_priority            (*highest_priority_p)
 #define lowest_priority             (*lowest_priority_p)
diff --git a/src/malloc-graph.c b/src/malloc-graph.c
index 477f579..85b137e 100644
--- a/src/malloc-graph.c
+++ b/src/malloc-graph.c
@@ -423,7 +423,7 @@ static bool push_stack(MallocGraph *g, Event *scope, bool recording) {
 static CUresult create_page(MallocGraph *g, PhysicalPage **page) {
     CUresult r;
 
-    vbars_free(budget_deficit(MG_PAGE));
+    stopgap_vbars_free(MG_PAGE);
     r = physical_page_alloc(page, MG_PAGE, g->device);
     if (r == CUDA_ERROR_OUT_OF_MEMORY) {
         vbars_free(MG_PAGE);
diff --git a/src/model-vbar.c b/src/model-vbar.c
index 83c1d2e..449e078 100644
--- a/src/model-vbar.c
+++ b/src/model-vbar.c
@@ -360,7 +360,7 @@ int vbar_fault(void *devctx, void *vbar, uint64_t offset, uint64_t size, uint32_
      * as the allocator is unreliable as it may not actually be called reliably when you
      * really need to know you have spilled.
      */
-    vbars_free(budget_deficit(0));
+    stopgap_vbars_free(0);
 
     if (page_end > mv->watermark) {
         log(VVERBOSE, "VBAR Allocation is above watermark\n");
diff --git a/src/plat.h b/src/plat.h
index a1cf4b0..2ba24cf 100644
--- a/src/plat.h
+++ b/src/plat.h
@@ -232,6 +232,16 @@ static inline void va_pool_cleanup(void) {}
 
 /* model_vbar.c */
 size_t vbars_free(ssize_t size);
+
+/* control.c: several call sites proactively evict against the current reported deficit
+ * before attempting their own allocation. budget_deficit()'s WDDM-backed component only
+ * refreshes every ~2s (see poll_budget_deficit's own throttle), so back-to-back calls
+ * in between a refresh all see the same stale deficit and would otherwise each
+ * redundantly re-run a full vbars_free() for it. Throttle those proactive flushes to
+ * that same cadence; a call still evicts the full deficit it reports when it does run,
+ * this only skips the immediate repeats of a flush a very recent call already did.
+ */
+void stopgap_vbars_free(size_t size);
 SHARED_EXPORT
 uint64_t vbars_analyze(void *devctx, bool only_dirty);
 
diff --git a/src/pyt-cu-plug-alloc-async.c b/src/pyt-cu-plug-alloc-async.c
index 9bea536..6aeb427 100644
--- a/src/pyt-cu-plug-alloc-async.c
+++ b/src/pyt-cu-plug-alloc-async.c
@@ -164,7 +164,7 @@ int aimdo_cuda_malloc(CUdeviceptr *devPtr, size_t size,
         return true_cuMemAlloc_v2(devPtr, size);
     }
 
-    vbars_free(budget_deficit(size + CUDA_MALLOC_HEADROOM));
+    stopgap_vbars_free(size + CUDA_MALLOC_HEADROOM);
 
     if (CHECK_CU(true_cuMemAlloc_v2(&dptr, size))) {
         *devPtr = dptr;
@@ -227,7 +227,7 @@ int aimdo_cuda_malloc_async(CUdeviceptr *devPtr, size_t size, CUstream hStream,
         return *devPtr ? 0 : CUDA_ERROR_OUT_OF_MEMORY;
     }
 
-    vbars_free(budget_deficit(MIN(size, malloc_async_clamp)));
+    stopgap_vbars_free(MIN(size, malloc_async_clamp));
 
     status = true_cuMemAllocAsync(&dptr, size, hStream);
     if (CHECK_CU(status)) {
diff --git a/src/vrambuf.c b/src/vrambuf.c
index 3503411..0738ac9 100644
--- a/src/vrambuf.c
+++ b/src/vrambuf.c
@@ -109,7 +109,7 @@ bool vrambuf_grow(void *arg, size_t required_size) {
         grow_to = buf->max_size;
     }
 
-    vbars_free(budget_deficit(grow_to - buf->allocated));
+    stopgap_vbars_free(grow_to - buf->allocated);
     while (buf->allocated < grow_to) {
         size_t to_allocate = grow_to - buf->allocated;
         if (to_allocate > VRAM_CHUNK_SIZE) {

Validation performed

Before the fix: identical workload reproducibly hung in the described thrash state (~59W, 100% reported utilization, VRAM pinned, sustained high memory-bus activity) for 20+ minutes on every attempt, confirmed via nvidia-smi dmon telemetry and DEBUG-level aimdo logs cross-referenced by timestamp.
After the fix: the same oversubscribed configuration now either completes successfully or fails fast (within ~5 minutes) with a normal torch.OutOfMemoryError when the request genuinely doesn't fit — no thrash, no hang, process remains stable and responsive throughout both outcomes.
Not yet tested: AMD/HIP backend, integrated GPUs, multi-GPU configurations, Linux (the logic is platform-generic and the Linux-side throttle has the identical 2000ms window, but I have not run it there). This was validated against one specific oversubscribed workload/GPU, not across the project's full support matrix.

Disclosure

This investigation and patch were produced with AI assistance (Claude Code) under my direction and review — the root-cause analysis, patch design, and validation testing were done interactively with an AI agent, not purely by hand. Flagging this so you can weigh the review accordingly.

Originally created by @sugarkrill on GitHub (Oct 22, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/128 **Title:** Redundant full-deficit eviction on every allocation causes multi-hour VRAM thrash under sustained oversubscription **Summary** Under sustained VRAM oversubscription on Windows, several allocation call sites each unconditionally re-run a full `vbars_free(budget_deficit(X))` against the current reported deficit before attempting their own allocation — even though `budget_deficit()`'s underlying data source is itself explicitly throttled and only refreshes every ~2s. On a hot allocation path (`cuMemAllocAsync`, invoked on every PyTorch allocation during model compute), this means hundreds of calls per second can each redundantly re-trigger a multi-GB evict-and-reload cycle against a deficit value that hasn't changed since the last check. The result is a GPU that reports ~100% utilization at a small fraction of its rated power draw (observed: ~59W vs. 150-220W under real compute), with VRAM usage pinned near capacity, for anywhere from 20 minutes to 2+ hours, before either completing or hitting an OOM. **Environment where this was reproduced** - Windows 11, RTX 4070 Super (12,281 MB VRAM), v0.5.6 (commit `08b37eec3`) - DynamicVRAM enabled, workload intentionally oversubscribed (~38.5GB combined model footprint) - Reproduced consistently with and without `--disable-dynamic-vram`'s equivalent legacy path involved (the thrash is in this shared allocation code, not specific to any one caller) **Root cause** Five call sites call `vbars_free(budget_deficit(X))` directly, with no debounce: - `src/model-vbar.c`, `vbar_fault()` — the per-fault "stopgap" call - `src/malloc-graph.c`, `create_page()` - `src/vrambuf.c`, `vrambuf_grow()` - `src/pyt-cu-plug-alloc-async.c`, `aimdo_cuda_malloc()` and `aimdo_cuda_malloc_async()` `budget_deficit()` itself is cheap to call, but its WDDM-backed component (`poll_budget_deficit()` in `src-win/shmem-detect.c`) only actually refreshes its view of GPU memory usage once every 2000ms (the non-Windows `cuda_budget_deficit()` in `src/control.c` has the identical 2000ms throttle). Between refreshes, every call to any of the five sites above sees the *same* deficit value and re-runs a full `vbars_free()` eviction for it — even though a very recent call already acted on that exact number. On Windows specifically, this is compounded by WDDM's reported "usage" bundling pinned host memory (used for the demand-paged weight-streaming staging buffer) in with VRAM under one combined budget, so the deficit this loop is reacting to is frequently much larger than the true in-use VRAM — but that inflation is a separate issue from the redundant-call bug itself, which would waste cycles even against an accurate deficit number. **Proposed fix** Replace the five raw calls with a single throttled helper that shares one per-device timestamp, gated to the same 2000ms cadence the deficit data itself refreshes on. When it does run, it still evicts the full deficit it computes — this only suppresses the immediate repeat calls that would otherwise act on stale data. Each call site's existing OOM-retry fallback (the unthrottled, bounded `vbars_free()` call used when the first allocation attempt still fails) is untouched, so correctness in the genuine-OOM case is unaffected. Diff against v0.5.6 (`08b37eec3`): ```diff diff --git a/src/control.c b/src/control.c index c7e2809..9757d5d 100644 --- a/src/control.c +++ b/src/control.c @@ -180,6 +180,16 @@ bool cuda_budget_deficit(const char **prevailing_deficit_method) { return true; } +void stopgap_vbars_free(size_t size) { + uint64_t now = GET_TICK(); + + if (now - stopgap_evict_last_check < 2000) { + return; + } + stopgap_evict_last_check = now; + vbars_free(budget_deficit(size)); +} + SHARED_EXPORT void aimdo_analyze(void *devctx) { size_t free_bytes = 0, total_bytes = 0; diff --git a/src/control.h b/src/control.h index 9c024c1..fc39531 100644 --- a/src/control.h +++ b/src/control.h @@ -39,6 +39,7 @@ typedef struct AimdoContext { uint64_t _total_vram_last_check; ssize_t _deficit_sync; uint64_t _control_timestamp_last_check; + uint64_t _stopgap_evict_last_check; void *_highest_priority; /* ModelVBAR * */ void *_lowest_priority; /* ModelVBAR * */ bool _vbars_dirty; @@ -99,6 +100,7 @@ bool set_devctx_for_current_cuda_device(void); #define wddm_timestamp_last_check (g_devctx->_wddm_timestamp_last_check) #endif #define control_timestamp_last_check (g_devctx->_control_timestamp_last_check) +#define stopgap_evict_last_check (g_devctx->_stopgap_evict_last_check) #define highest_priority (*highest_priority_p) #define lowest_priority (*lowest_priority_p) diff --git a/src/malloc-graph.c b/src/malloc-graph.c index 477f579..85b137e 100644 --- a/src/malloc-graph.c +++ b/src/malloc-graph.c @@ -423,7 +423,7 @@ static bool push_stack(MallocGraph *g, Event *scope, bool recording) { static CUresult create_page(MallocGraph *g, PhysicalPage **page) { CUresult r; - vbars_free(budget_deficit(MG_PAGE)); + stopgap_vbars_free(MG_PAGE); r = physical_page_alloc(page, MG_PAGE, g->device); if (r == CUDA_ERROR_OUT_OF_MEMORY) { vbars_free(MG_PAGE); diff --git a/src/model-vbar.c b/src/model-vbar.c index 83c1d2e..449e078 100644 --- a/src/model-vbar.c +++ b/src/model-vbar.c @@ -360,7 +360,7 @@ int vbar_fault(void *devctx, void *vbar, uint64_t offset, uint64_t size, uint32_ * as the allocator is unreliable as it may not actually be called reliably when you * really need to know you have spilled. */ - vbars_free(budget_deficit(0)); + stopgap_vbars_free(0); if (page_end > mv->watermark) { log(VVERBOSE, "VBAR Allocation is above watermark\n"); diff --git a/src/plat.h b/src/plat.h index a1cf4b0..2ba24cf 100644 --- a/src/plat.h +++ b/src/plat.h @@ -232,6 +232,16 @@ static inline void va_pool_cleanup(void) {} /* model_vbar.c */ size_t vbars_free(ssize_t size); + +/* control.c: several call sites proactively evict against the current reported deficit + * before attempting their own allocation. budget_deficit()'s WDDM-backed component only + * refreshes every ~2s (see poll_budget_deficit's own throttle), so back-to-back calls + * in between a refresh all see the same stale deficit and would otherwise each + * redundantly re-run a full vbars_free() for it. Throttle those proactive flushes to + * that same cadence; a call still evicts the full deficit it reports when it does run, + * this only skips the immediate repeats of a flush a very recent call already did. + */ +void stopgap_vbars_free(size_t size); SHARED_EXPORT uint64_t vbars_analyze(void *devctx, bool only_dirty); diff --git a/src/pyt-cu-plug-alloc-async.c b/src/pyt-cu-plug-alloc-async.c index 9bea536..6aeb427 100644 --- a/src/pyt-cu-plug-alloc-async.c +++ b/src/pyt-cu-plug-alloc-async.c @@ -164,7 +164,7 @@ int aimdo_cuda_malloc(CUdeviceptr *devPtr, size_t size, return true_cuMemAlloc_v2(devPtr, size); } - vbars_free(budget_deficit(size + CUDA_MALLOC_HEADROOM)); + stopgap_vbars_free(size + CUDA_MALLOC_HEADROOM); if (CHECK_CU(true_cuMemAlloc_v2(&dptr, size))) { *devPtr = dptr; @@ -227,7 +227,7 @@ int aimdo_cuda_malloc_async(CUdeviceptr *devPtr, size_t size, CUstream hStream, return *devPtr ? 0 : CUDA_ERROR_OUT_OF_MEMORY; } - vbars_free(budget_deficit(MIN(size, malloc_async_clamp))); + stopgap_vbars_free(MIN(size, malloc_async_clamp)); status = true_cuMemAllocAsync(&dptr, size, hStream); if (CHECK_CU(status)) { diff --git a/src/vrambuf.c b/src/vrambuf.c index 3503411..0738ac9 100644 --- a/src/vrambuf.c +++ b/src/vrambuf.c @@ -109,7 +109,7 @@ bool vrambuf_grow(void *arg, size_t required_size) { grow_to = buf->max_size; } - vbars_free(budget_deficit(grow_to - buf->allocated)); + stopgap_vbars_free(grow_to - buf->allocated); while (buf->allocated < grow_to) { size_t to_allocate = grow_to - buf->allocated; if (to_allocate > VRAM_CHUNK_SIZE) { ``` **Validation performed** - Before the fix: identical workload reproducibly hung in the described thrash state (~59W, 100% reported utilization, VRAM pinned, sustained high memory-bus activity) for 20+ minutes on every attempt, confirmed via `nvidia-smi dmon` telemetry and DEBUG-level aimdo logs cross-referenced by timestamp. - After the fix: the same oversubscribed configuration now either completes successfully or fails fast (within ~5 minutes) with a normal `torch.OutOfMemoryError` when the request genuinely doesn't fit — no thrash, no hang, process remains stable and responsive throughout both outcomes. - Not yet tested: AMD/HIP backend, integrated GPUs, multi-GPU configurations, Linux (the logic is platform-generic and the Linux-side throttle has the identical 2000ms window, but I have not run it there). This was validated against one specific oversubscribed workload/GPU, not across the project's full support matrix. **Disclosure** This investigation and patch were produced with AI assistance (Claude Code) under my direction and review — the root-cause analysis, patch design, and validation testing were done interactively with an AI agent, not purely by hand. Flagging this so you can weigh the review accordingly.

bsakaguchi referenced this issue

2026-10-26 22:24:25 +00:00

[PR #50] [MERGED] add re-registration APIs (CORE-43) #83