[PR #20] [MERGED] allocators: Dont use the VMA allocator for cuda_malloc #65

Closed
opened 2026-10-26 22:24:15 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-aimdo/pull/20
Author: @rattus128
Created: 3/11/2026
Status: ✅ Merged
Merged: 3/11/2026
Merged by: @rattus128

Base: master ← Head: dev/vanilla-cuda-malloc


📝 Commits (1)

  • 1c6fa0d allocators: Dont use the VMA allocator for cuda_malloc

📊 Changes

4 files changed (+125 additions, -72 deletions)

View changed files

📝 src-win/cuda-detour.c (+2 -2)
📝 src/plat.h (+4 -6)
📝 src/pyt-cu-plug-alloc-async.c (+119 -42)
📝 src/pyt-cu-plug-alloc.c (+0 -22)

📄 Description

This has some closed-source synchronization semantics so its actually impossible to self implement safely. Its subject to more fragmentation than the main allocator, but just add a little bit of extra headroom. Rejig the code to make the linux wrappers the same as for the async allocators.

Contribution Agreement

  • [ X ] I agree that my contributions are licensed under the GPLv3.
  • [ X ] I grant Comfy Org the rights to relicense these contributions as outlined in CONTRIBUTING.md.

Example Test Conditions:

This test case (vibe code):
foo.py

Before:

(venv) rattus@rattus-box2:~$ pip install comfy-aimdo --force-reinstall
Collecting comfy-aimdo
  Using cached comfy_aimdo-0.2.9-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl.metadata (4.2 kB)
Using cached comfy_aimdo-0.2.9-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl (82 kB)
Installing collected packages: comfy-aimdo
  Attempting uninstall: comfy-aimdo
    Found existing installation: comfy-aimdo 0.2.9
    Uninstalling comfy-aimdo-0.2.9:
      Successfully uninstalled comfy-aimdo-0.2.9
Successfully installed comfy-aimdo-0.2.9
(venv) rattus@rattus-box2:~$ python ./foo.py 
aimdo: src/control.c:68:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce GTX 1660 SUPER (VRAM: 5746 MB)
using aimdo cudaMalloc @ 0x77beb4f90000
using aimdo cudaFree   @ 0x77beb4f90400
free=4879 MiB, test_size=3072 MiB
aimdo: src/pyt-cu-plug-alloc.c:42:VERBOSE:alloc_fn (start): size=3145728k, device=0
aimdo: src/pyt-cu-plug-alloc.c:59:VERBOSE:alloc_fn (return): ptr=0x302000000
aimdo: src/pyt-cu-plug-alloc.c:78:DEBUG:Pytorch is freeing VRAM ...
aimdo: src/pyt-cu-plug-alloc.c:79:VERBOSE:free_fn (start) ptr=0x302000000 size=0k, device=0
aimdo: src/pyt-cu-plug-alloc.c:92:VERBOSE:Freed: ptr=0x302000000, size=0k, stream=(nil)
FAIL sync iter=0, free_ms=7.464: rc=700 (an illegal memory access was encountered)

After:

(venv) rattus@rattus-box2:~$ pip install ~/Downloads/wheels-ubuntu-latest/comfy_aimdo-0.2.10.dev1-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl 
Processing ./Downloads/wheels-ubuntu-latest/comfy_aimdo-0.2.10.dev1-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl
Installing collected packages: comfy-aimdo
  Attempting uninstall: comfy-aimdo
    Found existing installation: comfy-aimdo 0.2.9
    Uninstalling comfy-aimdo-0.2.9:
      Successfully uninstalled comfy-aimdo-0.2.9
Successfully installed comfy-aimdo-0.2.10.dev1
(venv) rattus@rattus-box2:~$ python ./foo.py 
aimdo: src/control.c:68:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce GTX 1660 SUPER (VRAM: 5746 MB)
using aimdo cudaMalloc @ 0x7f3363383fa0
using aimdo cudaFree   @ 0x7f3363384020
free=4877 MiB, test_size=3072 MiB
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=0, free_ms=16.203
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=1, free_ms=16.532
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=2, free_ms=17.337
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=3, free_ms=17.890
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=4, free_ms=17.183
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=5, free_ms=17.277
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=6, free_ms=17.316
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=7, free_ms=17.367
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=8, free_ms=17.327
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil)
ok iter=9, free_ms=17.217
PASS

Windows:

before:

(venvw) PS C:\users\rattus> python .\foo.py
aimdo: src-win/cuda-detour.c:77:INFO:aimdo_setup_hooks: found driver at 00007FFDF1990000, installing 4 hooks
aimdo: src-win/cuda-detour.c:61:DEBUG:install_hook_entrys: hooks successfully installed
aimdo: src/control.c:69:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 5060 (VRAM: 8150 MB)
using driver cuMemAlloc_v2 @ 0x7ffdf3f09e12
using driver cuMemFree_v2  @ 0x7ffdf3f09e30
free=7020 MiB, test_size=4352 MiB
aimdo: src/pyt-cu-plug-alloc.c:42:VERBOSE:alloc_fn (start): size=4456448k, device=0
aimdo: src/pyt-cu-plug-alloc.c:59:VERBOSE:alloc_fn (return): ptr=0000000204C00000
aimdo: src/pyt-cu-plug-alloc.c:78:DEBUG:Pytorch is freeing VRAM ...
aimdo: src/pyt-cu-plug-alloc.c:79:VERBOSE:free_fn (start) ptr=0000000204C00000 size=0k, device=0
aimdo: src/pyt-cu-plug-alloc.c:92:VERBOSE:Freed: ptr=0000000204C00000, size=0k, stream=0000000000000000
FAIL sync iter=0, free_ms=107.959: rc=700 (an illegal memory access was encountered)
aimdo: src-win/cuda-detour.c:99:DEBUG:aimdo_teardown_hooks: hooks successfully removed

after:

(venvw) PS C:\users\rattus> python .\foo.py
aimdo: src-win/cuda-detour.c:77:INFO:aimdo_setup_hooks: found driver at 00007FFDF1990000, installing 4 hooks
aimdo: src-win/cuda-detour.c:61:DEBUG:install_hook_entrys: hooks successfully installed
aimdo: src/control.c:69:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 5060 (VRAM: 8150 MB)
using driver cuMemAlloc_v2 @ 0x7ffdf3f09e12
using driver cuMemFree_v2  @ 0x7ffdf3f09e30
free=7020 MiB, test_size=4352 MiB
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=0, free_ms=98.138
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=1, free_ms=98.444
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=2, free_ms=98.665
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=3, free_ms=94.875
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=4, free_ms=93.728
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=5, free_ms=94.744
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=6, free_ms=93.978
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=7, free_ms=94.496
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=8, free_ms=93.804
aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000
ok iter=9, free_ms=95.184
PASS
aimdo: src-win/cuda-detour.c:99:DEBUG:aimdo_teardown_hooks: hooks successfully removed

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-aimdo/pull/20 **Author:** [@rattus128](https://github.com/rattus128) **Created:** 3/11/2026 **Status:** ✅ Merged **Merged:** 3/11/2026 **Merged by:** [@rattus128](https://github.com/rattus128) **Base:** `master` ← **Head:** `dev/vanilla-cuda-malloc` --- ### 📝 Commits (1) - [`1c6fa0d`](https://github.com/Comfy-Org/comfy-aimdo/commit/1c6fa0d889a9d51aba5ce88aa651cf3462a543e6) allocators: Dont use the VMA allocator for cuda_malloc ### 📊 Changes **4 files changed** (+125 additions, -72 deletions) <details> <summary>View changed files</summary> 📝 `src-win/cuda-detour.c` (+2 -2) 📝 `src/plat.h` (+4 -6) 📝 `src/pyt-cu-plug-alloc-async.c` (+119 -42) 📝 `src/pyt-cu-plug-alloc.c` (+0 -22) </details> ### 📄 Description This has some closed-source synchronization semantics so its actually impossible to self implement safely. Its subject to more fragmentation than the main allocator, but just add a little bit of extra headroom. Rejig the code to make the linux wrappers the same as for the async allocators. ### Contribution Agreement - [ X ] I agree that my contributions are licensed under the GPLv3. - [ X ] I grant **Comfy Org** the rights to relicense these contributions as outlined in [CONTRIBUTING.md](./CONTRIBUTING.md). Example Test Conditions: This test case (vibe code): [foo.py](https://github.com/user-attachments/files/25905435/foo.py) Before: ``` (venv) rattus@rattus-box2:~$ pip install comfy-aimdo --force-reinstall Collecting comfy-aimdo Using cached comfy_aimdo-0.2.9-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl.metadata (4.2 kB) Using cached comfy_aimdo-0.2.9-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl (82 kB) Installing collected packages: comfy-aimdo Attempting uninstall: comfy-aimdo Found existing installation: comfy-aimdo 0.2.9 Uninstalling comfy-aimdo-0.2.9: Successfully uninstalled comfy-aimdo-0.2.9 Successfully installed comfy-aimdo-0.2.9 (venv) rattus@rattus-box2:~$ python ./foo.py aimdo: src/control.c:68:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce GTX 1660 SUPER (VRAM: 5746 MB) using aimdo cudaMalloc @ 0x77beb4f90000 using aimdo cudaFree @ 0x77beb4f90400 free=4879 MiB, test_size=3072 MiB aimdo: src/pyt-cu-plug-alloc.c:42:VERBOSE:alloc_fn (start): size=3145728k, device=0 aimdo: src/pyt-cu-plug-alloc.c:59:VERBOSE:alloc_fn (return): ptr=0x302000000 aimdo: src/pyt-cu-plug-alloc.c:78:DEBUG:Pytorch is freeing VRAM ... aimdo: src/pyt-cu-plug-alloc.c:79:VERBOSE:free_fn (start) ptr=0x302000000 size=0k, device=0 aimdo: src/pyt-cu-plug-alloc.c:92:VERBOSE:Freed: ptr=0x302000000, size=0k, stream=(nil) FAIL sync iter=0, free_ms=7.464: rc=700 (an illegal memory access was encountered) ``` After: ``` (venv) rattus@rattus-box2:~$ pip install ~/Downloads/wheels-ubuntu-latest/comfy_aimdo-0.2.10.dev1-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl Processing ./Downloads/wheels-ubuntu-latest/comfy_aimdo-0.2.10.dev1-cp39-abi3-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl Installing collected packages: comfy-aimdo Attempting uninstall: comfy-aimdo Found existing installation: comfy-aimdo 0.2.9 Uninstalling comfy-aimdo-0.2.9: Successfully uninstalled comfy-aimdo-0.2.9 Successfully installed comfy-aimdo-0.2.10.dev1 (venv) rattus@rattus-box2:~$ python ./foo.py aimdo: src/control.c:68:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce GTX 1660 SUPER (VRAM: 5746 MB) using aimdo cudaMalloc @ 0x7f3363383fa0 using aimdo cudaFree @ 0x7f3363384020 free=4877 MiB, test_size=3072 MiB aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=0, free_ms=16.203 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=1, free_ms=16.532 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=2, free_ms=17.337 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=3, free_ms=17.890 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=4, free_ms=17.183 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=5, free_ms=17.277 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=6, free_ms=17.316 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=7, free_ms=17.367 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=8, free_ms=17.327 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0x7f3280000000, size=3145728k, stream=(nil) ok iter=9, free_ms=17.217 PASS ``` Windows: before: ``` (venvw) PS C:\users\rattus> python .\foo.py aimdo: src-win/cuda-detour.c:77:INFO:aimdo_setup_hooks: found driver at 00007FFDF1990000, installing 4 hooks aimdo: src-win/cuda-detour.c:61:DEBUG:install_hook_entrys: hooks successfully installed aimdo: src/control.c:69:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 5060 (VRAM: 8150 MB) using driver cuMemAlloc_v2 @ 0x7ffdf3f09e12 using driver cuMemFree_v2 @ 0x7ffdf3f09e30 free=7020 MiB, test_size=4352 MiB aimdo: src/pyt-cu-plug-alloc.c:42:VERBOSE:alloc_fn (start): size=4456448k, device=0 aimdo: src/pyt-cu-plug-alloc.c:59:VERBOSE:alloc_fn (return): ptr=0000000204C00000 aimdo: src/pyt-cu-plug-alloc.c:78:DEBUG:Pytorch is freeing VRAM ... aimdo: src/pyt-cu-plug-alloc.c:79:VERBOSE:free_fn (start) ptr=0000000204C00000 size=0k, device=0 aimdo: src/pyt-cu-plug-alloc.c:92:VERBOSE:Freed: ptr=0000000204C00000, size=0k, stream=0000000000000000 FAIL sync iter=0, free_ms=107.959: rc=700 (an illegal memory access was encountered) aimdo: src-win/cuda-detour.c:99:DEBUG:aimdo_teardown_hooks: hooks successfully removed ``` after: ``` (venvw) PS C:\users\rattus> python .\foo.py aimdo: src-win/cuda-detour.c:77:INFO:aimdo_setup_hooks: found driver at 00007FFDF1990000, installing 4 hooks aimdo: src-win/cuda-detour.c:61:DEBUG:install_hook_entrys: hooks successfully installed aimdo: src/control.c:69:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 5060 (VRAM: 8150 MB) using driver cuMemAlloc_v2 @ 0x7ffdf3f09e12 using driver cuMemFree_v2 @ 0x7ffdf3f09e30 free=7020 MiB, test_size=4352 MiB aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=0, free_ms=98.138 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=1, free_ms=98.444 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=2, free_ms=98.665 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=3, free_ms=94.875 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=4, free_ms=93.728 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=5, free_ms=94.744 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=6, free_ms=93.978 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=7, free_ms=94.496 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=8, free_ms=93.804 aimdo: src/pyt-cu-plug-alloc-async.c:72:VVERBOSE:Freed: ptr=0xb06800000, size=4456448k, stream=0000000000000000 ok iter=9, free_ms=95.184 PASS aimdo: src-win/cuda-detour.c:99:DEBUG:aimdo_teardown_hooks: hooks successfully removed ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-26 22:24:15 +00:00
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Comfy-Org/comfy-aimdo#65
No description provided.