[PR #34] [MERGED] Dev/multi gpu 2 (CORE-21) #72

New issue

Closed

opened 2026-10-26 22:24:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:24:19 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-aimdo/pull/34
Author: @rattus128
Created: 4/16/2026
Status: ✅ Merged
Merged: 4/16/2026
Merged by: @rattus128

Base: master ← Head: dev/multi-gpu-2

📝 Commits (10+)

715081b eliminate global var prevailing_deficit method.
2fb5998 poll: make last_check a regular global
fe62ed3 plat: Make shot_counter thread local
8cb4b96 shmem-detect: local WDDM factory
283d65e control: Split off global init to plat_init()
2190a28 fix compile warnings
4e29e5a pyt-cu-plug-alloc-async: init lock properly
36ebd15 Multi GPU handling
45a57f4 src-win: Add compat shim for MSVC TLS
a0c5aaf control: join context to device

📊 Changes

19 files changed (+634 additions, -258 deletions)

View changed files

📝 .github/workflows/build-wheels.yml (+2 -6)
📝 build-linux-docker (+2 -5)
📝 comfy_aimdo/control.py (+43 -7)
📝 comfy_aimdo/model_vbar.py (+38 -33)
📝 docker/cuda-on-manylinux.Dockerfile (+1 -0)
➕ scripts/build-linux-aimdo.sh (+53 -0)
➕ src-posix/cuda-funchooks.c (+89 -0)
📝 src-posix/model-mmap.c (+2 -1)
➕ src-win/compiler.h (+5 -0)
📝 src-win/cuda-detour.c (+1 -34)
📝 src-win/model-mmap.c (+2 -1)
📝 src-win/shmem-detect.c (+24 -22)
📝 src/control.c (+104 -32)
➕ src/control.h (+66 -0)
➕ src/cuda-hooks-shared.h (+54 -0)
📝 src/model-vbar.c (+49 -20)
📝 src/plat.h (+14 -28)
📝 src/pyt-cu-plug-alloc-async.c (+80 -65)
📝 src/pyt-cu-plug-alloc.c (+5 -4)

📄 Description

Contribution Agreement

[ X ] I agree that my contributions are licensed under the GPLv3.
[ X ] I grant Comfy Org the rights to relicense these contributions as outlined in CONTRIBUTING.md.

Two major features:

Linux funchooks.
Proper multi-GPU support.

The two issues are coupled as we use the funchooks to properly get rid of cudart on linux which in turns lets usscrap the poorly defined ensure_ctx which only gets worse with multi-gpu.

Aimdo is now inited with a list of cuda devices. analyze_vbars() is device specific.
GPU accessing Aimdo C APIs now all accept an AimdoContext which they must enter to set globally via TLS. This leaves the bulk of the code unchanged and the GPU context texually global while TLS implements the context switching.
Non GPU APIs like the hostmem stuff are un-affected and don't need the AimdoContext.
Linux no longer uses RTLD_GLOBAL to hook cuda, instead it uses funchooks to implement the same trampoline system as detours does on windows. This allows a more consistent lower level hook.
Cudart is fully removed and permanently defeatured/removed from aimdo dependencies.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-aimdo/pull/34 **Author:** [@rattus128](https://github.com/rattus128) **Created:** 4/16/2026 **Status:** ✅ Merged **Merged:** 4/16/2026 **Merged by:** [@rattus128](https://github.com/rattus128) **Base:** `master` ← **Head:** `dev/multi-gpu-2` --- ### 📝 Commits (10+) - [`715081b`](https://github.com/Comfy-Org/comfy-aimdo/commit/715081bb53d79fab1d829b3f489d62a222f6ab92) eliminate global var prevailing_deficit method. - [`2fb5998`](https://github.com/Comfy-Org/comfy-aimdo/commit/2fb59986132c30940cac152525dd0329252e5480) poll: make last_check a regular global - [`fe62ed3`](https://github.com/Comfy-Org/comfy-aimdo/commit/fe62ed3fa1037955b3a011c7fbf79183fc87ace3) plat: Make shot_counter thread local - [`8cb4b96`](https://github.com/Comfy-Org/comfy-aimdo/commit/8cb4b96a0258a37401b5b53e9ae801f2bf03dc18) shmem-detect: local WDDM factory - [`283d65e`](https://github.com/Comfy-Org/comfy-aimdo/commit/283d65e046cc7f4f9ef08a262a0342fde265f2b5) control: Split off global init to plat_init() - [`2190a28`](https://github.com/Comfy-Org/comfy-aimdo/commit/2190a28f7674fe4675ee5d39e51ad55143c693b1) fix compile warnings - [`4e29e5a`](https://github.com/Comfy-Org/comfy-aimdo/commit/4e29e5a06dc71ac7d637db3a5f7c10a4235b36d3) pyt-cu-plug-alloc-async: init lock properly - [`36ebd15`](https://github.com/Comfy-Org/comfy-aimdo/commit/36ebd155c2a6be1636002ae712dd38f647092007) Multi GPU handling - [`45a57f4`](https://github.com/Comfy-Org/comfy-aimdo/commit/45a57f4eb281a00ad44f5cefed32b0973b594bf9) src-win: Add compat shim for MSVC TLS - [`a0c5aaf`](https://github.com/Comfy-Org/comfy-aimdo/commit/a0c5aaf4fca57178d9aab822512e10cd89d8485e) control: join context to device ### 📊 Changes **19 files changed** (+634 additions, -258 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/build-wheels.yml` (+2 -6) 📝 `build-linux-docker` (+2 -5) 📝 `comfy_aimdo/control.py` (+43 -7) 📝 `comfy_aimdo/model_vbar.py` (+38 -33) 📝 `docker/cuda-on-manylinux.Dockerfile` (+1 -0) ➕ `scripts/build-linux-aimdo.sh` (+53 -0) ➕ `src-posix/cuda-funchooks.c` (+89 -0) 📝 `src-posix/model-mmap.c` (+2 -1) ➕ `src-win/compiler.h` (+5 -0) 📝 `src-win/cuda-detour.c` (+1 -34) 📝 `src-win/model-mmap.c` (+2 -1) 📝 `src-win/shmem-detect.c` (+24 -22) 📝 `src/control.c` (+104 -32) ➕ `src/control.h` (+66 -0) ➕ `src/cuda-hooks-shared.h` (+54 -0) 📝 `src/model-vbar.c` (+49 -20) 📝 `src/plat.h` (+14 -28) 📝 `src/pyt-cu-plug-alloc-async.c` (+80 -65) 📝 `src/pyt-cu-plug-alloc.c` (+5 -4) </details> ### 📄 Description ### Contribution Agreement - [ X ] I agree that my contributions are licensed under the GPLv3. - [ X ] I grant **Comfy Org** the rights to relicense these contributions as outlined in [CONTRIBUTING.md](./CONTRIBUTING.md). Two major features: Linux funchooks. Proper multi-GPU support. The two issues are coupled as we use the funchooks to properly get rid of cudart on linux which in turns lets usscrap the poorly defined ensure_ctx which only gets worse with multi-gpu. * Aimdo is now inited with a list of cuda devices. analyze_vbars() is device specific. * GPU accessing Aimdo C APIs now all accept an AimdoContext which they must enter to set globally via TLS. This leaves the bulk of the code unchanged and the GPU context texually global while TLS implements the context switching. * Non GPU APIs like the hostmem stuff are un-affected and don't need the AimdoContext. * Linux no longer uses RTLD_GLOBAL to hook cuda, instead it uses funchooks to implement the same trampoline system as detours does on windows. This allows a more consistent lower level hook. * Cudart is fully removed and permanently defeatured/removed from aimdo dependencies. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:24:19 +00:00