[GH-ISSUE #9] Could this be modified to combine multiple GPUs into a virtual one?

bsakaguchi commented

2026-10-26 22:23:23 +00:00

Owner

Originally created by @Bigfield77 on GitHub (Mar 1, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/9

Originally created by @Bigfield77 on GitHub (Mar 1, 2026). Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/9

bsakaguchi closed this issue

2026-10-26 22:23:23 +00:00

bsakaguchi commented

2026-10-26 22:23:24 +00:00

Author

Owner

@supereth commented on GitHub (Mar 1, 2026):

As an eGPU Thunderbolt Adapter and a few 1080ti's laying around owner I would be really interested if this could be even possible :)

@supereth commented on GitHub (Mar 1, 2026): As an eGPU Thunderbolt Adapter and a few 1080ti's laying around owner I would be really interested if this could be even possible :)

bsakaguchi commented

2026-10-26 22:23:24 +00:00

Author

Owner

@rattus128 commented on GitHub (Mar 2, 2026):

I'm not really seeing the direct path unfortunately. We do make heavy use of the low level Cuda mem apis that virtualize the GPU memory and these apis have modes that let gpus see each other and potentially present VRAM where you want it to look like one but that's all Cuda. We just consume it on behalf of comfy for memory management.

That said, support for multiple GPUs housing different models would be pretty easy. With that you could actually split layers across gpus but the correct way to do that is just add vbar per GPU support to aimdo then comfy does the layer split logic on the higher level.

The holy grail is the work split GPU effort where multiple GPUs do compute in parallel but that would also be best implemented on the higher level and aimdo just compliments the approach with more robust memory management

@rattus128 commented on GitHub (Mar 2, 2026): I'm not really seeing the direct path unfortunately. We do make heavy use of the low level Cuda mem apis that virtualize the GPU memory and these apis have modes that let gpus see each other and potentially present VRAM where you want it to look like one but that's all Cuda. We just consume it on behalf of comfy for memory management. That said, support for multiple GPUs housing different models would be pretty easy. With that you could actually split layers across gpus but the correct way to do that is just add vbar per GPU support to aimdo then comfy does the layer split logic on the higher level. The holy grail is the work split GPU effort where multiple GPUs do compute in parallel but that would also be best implemented on the higher level and aimdo just compliments the approach with more robust memory management

bsakaguchi commented

2026-10-26 22:23:24 +00:00

Author

Owner

@Bigfield77 commented on GitHub (Mar 3, 2026):

Thanks for looking into this!

Concerning "That said, support for multiple GPUs housing different models would be pretty easy. With that you could actually split layers across gpus but the correct way to do that is just add vbar per GPU support"

You mean, some mod to that code would allow me to pass a device_map ie {layer_1:0, ... layer_15:1.. } and it would overflow to system ram automatically when each device reaches max vram..? that would be pretty great if i understand correctly!

Windows nvidia driver have something like this but it only works when one device is exposed.

@Bigfield77 commented on GitHub (Mar 3, 2026): Thanks for looking into this! Concerning "That said, support for multiple GPUs housing different models would be pretty easy. With that you could actually split layers across gpus but the correct way to do that is just add vbar per GPU support" You mean, some mod to that code would allow me to pass a device_map ie {layer_1:0, ... layer_15:1.. } and it would overflow to system ram automatically when each device reaches max vram..? that would be pretty great if i understand correctly! Windows nvidia driver have something like this but it only works when one device is exposed.

bsakaguchi commented

2026-10-26 22:23:24 +00:00

Author

Owner

@rattus128 commented on GitHub (Mar 4, 2026):

Thanks for looking into this!

Concerning "That said, support for multiple GPUs housing different models would be pretty easy. With that you could actually split layers across gpus but the correct way to do that is just add vbar per GPU support"

You mean, some mod to that code would allow me to pass a device_map ie {layer_1:0, ... layer_15:1.. } and it would overflow to system ram automatically when each device reaches max vram..? that would be pretty great if i understand correctly!

We don't implement any layer awarness is aimdo, but yes you can alloc your layers exactly like that into the VBARs on the application level. But you can actually do something fancier than that. VBAR space is free, so what you can do is allocate the entire model to both GPUs, (this doesnt actually use the VRAM). You can then setup a priority order between the two GPUs:

if fault(layer.alloc1, GPU1):
    #use is as per aimdo on GPU1
    input.to(GPU1)
elif fault(layer.alloc2, GPU2):
    #use is as per aimdo on GPU1
    input.to(GPU2)
else:
    #fully offload, use CPU or demand page the weights to wherever input is

You need to order your layers in the VBARs in usage order for it to be effective and avoid back-and-forthing the input, but you already have layer order awareness so you are good on that front.

This will automatically balance your layers accross the two GPUs without you have to pre-allocate (this is what Aimdo is all about!).

Windows nvidia driver have something like this but it only works when one device is exposed.

The Windows Nvidia drivers are ineffective as WDDM doesnt have a good awareness of fast vs slow moving VRAM which is also part of the problem.

@rattus128 commented on GitHub (Mar 4, 2026): > Thanks for looking into this! > > Concerning "That said, support for multiple GPUs housing different models would be pretty easy. With that you could actually split layers across gpus but the correct way to do that is just add vbar per GPU support" > > You mean, some mod to that code would allow me to pass a device_map ie {layer_1:0, ... layer_15:1.. } and it would overflow to system ram automatically when each device reaches max vram..? that would be pretty great if i understand correctly! > We don't implement any layer awarness is aimdo, but yes you can alloc your layers exactly like that into the VBARs on the application level. But you can actually do something fancier than that. VBAR space is free, so what you can do is allocate the entire model to both GPUs, (this doesnt actually use the VRAM). You can then setup a priority order between the two GPUs: ``` if fault(layer.alloc1, GPU1): #use is as per aimdo on GPU1 input.to(GPU1) elif fault(layer.alloc2, GPU2): #use is as per aimdo on GPU1 input.to(GPU2) else: #fully offload, use CPU or demand page the weights to wherever input is ``` You need to order your layers in the VBARs in usage order for it to be effective and avoid back-and-forthing the input, but you already have layer order awareness so you are good on that front. This will automatically balance your layers accross the two GPUs without you have to pre-allocate (this is what Aimdo is all about!). > Windows nvidia driver have something like this but it only works when one device is exposed. The Windows Nvidia drivers are ineffective as WDDM doesnt have a good awareness of fast vs slow moving VRAM which is also part of the problem.

bsakaguchi commented

2026-10-26 22:23:24 +00:00

Author

Owner

@Bigfield77 commented on GitHub (Mar 5, 2026):

Thanks, i will try that!

@Bigfield77 commented on GitHub (Mar 5, 2026): Thanks, i will try that!

bsakaguchi commented

2026-10-26 22:23:24 +00:00

Author

Owner

@rattus128 commented on GitHub (Mar 5, 2026):

want to just leave it open and keep me posted? :)

@rattus128 commented on GitHub (Mar 5, 2026): want to just leave it open and keep me posted? :)

bsakaguchi commented

2026-10-26 22:23:25 +00:00

Author

Owner

@Bigfield77 commented on GitHub (Mar 5, 2026):

will do!

@Bigfield77 commented on GitHub (Mar 5, 2026): will do!

bsakaguchi commented

2026-10-26 22:23:25 +00:00

Author

Owner

@Bigfield77 commented on GitHub (Mar 11, 2026):

Actually,

I was looking at this for a custom node used to run hunyuan image 3 instruct, it is using non-comfy loader.

Transforming it to comfy compatible is well beyond my skill :)

@Bigfield77 commented on GitHub (Mar 11, 2026): Actually, I was looking at this for a custom node used to run hunyuan image 3 instruct, it is using non-comfy loader. Transforming it to comfy compatible is well beyond my skill :)

bsakaguchi referenced this issue

2026-10-26 22:23:33 +00:00

[GH-ISSUE #29] HIP/ROCm support for AMD GPUs — working port on RX 9070 (gfx1201) #11

bsakaguchi referenced this issue

2026-10-26 22:23:33 +00:00

[GH-ISSUE #29] HIP/ROCm support for AMD GPUs — working port on RX 9070 (gfx1201) #11

bsakaguchi referenced this issue

2026-10-26 22:23:40 +00:00

[GH-ISSUE #61] hostbuf_file_reader_read failed with all mixed precision quantized video models on RTX 5090 Laptop #17

bsakaguchi referenced this issue

2026-10-26 22:24:09 +00:00

[PR #2] [CLOSED] [DRAFT] Barebones ROCM support #53