Comfy-Org/ComfyUI_Nvidia_VFX_Nodes

Fork 0

mirror of https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes.git synced 2026-11-04 23:37:30 +00:00

[GH-ISSUE #28] NvVFX SDK does not support multiple effect types across GPUs simultaneously #22

New issue

Open

opened 2026-10-26 22:28:06 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:28:06 +00:00

Owner

Originally created by @bencoster on GitHub (Apr 1, 2026).
Original GitHub issue: https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes/issues/28

Summary

The NVIDIA Video Effects (NvVFX) SDK has a design limitation where GPU selection (NvVFX_SetS32(NULL, NVVFX_GPU, whichGPU)) is a global, one-time setting — not per-instance. Running multiple effect types (e.g., denoise + super-resolution + deblur) simultaneously across multiple GPUs causes NvVFX_Run to fail with error code -7 ("invalid parameter value") or -1 ("unspecified error") on all GPUs except the first.

This affects any multi-GPU implementation that creates denoise, SR, and deblur instances on different devices concurrently.

Environment

NvVFX SDK: v1.2.0 (nvvfx Python package)
GPUs: 1× RTX 4090 (SM 8.9) + 3× RTX 3090 (SM 8.6)
Driver: 591.86
OS: Windows 11
PyTorch: 2.10.0+cu130

Reproduction

Works ✅ — Single effect type across all GPUs concurrently

import torch, nvvfx, concurrent.futures
from nvvfx.effects import QualityLevel

# SR-only: all 4 GPUs succeed
instances = {}
for gpu_id in range(4):
    with torch.cuda.device(gpu_id):
        sr = nvvfx.VideoSuperRes(QualityLevel.HIGH, device=gpu_id)
        sr.output_width, sr.output_height = 856, 480
        sr.load()
        instances[gpu_id] = sr

def worker(gpu_id):
    frame = torch.rand(3, 240, 428, device=f"cuda:{gpu_id}", dtype=torch.float32)
    torch.cuda.set_device(gpu_id)
    out = instances[gpu_id].run(frame)
    return torch.from_dlpack(out.image).clone()

with concurrent.futures.ThreadPoolExecutor(max_workers=4) as ex:
    futures = {ex.submit(worker, g): g for g in range(4)}
    for f in concurrent.futures.as_completed(futures):
        print(f"GPU {futures[f]}: OK")  # All 4 succeed

Works ✅ — Full pipeline on a single GPU

# denoise → SR → deblur on one GPU: works perfectly
dn = nvvfx.VideoSuperRes(QualityLevel.DENOISE_MEDIUM, device=1)
sr = nvvfx.VideoSuperRes(QualityLevel.HIGH, device=1)
db = nvvfx.VideoSuperRes(QualityLevel.DEBLUR_MEDIUM, device=1)
# ... load all, run sequentially — all succeed

Fails ❌ — Multiple effect types across GPUs concurrently

# Create denoise + SR + deblur on each GPU, run concurrently
instances = {}
for gpu_id in range(4):
    with torch.cuda.device(gpu_id):
        dn = nvvfx.VideoSuperRes(QualityLevel.DENOISE_MEDIUM, device=gpu_id)
        dn.output_width, dn.output_height = 432, 240
        dn.load()
        sr = nvvfx.VideoSuperRes(QualityLevel.HIGH, device=gpu_id)
        sr.output_width, sr.output_height = 856, 480
        sr.load()
        db = nvvfx.VideoSuperRes(QualityLevel.DEBLUR_MEDIUM, device=gpu_id)
        db.output_width, db.output_height = 856, 480
        db.load()
    instances[gpu_id] = (dn, sr, db)

# Run pipeline on all GPUs concurrently → GPU 0 succeeds, GPUs 1-3 fail
# Error: "NvVFX_Run failed: An invalid parameter value (code -7)"
#    or: "An otherwise unspecified error (code -1)"

Also fails ❌ — Even with serialized `run()` calls (lock)

run_lock = threading.Lock()
# Same setup, but wrap each run() in: with run_lock: inst.run(frame)
# Still fails — proving it's not a race condition during execution

Also fails ❌ — Even with identical GPUs (3× RTX 3090 only)

# 3× RTX 3090, no 4090 involved:
# GPU 1: OK, GPUs 2-3: FAIL
# Only the first GPU to start processing succeeds

Root Cause

The NvVFX SDK Programming Guide states:

NvVFX_SetS32(NULL, NVVFX_GPU, whichGPU) is intended to be called only once for the Video Effects SDK before any effects are created.

Images that are allocated on one GPU cannot be transparently passed to another GPU, so you must ensure that the same GPU is used for all video effects.

GPU selection is a global SDK-wide state, not per-instance. When creating instances on different GPUs, each creation overwrites the global GPU setting. The TensorRT engines for different effect types end up internally bound to the wrong GPU context, causing NvVFX_Run to fail.

An unanswered forum post from another developer asked the same question about multiple CUDA contexts — no response from NVIDIA.

Workaround: Phased Processing

Process one effect type at a time across all GPUs — create instances, process all frames, close instances, then move to the next effect type:

Phase 1: Create denoise on all GPUs → process all frames → close all denoise
Phase 2: Create SR on all GPUs → process all frames → close all SR
Phase 3: Create deblur on all GPUs → process all frames → close all deblur

This ensures only one effect type exists globally at any time, matching the SDK's single-GPU design. Intermediate results are stored in CPU tensors between phases.

Performance cost: ~1-2s fixed overhead per run (3× instance creation/TensorRT engine loading instead of 1×), plus 2× additional CPU↔GPU data transfer for intermediate frames. For small frames (e.g., 428×240) and 150+ frame batches, the overhead is modest (~5-10% of total processing time).

Tested Confirmation

Scenario	Result
SR-only, 4 GPUs concurrent	✅ All GPUs OK
Deblur-only, 4 GPUs concurrent	✅ All GPUs OK
Denoise-only, 4 GPUs concurrent	✅ All GPUs OK
Denoise+SR, 4 GPUs concurrent	❌ Only first GPU OK
Full pipeline, single GPU	✅ OK
Full pipeline, phased approach	✅ All GPUs OK
Full pipeline with lock around run()	❌ Still fails

Originally created by @bencoster on GitHub (Apr 1, 2026). Original GitHub issue: https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes/issues/28 ## Summary The NVIDIA Video Effects (NvVFX) SDK has a **design limitation** where GPU selection (`NvVFX_SetS32(NULL, NVVFX_GPU, whichGPU)`) is a **global, one-time setting** — not per-instance. Running multiple effect types (e.g., denoise + super-resolution + deblur) simultaneously across multiple GPUs causes `NvVFX_Run` to fail with error code **-7** ("invalid parameter value") or **-1** ("unspecified error") on all GPUs except the first. This affects any multi-GPU implementation that creates denoise, SR, and deblur instances on different devices concurrently. ## Environment - **NvVFX SDK**: v1.2.0 (`nvvfx` Python package) - **GPUs**: 1× RTX 4090 (SM 8.9) + 3× RTX 3090 (SM 8.6) - **Driver**: 591.86 - **OS**: Windows 11 - **PyTorch**: 2.10.0+cu130 ## Reproduction ### Works ✅ — Single effect type across all GPUs concurrently ```python import torch, nvvfx, concurrent.futures from nvvfx.effects import QualityLevel # SR-only: all 4 GPUs succeed instances = {} for gpu_id in range(4): with torch.cuda.device(gpu_id): sr = nvvfx.VideoSuperRes(QualityLevel.HIGH, device=gpu_id) sr.output_width, sr.output_height = 856, 480 sr.load() instances[gpu_id] = sr def worker(gpu_id): frame = torch.rand(3, 240, 428, device=f"cuda:{gpu_id}", dtype=torch.float32) torch.cuda.set_device(gpu_id) out = instances[gpu_id].run(frame) return torch.from_dlpack(out.image).clone() with concurrent.futures.ThreadPoolExecutor(max_workers=4) as ex: futures = {ex.submit(worker, g): g for g in range(4)} for f in concurrent.futures.as_completed(futures): print(f"GPU {futures[f]}: OK") # All 4 succeed ``` ### Works ✅ — Full pipeline on a single GPU ```python # denoise → SR → deblur on one GPU: works perfectly dn = nvvfx.VideoSuperRes(QualityLevel.DENOISE_MEDIUM, device=1) sr = nvvfx.VideoSuperRes(QualityLevel.HIGH, device=1) db = nvvfx.VideoSuperRes(QualityLevel.DEBLUR_MEDIUM, device=1) # ... load all, run sequentially — all succeed ``` ### Fails ❌ — Multiple effect types across GPUs concurrently ```python # Create denoise + SR + deblur on each GPU, run concurrently instances = {} for gpu_id in range(4): with torch.cuda.device(gpu_id): dn = nvvfx.VideoSuperRes(QualityLevel.DENOISE_MEDIUM, device=gpu_id) dn.output_width, dn.output_height = 432, 240 dn.load() sr = nvvfx.VideoSuperRes(QualityLevel.HIGH, device=gpu_id) sr.output_width, sr.output_height = 856, 480 sr.load() db = nvvfx.VideoSuperRes(QualityLevel.DEBLUR_MEDIUM, device=gpu_id) db.output_width, db.output_height = 856, 480 db.load() instances[gpu_id] = (dn, sr, db) # Run pipeline on all GPUs concurrently → GPU 0 succeeds, GPUs 1-3 fail # Error: "NvVFX_Run failed: An invalid parameter value (code -7)" # or: "An otherwise unspecified error (code -1)" ``` ### Also fails ❌ — Even with serialized `run()` calls (lock) ```python run_lock = threading.Lock() # Same setup, but wrap each run() in: with run_lock: inst.run(frame) # Still fails — proving it's not a race condition during execution ``` ### Also fails ❌ — Even with identical GPUs (3× RTX 3090 only) ```python # 3× RTX 3090, no 4090 involved: # GPU 1: OK, GPUs 2-3: FAIL # Only the first GPU to start processing succeeds ``` ## Root Cause The [NvVFX SDK Programming Guide](https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-programming-guide/index.html) states: > `NvVFX_SetS32(NULL, NVVFX_GPU, whichGPU)` **is intended to be called only once** for the Video Effects SDK before any effects are created. > Images that are allocated on one GPU cannot be transparently passed to another GPU, so **you must ensure that the same GPU is used for all video effects.** GPU selection is a **global SDK-wide state**, not per-instance. When creating instances on different GPUs, each creation overwrites the global GPU setting. The TensorRT engines for different effect types end up internally bound to the wrong GPU context, causing `NvVFX_Run` to fail. An [unanswered forum post](https://forums.developer.nvidia.com/t/maxine-videofx-on-multiple-cuda-context/228998) from another developer asked the same question about multiple CUDA contexts — no response from NVIDIA. ## Workaround: Phased Processing Process **one effect type at a time** across all GPUs — create instances, process all frames, close instances, then move to the next effect type: ``` Phase 1: Create denoise on all GPUs → process all frames → close all denoise Phase 2: Create SR on all GPUs → process all frames → close all SR Phase 3: Create deblur on all GPUs → process all frames → close all deblur ``` This ensures only one effect type exists globally at any time, matching the SDK's single-GPU design. Intermediate results are stored in CPU tensors between phases. **Performance cost**: ~1-2s fixed overhead per run (3× instance creation/TensorRT engine loading instead of 1×), plus 2× additional CPU↔GPU data transfer for intermediate frames. For small frames (e.g., 428×240) and 150+ frame batches, the overhead is modest (~5-10% of total processing time). ## Tested Confirmation | Scenario | Result | |---|---| | SR-only, 4 GPUs concurrent | ✅ All GPUs OK | | Deblur-only, 4 GPUs concurrent | ✅ All GPUs OK | | Denoise-only, 4 GPUs concurrent | ✅ All GPUs OK | | Denoise+SR, 4 GPUs concurrent | ❌ Only first GPU OK | | Full pipeline, single GPU | ✅ OK | | Full pipeline, phased approach | ✅ All GPUs OK | | Full pipeline with lock around run() | ❌ Still fails |

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/ComfyUI_Nvidia_VFX_Nodes#22

No description provided.

Rows
Columns