[GH-ISSUE #40] nvvfx bundled libcudnn.so.9 (9.7.1 shim, no sub-libraries) is loaded RTLD_GLOBAL and aborts any later cuDNN consumer (onnxruntime / DWPose) — same class as #34 #32

New issue

Open

opened 2026-10-26 22:28:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:28:13 +00:00

Owner

Originally created by @haroldparis on GitHub (Sep 9, 2026).
Original GitHub issue: https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes/issues/40

Symptom

With this node pack installed, any custom node that binds to cuDNN after ComfyUI startup crashes the whole process. First victim here: comfyui_controlnet_aux DWPose, whose onnxruntime CUDA session is created lazily on first use:

DWPose: Caching ONNXRuntime session yolox_l.onnx...
Unable to load any of {libcudnn_ops.so.9.7.1, libcudnn_ops.so.9.7, libcudnn_ops.so.9, libcudnn_ops.so}
Invalid handle. Cannot load symbol cudnnCreateFilterDescriptor
Fatal Python error: Aborted

ComfyUI dies (SIGABRT), the in-memory history is lost. Note the version in the message: 9.7.1 is neither the cuDNN torch ships (9.19 / 9.20) nor anything else in the venv. It is the version of the shim that ends up being called.

Root cause

nvvfx/_lib_loader.py (nvidia-vfx 0.1.0.1) does, at import time:

for lib_name in _LINUX_LIBS:          # includes "libcudnn.so.9"
    ctypes.CDLL(str(lib_path), mode=ctypes.RTLD_GLOBAL)

nvvfx/libs/libcudnn.so.9 is a cuDNN 9.7.1 shim (125 KB) shipped without its sub-libraries (libcudnn_ops, libcudnn_graph, ...). Torch loads its own cuDNN with RTLD_LOCAL, so it is not in the global symbol scope. Anything loaded after import nvvfx (onnxruntime's CUDA provider, or any other cuDNN user) resolves cudnnCreate* through the global scope, hits the 9.7.1 shim, which then tries to dlopen libcudnn_ops.so.9.7.1, fails, and aborts.

This is the same mechanism as #34 (bundled libnvinfer.so.10 10.9.0 loaded first, breaking TensorRT 10.12 builders), with cuDNN instead of TensorRT.

Minimal reproduction (outside ComfyUI)

import torch, onnxruntime as ort                      # OK: CUDAExecutionProvider session works
import torch, nvvfx, onnxruntime as ort               # abort on ort.InferenceSession(..., providers=["CUDAExecutionProvider"])

Environment: Ubuntu, RTX 5060 Ti, torch 2.12.0+cu132 (nvidia-cudnn-cu13 9.20), onnxruntime-gpu 1.24.4, nvidia-vfx 0.1.0.1, ComfyUI v0.34, this pack at 892515e.

Workaround that works

Removing the bundled shim makes the VFX libraries resolve libcudnn.so.9 against the cuDNN already loaded by torch:

mv site-packages/nvvfx/libs/libcudnn.so.9 site-packages/nvvfx/libs/libcudnn.so.9.disabled

Verified after that: onnxruntime CUDA sessions work, DWPose works, and RTXVideoSuperResolution still runs fine on the torch cuDNN (full VSR runs, 1080p, ULTRA).

Suggested fix

Either do not bundle libcudnn.so.9 in the nvidia-vfx wheel (torch always provides one), or ship the complete cuDNN (shim + sub-libraries), or load the bundled libraries without RTLD_GLOBAL so they cannot shadow the environment's own cuDNN / TensorRT for other nodes. I understand the wheel is NVIDIA's rather than this repo's, but this repo is where users hit it.

Originally created by @haroldparis on GitHub (Sep 9, 2026). Original GitHub issue: https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes/issues/40 ## Symptom With this node pack installed, any custom node that binds to cuDNN **after** ComfyUI startup crashes the whole process. First victim here: `comfyui_controlnet_aux` DWPose, whose onnxruntime CUDA session is created lazily on first use: ``` DWPose: Caching ONNXRuntime session yolox_l.onnx... Unable to load any of {libcudnn_ops.so.9.7.1, libcudnn_ops.so.9.7, libcudnn_ops.so.9, libcudnn_ops.so} Invalid handle. Cannot load symbol cudnnCreateFilterDescriptor Fatal Python error: Aborted ``` ComfyUI dies (SIGABRT), the in-memory history is lost. Note the version in the message: **9.7.1** is neither the cuDNN torch ships (9.19 / 9.20) nor anything else in the venv. It is the version of the shim that ends up being called. ## Root cause `nvvfx/_lib_loader.py` (nvidia-vfx 0.1.0.1) does, at import time: ```python for lib_name in _LINUX_LIBS: # includes "libcudnn.so.9" ctypes.CDLL(str(lib_path), mode=ctypes.RTLD_GLOBAL) ``` `nvvfx/libs/libcudnn.so.9` is a cuDNN 9.7.1 **shim** (125 KB) shipped **without** its sub-libraries (`libcudnn_ops`, `libcudnn_graph`, ...). Torch loads its own cuDNN with `RTLD_LOCAL`, so it is not in the global symbol scope. Anything loaded after `import nvvfx` (onnxruntime's CUDA provider, or any other cuDNN user) resolves `cudnnCreate*` through the global scope, hits the 9.7.1 shim, which then tries to dlopen `libcudnn_ops.so.9.7.1`, fails, and aborts. This is the same mechanism as #34 (bundled `libnvinfer.so.10` 10.9.0 loaded first, breaking TensorRT 10.12 builders), with cuDNN instead of TensorRT. ## Minimal reproduction (outside ComfyUI) ```python import torch, onnxruntime as ort # OK: CUDAExecutionProvider session works import torch, nvvfx, onnxruntime as ort # abort on ort.InferenceSession(..., providers=["CUDAExecutionProvider"]) ``` Environment: Ubuntu, RTX 5060 Ti, torch 2.12.0+cu132 (nvidia-cudnn-cu13 9.20), onnxruntime-gpu 1.24.4, nvidia-vfx 0.1.0.1, ComfyUI v0.34, this pack at 892515e. ## Workaround that works Removing the bundled shim makes the VFX libraries resolve `libcudnn.so.9` against the cuDNN already loaded by torch: ``` mv site-packages/nvvfx/libs/libcudnn.so.9 site-packages/nvvfx/libs/libcudnn.so.9.disabled ``` Verified after that: onnxruntime CUDA sessions work, DWPose works, and `RTXVideoSuperResolution` still runs fine on the torch cuDNN (full VSR runs, 1080p, ULTRA). ## Suggested fix Either do not bundle `libcudnn.so.9` in the `nvidia-vfx` wheel (torch always provides one), or ship the complete cuDNN (shim + sub-libraries), or load the bundled libraries without `RTLD_GLOBAL` so they cannot shadow the environment's own cuDNN / TensorRT for other nodes. I understand the wheel is NVIDIA's rather than this repo's, but this repo is where users hit it.

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/ComfyUI_Nvidia_VFX_Nodes#32

No description provided.

Rows
Columns