[GH-ISSUE #405] gst-cudanvrtc: nvrtcGetCUBINSize undefined symbol with NVIDIA 595 driver (RTX 5080 / Blackwell) #265

New issue

Closed

opened 2026-10-15 12:59:02 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-15 12:59:02 +00:00

Owner

Originally created by @JBailes on GitHub (May 3, 2026).
Original GitHub issue: https://github.com/games-on-whales/wolf/issues/405

Description

When running Wolf with NVIDIA driver 595.71.05 on an RTX 5080 (Blackwell / GB203), the GStreamer cudanvrtc plugin fails to load with:

WARN cudanvrtc gstcudanvrtc.cpp:179:gst_cuda_nvrtc_load_library_once:
Failed to load 'nvrtcGetCUBINSize', 'nvrtcGetCUBINSize':
/usr/local/nvidia/lib/libnvrtc.so: undefined symbol: nvrtcGetCUBINSize

The NVIDIA Container Toolkit injects the host's libnvrtc.so into the container at /usr/local/nvidia/lib/. The wolf container's GStreamer cudanvrtc plugin was compiled expecting nvrtcGetCUBINSize, which is not exported from libnvrtc.so as provided by NVIDIA 595 on this GPU.

Impact

The cudanvrtc GStreamer plugin does not load. Streaming itself still works because nvcodec initializes correctly and Wolf selects nvh264enc/nvav1enc for encoding. The broken plugin is not on the hot path for streaming.

Environment

GPU: NVIDIA GeForce RTX 5080 (GB203 / Blackwell)
Host driver: 595.71.05 (.run installer, required for Blackwell — Debian trixie only packages 550 which fails to compile on kernel 6.16+)
Host OS: Proxmox VE 9 (Debian trixie), kernel 6.17.2-1-pve
Container: LXC (privileged) with NVIDIA device passthrough
Docker runtime: NVIDIA Container Toolkit 1.19.0
Wolf image: ghcr.io/games-on-whales/wolf:stable

Relevant log lines (working)

For reference, the lines that confirm nvcodec IS working despite the nvrtc issue:

INFO | Using h264 encoder: nvcodec
INFO | Using h265 encoder: nvcodec
INFO | Using av1 encoder: nvcodec
INFO | Using zero copy pipeline on Nvidia (/dev/dri/renderD130)

Possible cause

nvrtcGetCUBINSize compiles PTX to CUBIN — the symbol may have been renamed or the function signature changed in the NVRTC API version bundled with driver 595 on Blackwell. The wolf container's GStreamer gst-plugins-bad cudanvrtc plugin may have been compiled against a different CUDA toolkit version.

Checking whether the symbol is present in the injected library:

nm -D /usr/local/nvidia/lib/libnvrtc.so | grep CUBIN

would confirm whether the symbol is absent, renamed, or versioned differently.

Originally created by @JBailes on GitHub (May 3, 2026). Original GitHub issue: https://github.com/games-on-whales/wolf/issues/405 ## Description When running Wolf with NVIDIA driver 595.71.05 on an RTX 5080 (Blackwell / GB203), the GStreamer `cudanvrtc` plugin fails to load with: ``` WARN cudanvrtc gstcudanvrtc.cpp:179:gst_cuda_nvrtc_load_library_once: Failed to load 'nvrtcGetCUBINSize', 'nvrtcGetCUBINSize': /usr/local/nvidia/lib/libnvrtc.so: undefined symbol: nvrtcGetCUBINSize ``` The NVIDIA Container Toolkit injects the host's `libnvrtc.so` into the container at `/usr/local/nvidia/lib/`. The wolf container's GStreamer `cudanvrtc` plugin was compiled expecting `nvrtcGetCUBINSize`, which is not exported from `libnvrtc.so` as provided by NVIDIA 595 on this GPU. ## Impact The `cudanvrtc` GStreamer plugin does not load. Streaming itself still works because `nvcodec` initializes correctly and Wolf selects `nvh264enc`/`nvav1enc` for encoding. The broken plugin is not on the hot path for streaming. ## Environment - GPU: NVIDIA GeForce RTX 5080 (GB203 / Blackwell) - Host driver: 595.71.05 (`.run` installer, required for Blackwell — Debian trixie only packages 550 which fails to compile on kernel 6.16+) - Host OS: Proxmox VE 9 (Debian trixie), kernel 6.17.2-1-pve - Container: LXC (privileged) with NVIDIA device passthrough - Docker runtime: NVIDIA Container Toolkit 1.19.0 - Wolf image: `ghcr.io/games-on-whales/wolf:stable` ## Relevant log lines (working) For reference, the lines that confirm nvcodec IS working despite the nvrtc issue: ``` INFO | Using h264 encoder: nvcodec INFO | Using h265 encoder: nvcodec INFO | Using av1 encoder: nvcodec INFO | Using zero copy pipeline on Nvidia (/dev/dri/renderD130) ``` ## Possible cause `nvrtcGetCUBINSize` compiles PTX to CUBIN — the symbol may have been renamed or the function signature changed in the NVRTC API version bundled with driver 595 on Blackwell. The wolf container's GStreamer `gst-plugins-bad` `cudanvrtc` plugin may have been compiled against a different CUDA toolkit version. Checking whether the symbol is present in the injected library: ```bash nm -D /usr/local/nvidia/lib/libnvrtc.so | grep CUBIN ``` would confirm whether the symbol is absent, renamed, or versioned differently.

bsakaguchi closed this issue

2026-10-15 12:59:02 +00:00

bsakaguchi commented

2026-10-15 12:59:02 +00:00

Author

Owner

@JBailes commented on GitHub (May 3, 2026):

Confirmed. The libnvrtc.so in question is baked into the wolf container image itself (not injected by the toolkit), and it's the CUDA 11.0 version:

$ nm -D /proc/<container-pid>/root/usr/local/nvidia/lib/libnvrtc.so | grep -i nvrtc
0000000000000000 A libnvrtc.so.11.0
T nvrtcAddNameExpression@@libnvrtc.so.11.0
T nvrtcCompileProgram@@libnvrtc.so.11.0
T nvrtcCreateProgram@@libnvrtc.so.11.0
T nvrtcDestroyProgram@@libnvrtc.so.11.0
T nvrtcGetErrorString@@libnvrtc.so.11.0
T nvrtcGetLoweredName@@libnvrtc.so.11.0
T nvrtcGetProgramLog@@libnvrtc.so.11.0
T nvrtcGetProgramLogSize@@libnvrtc.so.11.0
T nvrtcGetPTX@@libnvrtc.so.11.0
T nvrtcGetPTXSize@@libnvrtc.so.11.0
T nvrtcVersion@@libnvrtc.so.11.0

No nvrtcGetCUBIN or nvrtcGetCUBINSize — those were added in CUDA 11.1. The file is libnvrtc.so.11.0 i.e. CUDA 11.0, and the gst-cuda plugin was compiled expecting at least 11.1.

The file lives at /usr/local/nvidia/lib/libnvrtc.so{,.11.0} inside the image — it's not injected by the NVIDIA Container Toolkit. Bumping the baked-in NVRTC to 11.1+ should fix it.

@JBailes commented on GitHub (May 3, 2026): Confirmed. The `libnvrtc.so` in question is baked into the wolf container image itself (not injected by the toolkit), and it's the CUDA 11.0 version: ``` $ nm -D /proc/<container-pid>/root/usr/local/nvidia/lib/libnvrtc.so | grep -i nvrtc 0000000000000000 A libnvrtc.so.11.0 T nvrtcAddNameExpression@@libnvrtc.so.11.0 T nvrtcCompileProgram@@libnvrtc.so.11.0 T nvrtcCreateProgram@@libnvrtc.so.11.0 T nvrtcDestroyProgram@@libnvrtc.so.11.0 T nvrtcGetErrorString@@libnvrtc.so.11.0 T nvrtcGetLoweredName@@libnvrtc.so.11.0 T nvrtcGetProgramLog@@libnvrtc.so.11.0 T nvrtcGetProgramLogSize@@libnvrtc.so.11.0 T nvrtcGetPTX@@libnvrtc.so.11.0 T nvrtcGetPTXSize@@libnvrtc.so.11.0 T nvrtcVersion@@libnvrtc.so.11.0 ``` No `nvrtcGetCUBIN` or `nvrtcGetCUBINSize` — those were added in CUDA 11.1. The file is `libnvrtc.so.11.0` i.e. CUDA 11.0, and the `gst-cuda` plugin was compiled expecting at least 11.1. The file lives at `/usr/local/nvidia/lib/libnvrtc.so{,.11.0}` inside the image — it's not injected by the NVIDIA Container Toolkit. Bumping the baked-in NVRTC to 11.1+ should fix it.

bsakaguchi commented

2026-10-15 12:59:03 +00:00

Author

Owner

@JBailes commented on GitHub (May 3, 2026):

Fixed in https://github.com/games-on-whales/gow/pull/331

@JBailes commented on GitHub (May 3, 2026): Fixed in https://github.com/games-on-whales/gow/pull/331

bsakaguchi commented

2026-10-15 12:59:03 +00:00

Author

Owner

@JBailes commented on GitHub (May 12, 2026):