games-on-whales/wolf

Fork 0

mirror of https://github.com/games-on-whales/wolf.git synced 2026-10-15 13:02:03 +00:00

[GH-ISSUE #503] Feature Request: Update the manual nvidia driver documentation with an automated method #309

New issue

Open

opened 2026-10-15 12:59:31 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-15 12:59:31 +00:00

Owner

Originally created by @r0b0tb4c0n on GitHub (Sep 17, 2026).
Original GitHub issue: https://github.com/games-on-whales/wolf/issues/503

in the documentation found here:
https://games-on-whales.github.io/wolf/stable/user/quickstart.html

it says that:
"you have to manually re-create the Nvidia driver volume every time you update the drivers"

however, this is not the case.

to automate the process you can do the following in docker-compose:
make a nv-driver sub-directory in the same place as the docker-compose.yml
in the nv-driver sub-directory create 2 files:
Dockerfile.nv-driver

FROM ubuntu:22.04
COPY nv-driver.sh /nv-driver.sh
RUN chmod +x /nv-driver.sh
ENTRYPOINT ["/nv-driver.sh"]

nv-driver.sh

#!/bin/sh
set -eu


HOST_VERSION=$(cat /sys/module/nvidia/version)
VOL_VERSION=""
VOL_LIB=$(find /usr/nvidia/lib -maxdepth 1 -type f -name 'libnvidia-ml.so.*' 2>/dev/null | head -n 1)

if [ -n "$VOL_LIB" ]; then
    VOL_VERSION=$(basename "$VOL_LIB" | sed 's/^libnvidia-ml\.so\.//')
fi

echo "Host driver version:   $HOST_VERSION"
echo "Volume driver version: $VOL_VERSION"

if [ "$HOST_VERSION" = "$VOL_VERSION" ]; then
    echo "Driver volume is up to date, nothing to do."
    exit 0
fi

echo "System kernel module and container driver versions differ, rebuilding the driver volume."

rm -rf /usr/nvidia/*

apt-get update -y
apt-get install -y --no-install-recommends curl ca-certificates kmod pkg-config libglvnd-dev vulkan-tools

echo "Downloading NVIDIA driver package..."
curl -LO "https://download.nvidia.com/XFree86/Linux-x86_64/${HOST_VERSION}/NVIDIA-Linux-x86_64-${HOST_VERSION}.run"
chmod +x "NVIDIA-Linux-x86_64-${HOST_VERSION}.run"

./"NVIDIA-Linux-x86_64-${HOST_VERSION}.run" \
    --silent -z --skip-depmod --skip-module-unload --no-nvidia-modprobe \
    --no-kernel-modules --no-kernel-module-source \
    --opengl-prefix=/usr/nvidia --wine-prefix=/usr/nvidia \
    --utility-prefix=/usr/nvidia --utility-libdir=lib \
    --compat32-prefix=/usr/nvidia --compat32-libdir=lib32 \
    --egl-external-platform-config-path=/usr/nvidia/share/egl/egl_external_platform.d \
    --glvnd-egl-config-path=/usr/nvidia/share/glvnd/egl_vendor.d \
    --no-distro-scripts

mkdir -p /usr/nvidia/share/vulkan/icd.d
cp /etc/vulkan/icd.d/nvidia_icd.json /usr/nvidia/share/vulkan/icd.d/nvidia_icd.json

rm -f "NVIDIA-Linux-x86_64-${HOST_VERSION}.run"

echo "Driver volume (re)built to version $HOST_VERSION"
exit 0

in the main docker-compose.yml add a nv-driver-init container:

  nvidia-driver-init:
    container_name: nvidia-driver-init
    image: nvidia-driver-init
    build:
      context: ./nv-driver
      dockerfile: Dockerfile.nv-driver
    volumes:
      - nvidia-driver-vol:/usr/nvidia
    restart: "no"

then in the wolf section add:

    depends_on:
      nvidia-driver-init:
        condition: service_completed_successfully

this will then run a driver check and if necessary will rebuild the driver volume builder before wolf on every run. If the driver volume builder fails it will prevent wolf from starting

Originally created by @r0b0tb4c0n on GitHub (Sep 17, 2026). Original GitHub issue: https://github.com/games-on-whales/wolf/issues/503 in the documentation found here: https://games-on-whales.github.io/wolf/stable/user/quickstart.html it says that: "you have to manually re-create the Nvidia driver volume every time you update the drivers" however, this is not the case. to automate the process you can do the following in docker-compose: make a `nv-driver` sub-directory in the same place as the docker-compose.yml in the `nv-driver` sub-directory create 2 files: Dockerfile.nv-driver ``` FROM ubuntu:22.04 COPY nv-driver.sh /nv-driver.sh RUN chmod +x /nv-driver.sh ENTRYPOINT ["/nv-driver.sh"] ``` nv-driver.sh ``` #!/bin/sh set -eu HOST_VERSION=$(cat /sys/module/nvidia/version) VOL_VERSION="" VOL_LIB=$(find /usr/nvidia/lib -maxdepth 1 -type f -name 'libnvidia-ml.so.*' 2>/dev/null | head -n 1) if [ -n "$VOL_LIB" ]; then VOL_VERSION=$(basename "$VOL_LIB" | sed 's/^libnvidia-ml\.so\.//') fi echo "Host driver version: $HOST_VERSION" echo "Volume driver version: $VOL_VERSION" if [ "$HOST_VERSION" = "$VOL_VERSION" ]; then echo "Driver volume is up to date, nothing to do." exit 0 fi echo "System kernel module and container driver versions differ, rebuilding the driver volume." rm -rf /usr/nvidia/* apt-get update -y apt-get install -y --no-install-recommends curl ca-certificates kmod pkg-config libglvnd-dev vulkan-tools echo "Downloading NVIDIA driver package..." curl -LO "https://download.nvidia.com/XFree86/Linux-x86_64/${HOST_VERSION}/NVIDIA-Linux-x86_64-${HOST_VERSION}.run" chmod +x "NVIDIA-Linux-x86_64-${HOST_VERSION}.run" ./"NVIDIA-Linux-x86_64-${HOST_VERSION}.run" \ --silent -z --skip-depmod --skip-module-unload --no-nvidia-modprobe \ --no-kernel-modules --no-kernel-module-source \ --opengl-prefix=/usr/nvidia --wine-prefix=/usr/nvidia \ --utility-prefix=/usr/nvidia --utility-libdir=lib \ --compat32-prefix=/usr/nvidia --compat32-libdir=lib32 \ --egl-external-platform-config-path=/usr/nvidia/share/egl/egl_external_platform.d \ --glvnd-egl-config-path=/usr/nvidia/share/glvnd/egl_vendor.d \ --no-distro-scripts mkdir -p /usr/nvidia/share/vulkan/icd.d cp /etc/vulkan/icd.d/nvidia_icd.json /usr/nvidia/share/vulkan/icd.d/nvidia_icd.json rm -f "NVIDIA-Linux-x86_64-${HOST_VERSION}.run" echo "Driver volume (re)built to version $HOST_VERSION" exit 0 ``` in the main docker-compose.yml add a nv-driver-init container: ``` nvidia-driver-init: container_name: nvidia-driver-init image: nvidia-driver-init build: context: ./nv-driver dockerfile: Dockerfile.nv-driver volumes: - nvidia-driver-vol:/usr/nvidia restart: "no" ``` then in the wolf section add: ``` depends_on: nvidia-driver-init: condition: service_completed_successfully ``` this will then run a driver check and if necessary will rebuild the driver volume builder before wolf on every run. If the driver volume builder fails it will prevent wolf from starting

bsakaguchi added the

documentation

label

2026-10-15 12:59:31 +00:00

bsakaguchi commented

2026-10-15 12:59:32 +00:00

Author

Owner

@r0b0tb4c0n commented on GitHub (Sep 17, 2026):

one other thing that can be added to the documentation.

sometimes not all the nvidia devs are automatically created. by default, on boot i only had these devs created:

/dev/nvidia-uvm
/dev/nvidia-uvm-tools

to make sure this happens on boot you can create a systemd service:
/etc/systemd/system/nvidia-init.service

[Unit]
Description=Initialize NVIDIA devices

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/nvidia-container-cli --load-kmods info

[Install]
WantedBy=multi-user.target

and then enable the service to run on boot:
systemctl enable nvidia-init.service

this will create all the devs that the documentation recommends should be passed in:

      - /dev/nvidia-uvm
      - /dev/nvidia-uvm-tools
      - /dev/nvidia-caps/nvidia-cap1
      - /dev/nvidia-caps/nvidia-cap2
      - /dev/nvidiactl
      - /dev/nvidia0
      - /dev/nvidia-modeset

this will need to run before docker, so you need to put an override in via systemctl edit docker

[Unit]
After=nvidia-init.service
Wants=nvidia-init.service

The other thing that i did test is the udev rules as found in the documentation:

# create necessary NVIDIA device files in /dev/*
KERNEL=="nvidia", RUN+="/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 0666 /dev/nvidia*'"
KERNEL=="nvidia_uvm", RUN+="/bin/bash -c '/usr/bin/nvidia-modprobe -c0 -u && /bin/chmod 0666 /dev/nvidia-uvm*'"

what i found was that /dev/nvidia-modeset was not automatically created, that would be done by nvidia-modprobe -m

if a service isn't wanted, then the udev rules could be modified to this:

KERNEL=="nvidia", RUN+="/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 0666 /dev/nvidia*'"
KERNEL=="nvidia_uvm", RUN+="/bin/bash -c '/usr/bin/nvidia-modprobe -c0 -u -m && /bin/chmod 0666 /dev/nvidia-uvm*'"

where all that would be needed is adding a -m to nvidia-modprobe for the last device creation

@r0b0tb4c0n commented on GitHub (Sep 17, 2026): one other thing that can be added to the documentation. sometimes not all the nvidia devs are automatically created. by default, on boot i only had these devs created: ``` /dev/nvidia-uvm /dev/nvidia-uvm-tools ``` to make sure this happens on boot you can create a systemd service: `/etc/systemd/system/nvidia-init.service` ``` [Unit] Description=Initialize NVIDIA devices [Service] Type=oneshot RemainAfterExit=yes ExecStart=/usr/bin/nvidia-container-cli --load-kmods info [Install] WantedBy=multi-user.target ``` and then enable the service to run on boot: `systemctl enable nvidia-init.service` this will create all the devs that the documentation recommends should be passed in: ``` - /dev/nvidia-uvm - /dev/nvidia-uvm-tools - /dev/nvidia-caps/nvidia-cap1 - /dev/nvidia-caps/nvidia-cap2 - /dev/nvidiactl - /dev/nvidia0 - /dev/nvidia-modeset ``` this will need to run before docker, so you need to put an override in via `systemctl edit docker` ``` [Unit] After=nvidia-init.service Wants=nvidia-init.service ``` The other thing that i did test is the udev rules as found in the documentation: ``` # create necessary NVIDIA device files in /dev/* KERNEL=="nvidia", RUN+="/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 0666 /dev/nvidia*'" KERNEL=="nvidia_uvm", RUN+="/bin/bash -c '/usr/bin/nvidia-modprobe -c0 -u && /bin/chmod 0666 /dev/nvidia-uvm*'" ``` what i found was that `/dev/nvidia-modeset` was not automatically created, that would be done by `nvidia-modprobe -m` if a service isn't wanted, then the udev rules could be modified to this: ``` KERNEL=="nvidia", RUN+="/bin/bash -c '/usr/bin/nvidia-smi -L && /bin/chmod 0666 /dev/nvidia*'" KERNEL=="nvidia_uvm", RUN+="/bin/bash -c '/usr/bin/nvidia-modprobe -c0 -u -m && /bin/chmod 0666 /dev/nvidia-uvm*'" ``` where all that would be needed is adding a -m to nvidia-modprobe for the last device creation

bsakaguchi commented

2026-10-15 12:59:32 +00:00

Author

Owner

@ABeltramo commented on GitHub (Sep 21, 2026):

This all sounds very sensible, would you like to open a PR for these changes?

@ABeltramo commented on GitHub (Sep 21, 2026): This all sounds very sensible, would you like to open a PR for these changes?

bsakaguchi commented

2026-10-15 12:59:32 +00:00

Author

Owner

@r0b0tb4c0n commented on GitHub (Sep 21, 2026):

I'll take care of this today (hopefully)

@r0b0tb4c0n commented on GitHub (Sep 21, 2026): I'll take care of this today (hopefully)

bsakaguchi commented

2026-10-15 12:59:32 +00:00

Author

Owner

@To0wnn commented on GitHub (Oct 5, 2026):

Same idea works fine with a rootful podman quadlet: we run the check as ExecStartPre= of wolf.service (compare /sys/module/nvidia/version with a version stamp in the volume, rebuild only when different, and refuse to start Wolf if the rebuild fails, so you never get a half-working mix).

One extra tip for the docs: apps that run as a flatpak inside the container (e.g. Sober/Roblox) don't use the driver volume but flatpak's own org.freedesktop.Platform.GL.nvidia-<version> extension, so that needs updating on a driver change too.

@To0wnn commented on GitHub (Oct 5, 2026): Same idea works fine with a rootful **podman quadlet**: we run the check as `ExecStartPre=` of `wolf.service` (compare `/sys/module/nvidia/version` with a version stamp in the volume, rebuild only when different, and refuse to start Wolf if the rebuild fails, so you never get a half-working mix). One extra tip for the docs: apps that run as a **flatpak inside the container** (e.g. Sober/Roblox) don't use the driver volume but flatpak's own `org.freedesktop.Platform.GL.nvidia-<version>` extension, so that needs updating on a driver change too.