[GH-ISSUE #128] How to properly enable GPU when device is iommu? #79

Closed
opened 2026-10-15 12:56:01 +00:00 by bsakaguchi · 5 comments
Owner

Originally created by @futurepr0n on GitHub (Sep 27, 2024).
Original GitHub issue: https://github.com/games-on-whales/wolf/issues/128

Trying to run as a proxmox, started by creating a container and then installing using a combo of the dockerfile and docker-compose.yml I created. I didn't have /dev/dri, so I mapped as iommu - maybe thats my issue. But righ now heres how it looks and my crash info.

GNU nano 7.2 docker-compose.yml

version: "3"
services:
  wolf:
    image: ghcr.io/games-on-whales/wolf:stable
    environment:
      - XDG_RUNTIME_DIR=/tmp/sockets
      - HOST_APPS_STATE_FOLDER=/etc/wolf
    volumes:
      - /etc/wolf/:/etc/wolf
      - /tmp/sockets:/tmp/sockets:rw
      - /var/run/docker.sock:/var/run/docker.sock:rw
      - /dev/:/dev/:rw
      - /mnt/udev:/run/udev:rw
    device_cgroup_rules:
      - 'c 13:* rmw'
    devices:
      - /dev/iommu
      - /dev/uinput
      - /dev/uhid
    network_mode: host
    restart: unless-stopped


GNU nano 7.2 Dockerfile

# HTTPS
EXPOSE 47984/tcp
# HTTP
EXPOSE 47989/tcp
# Control
EXPOSE 47999/udp
# RTSP
EXPOSE 48010/tcp
# Video (up to 10 users, you can open more ports if needed)
EXPOSE 48100-48110/udp
# Audio (up to 10 users, you can open more ports if needed)
EXPOSE 48200-48210/udp

my LXC.conf

arch: amd64
cores: 12
dev0: /dev/uinput
dev1: /dev/uhid
dev2: /dev/iommu
features: nesting=1
hostname: wolf
memory: 8192
net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:CC:A7:93,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-141-disk-0,size=22G
swap: 512
tags: proxmox-helper-scripts
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.mount.entry: /dev/iommu dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/input dev/input none bind,optional,create=dir
lxc.mount.entry: /run/udev mnt/udev none bind,optional,create=dir

Things crashed after i set the pin and connected the server successfully.

Screenshot 2024-09-26 at 11 43 13 PM
Screenshot 2024-09-26 at 11 44 36 PM

If I run screenfetch on the container, i can see things look ok, same with lspci
Screenshot 2024-09-26 at 11 49 56 PM

Originally created by @futurepr0n on GitHub (Sep 27, 2024). Original GitHub issue: https://github.com/games-on-whales/wolf/issues/128 Trying to run as a proxmox, started by creating a container and then installing using a combo of the dockerfile and docker-compose.yml I created. I didn't have /dev/dri, so I mapped as iommu - maybe thats my issue. But righ now heres how it looks and my crash info. GNU nano 7.2 docker-compose.yml ``` version: "3" services: wolf: image: ghcr.io/games-on-whales/wolf:stable environment: - XDG_RUNTIME_DIR=/tmp/sockets - HOST_APPS_STATE_FOLDER=/etc/wolf volumes: - /etc/wolf/:/etc/wolf - /tmp/sockets:/tmp/sockets:rw - /var/run/docker.sock:/var/run/docker.sock:rw - /dev/:/dev/:rw - /mnt/udev:/run/udev:rw device_cgroup_rules: - 'c 13:* rmw' devices: - /dev/iommu - /dev/uinput - /dev/uhid network_mode: host restart: unless-stopped ``` GNU nano 7.2 Dockerfile ``` # HTTPS EXPOSE 47984/tcp # HTTP EXPOSE 47989/tcp # Control EXPOSE 47999/udp # RTSP EXPOSE 48010/tcp # Video (up to 10 users, you can open more ports if needed) EXPOSE 48100-48110/udp # Audio (up to 10 users, you can open more ports if needed) EXPOSE 48200-48210/udp ``` my LXC.conf ``` arch: amd64 cores: 12 dev0: /dev/uinput dev1: /dev/uhid dev2: /dev/iommu features: nesting=1 hostname: wolf memory: 8192 net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:CC:A7:93,ip=dhcp,type=veth onboot: 1 ostype: debian rootfs: local-lvm:vm-141-disk-0,size=22G swap: 512 tags: proxmox-helper-scripts lxc.cgroup2.devices.allow: a lxc.cap.drop: lxc.mount.entry: /dev/iommu dev/dri none bind,optional,create=dir lxc.mount.entry: /dev/input dev/input none bind,optional,create=dir lxc.mount.entry: /run/udev mnt/udev none bind,optional,create=dir ``` Things crashed after i set the pin and connected the server successfully. ![Screenshot 2024-09-26 at 11 43 13 PM](https://github.com/user-attachments/assets/14b00019-2686-4299-b406-c69f521ae68c) ![Screenshot 2024-09-26 at 11 44 36 PM](https://github.com/user-attachments/assets/e1669326-6508-47e0-ba2f-e22f96248b66) If I run screenfetch on the container, i can see things look ok, same with lspci ![Screenshot 2024-09-26 at 11 49 56 PM](https://github.com/user-attachments/assets/1f396a49-1f7a-4c5f-8ad2-d6c85e10b117)
bsakaguchi 2026-10-15 12:56:01 +00:00
Author
Owner

@ABeltramo commented on GitHub (Sep 27, 2024):

I didn't have /dev/dri, so I mapped as iommu - maybe thats my issue

Most likely, as you can see from the warnings in the logs 😉
You can run Wolf with SW acceleration by setting the env variable WOLF_RENDER_NODE=software, don't expect great gaming results, though.

<!-- gh-comment-id:2378473663 --> @ABeltramo commented on GitHub (Sep 27, 2024): > I didn't have /dev/dri, so I mapped as iommu - maybe thats my issue Most likely, as you can see from the warnings in the logs 😉 You can run Wolf with SW acceleration by setting the env variable `WOLF_RENDER_NODE=software`, don't expect great gaming results, though.
Author
Owner

@futurepr0n commented on GitHub (Sep 27, 2024):

@ABeltramo but I have a GPU, why is it changed to No Gpu? I am trying to enable the HW acceleration. How do I properly map/passthrough the gpu? I see the gpu in the container, but I can't figure out how to send it because its not mapped on my host as /dev/dri - i thought it was mapped as /dev/iommu so I tried doing that as the passthrough but its failing the wayland load and crashes. Is there any help in getting the hardware working in my case? Thats the goal.
Screenshot 2024-09-27 at 10 25 13 AM

<!-- gh-comment-id:2379403229 --> @futurepr0n commented on GitHub (Sep 27, 2024): @ABeltramo but I have a GPU, why is it changed to No Gpu? I am trying to enable the HW acceleration. How do I properly map/passthrough the gpu? I see the gpu in the container, but I can't figure out how to send it because its not mapped on my host as /dev/dri - i thought it was mapped as /dev/iommu so I tried doing that as the passthrough but its failing the wayland load and crashes. Is there any help in getting the hardware working in my case? Thats the goal. ![Screenshot 2024-09-27 at 10 25 13 AM](https://github.com/user-attachments/assets/fca1f50a-d406-422e-9f7c-42bb1b2f1bb7)
Author
Owner

@futurepr0n commented on GitHub (Sep 27, 2024):

Going to close this as the issue isnt related to wolf so much as me needing to figure out this GPU passthrough properly. Will fiddle around - if anyone feels inclined to help thats appreciated, if I find success I will update this comment with the tips, other than that I'll close the issue, thanks for sharing the project!

<!-- gh-comment-id:2379830826 --> @futurepr0n commented on GitHub (Sep 27, 2024): Going to close this as the issue isnt related to wolf so much as me needing to figure out this GPU passthrough properly. Will fiddle around - if anyone feels inclined to help thats appreciated, if I find success I will update this comment with the tips, other than that I'll close the issue, thanks for sharing the project!
Author
Owner

@ABeltramo commented on GitHub (Sep 27, 2024):

I'm sorry I've completely misunderstood your issue!

I see the gpu in the container, but I can't figure out how to send it because its not mapped on my host as /dev/dri

This is the crux of the problem, it's hard to tell given the little info that I've got, but my guess is that Mesa is failing to load up the GPU.
I'd start by scanning the startup logs:

sudo journalctl -b

There will probably be some error or relevant warning in there somewhere.

As for the IOMMU it really depends on the motherboard and configuration, you have to pass a full group to the VM in order to properly use the GPU, I'm going to link the official Proxmox docs if you haven't seen them already, a quick Google search brings up also a few posts that might be of help: 1, 2, 3

<!-- gh-comment-id:2379898615 --> @ABeltramo commented on GitHub (Sep 27, 2024): I'm sorry I've completely misunderstood your issue! > I see the gpu in the container, but I can't figure out how to send it because its not mapped on my host as /dev/dri This is the crux of the problem, it's hard to tell given the little info that I've got, but my guess is that `Mesa` is failing to load up the GPU. I'd start by scanning the startup logs: ``` sudo journalctl -b ``` There will probably be some error or relevant warning in there somewhere. As for the IOMMU it really depends on the motherboard and configuration, you have to pass a full group to the VM in order to properly use the GPU, I'm going to link the [official Proxmox docs](https://pve.proxmox.com/wiki/PCI_Passthrough#GPU_passthrough) if you haven't seen them already, a quick Google search brings up also a few posts that might be of help: [1](https://forum.proxmox.com/threads/solved-igpu-passthrough-trouble-cannot-access-dev-dri-no-such-file-or-directory.109801/), [2](https://forum.proxmox.com/threads/amd-igpu-not-working.148550/), [3](https://www.reddit.com/r/Proxmox/comments/190ewcc/devdri_missing_broken_igpu_passthrough/)
Author
Owner

@futurepr0n commented on GitHub (Sep 27, 2024):

appreciate it! was hoping there was some expertise on this specific issue here that could give me a quick fix immediately but this is going to take more investigation on my part as well. I appreciate these links a few of them are new to me and also will be experimenting more over the weekend. Hopeful!

<!-- gh-comment-id:2380051274 --> @futurepr0n commented on GitHub (Sep 27, 2024): appreciate it! was hoping there was some expertise on this specific issue here that could give me a quick fix immediately but this is going to take more investigation on my part as well. I appreciate these links a few of them are new to me and also will be experimenting more over the weekend. Hopeful!
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
games-on-whales/wolf#79
No description provided.