mirror of
https://github.com/games-on-whales/wolf.git
synced 2026-10-15 21:12:08 +00:00
[GH-ISSUE #214] Ungraceful crash when running out of memory #142
Labels
No labels
🧠 1 - 🤖 999
AMD
bug
bug
cannot reproduce
documentation
duplicate
easy
enhancement
enhancement
enhancement
fixed?
fixed?
good first issue
hard
help wanted
Intel
invalid
LXC
medium
moonlight-protocol
Multi GPU
nixos
not a bug
nvidia
pull-request
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
games-on-whales/wolf#142
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Lombra on GitHub (Jun 19, 2025).
Original GitHub issue: https://github.com/games-on-whales/wolf/issues/214
Hi,
I run Wolf inside an LXC container. When an application crashes due to what I've been able to discern is memory shortage, the whole LXC container becomes nearly unresponsive, CPU usage bottoming out. The Wolf Docker container continues to run, though you can't connect to it or anything, and if I'm lucky, I can get a command through to (slowly) stop it, which resolves the situation.
I don't know where in the stack this would or could be alleviated, if at all. Docker log is not really showing anything at the time of the crash.
@ABeltramo commented on GitHub (Jun 19, 2025):
When the host OS runs out of memory there are so many things that start failing in a "last resort" attempt at claiming some memory.
I'm not sure there's anything specifically that we can do in Wolf, but I'm open to suggestions..
@Trasen commented on GitHub (Jun 20, 2025):
Is there any reliable profiling tools available? If so we could hook that in to the Wolf containers and at least be able to get a heat map of what is causing this recursive "last ditch attempt of survival but never outright crashing" behaviour.
Understanding the "how it happens" and in which order things start to choke could lead to a possible solution.
But as with most software, outside of writing an entire scheduler around wolf which actively manages all possible processes, when hardware runs out it runs out.
@Lombra commented on GitHub (Jun 21, 2025):
I've not used these Docker features before, but I learnt that you can set some restrictions on system resources on containers. Wolf applications are launched as separate containers as I discovered, so I can't set any restrictions on those. (?) I tried setting 1 GB reserved memory for the main container, but that didn't seem to help. It wouldn't be a particularly smart solution, but if I could hardcode a memory limit for the application containers, maybe that could work?
@salty2011 commented on GitHub (Jun 21, 2025):
Hey @Lombra
In the config.toml you can apply launch options for the app in the variable
There are options for setting memory and cpu limits here:
https://docs.docker.com/engine/api/v1.40/#tag/Container/operation/ContainerCreate
@Trasen commented on GitHub (Jun 21, 2025):
Salty being an absolute unit as always, hats off :D
@Lombra commented on GitHub (Jun 22, 2025):
Thank you!
I set a memory limit on the Steam container, which prevents it from taking the rest of the system down with it, so that's a big improvement, at least. The Moonlight stream remains intact, but the application soft locks. In the case of Steam, it seems like
steamwebhelperlocks up. Terminating it via the shell salvages the session.A couple of similar options to explore:
https://superuser.com/questions/1645722/how-to-limit-the-sum-of-memory-used-by-a-group-of-docker-containers
https://unix.stackexchange.com/questions/537645/how-to-limit-docker-total-resources
@Lombra commented on GitHub (Jul 6, 2025):
I've settled for a cgroup for now, managed by systemd.
Create a slice file, defining the resource limits:
/etc/systemd/system/docker.sliceDocumentation recommends using
MemoryHighrather thanMemoryMax, as a soft limit, but that just seems to cause containers to freeze rather than crash.Configure Docker to use the cgroup.
/etc/docker/daemon.jsonNow, all Docker containers has a shared memory quota of 19 GB. (as far as I understand) This way I don't need to configure containers individually.
It is also possible to set a percentage value, but that didn't seem as useful as I first imagined.