[PR #517] Add PyroWave video codec support #505

New issue

Open

opened 2026-10-15 13:01:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 13:01:05 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/games-on-whales/wolf/pull/517
Author: @bscubed
Created: 9/29/2026
Status: 🔄 Open

Base: beta ← Head: pyrowave

📝 Commits (6)

57a25a7 Add PyroWave video codec support
bfc35ca Scale video pacing rate with the negotiated bitrate
dee1094 Send legacy RGB dma-bufs to the PyroWave GPU importer
b360fae Split large video frames into consistent FEC blocks
5c8ad56 Update PyroWave to the 1.0 API
090b13b Encode PyroWave in limited range to match Aurora

📊 Changes

34 files changed (+3011 additions, -76 deletions)

View changed files

📝 docker/wolf.Dockerfile (+20 -0)
📝 src/core/CMakeLists.txt (+7 -1)
📝 src/moonlight-protocol/moonlight.cpp (+10 -1)
📝 src/moonlight-protocol/moonlight/data-structures.hpp (+1 -0)
📝 src/moonlight-protocol/moonlight/protocol.hpp (+5 -1)
📝 src/moonlight-server/CMakeLists.txt (+18 -0)
📝 src/moonlight-server/api/endpoints.cpp (+3 -1)
📝 src/moonlight-server/events/events.hpp (+1 -0)
📝 src/moonlight-server/events/reflectors.hpp (+3 -0)
➕ src/moonlight-server/gst-plugin/pyrowave/gstpyrowaveenc.cpp (+975 -0)
➕ src/moonlight-server/gst-plugin/pyrowave/gstpyrowaveenc.hpp (+57 -0)
➕ src/moonlight-server/gst-plugin/pyrowave/pyrowave_align.hpp (+240 -0)
➕ src/moonlight-server/gst-plugin/pyrowave/pyrowave_device.cpp (+404 -0)
➕ src/moonlight-server/gst-plugin/pyrowave/pyrowave_device.hpp (+236 -0)
📝 src/moonlight-server/gst-plugin/video.hpp (+136 -23)
📝 src/moonlight-server/platforms/hw.hpp (+16 -0)
📝 src/moonlight-server/platforms/hw_linux.cpp (+20 -7)
📝 src/moonlight-server/platforms/hw_unknown.cpp (+5 -3)
📝 src/moonlight-server/rest/endpoints.hpp (+4 -2)
📝 src/moonlight-server/rtsp/commands.hpp (+22 -4)

...and 14 more files

📄 Description

Closes #515

This PR adds PyroWave as a fourth video codec in Wolf, alongside H.264, HEVC and AV1. Other codecs save bandwidth by sending only what changed since the previous frame. That makes encoding slower, we have to wait an extra frame to compare the current frame with the previous, and one lost packet can corrupt the frames that follow. PyroWave compresses each frame on its own with a wavelet transform on the GPU. Encoding and decoding are very fast, but it needs much more bandwidth as a result. On a wired home network, that's a good trade for lower latency.

How

Wolf will offer PyroWave only if a working encoder was found at startup. Clients that support it (only Aurora at the moment) can request it, and if negotiation is successful, the stream begins, same as any other codec
If a client asks for PyroWave and the host can't provide it, the connection is refused instead of quietly falling back to another codec.
The encoder reads frames straight from GPU memory and uses the same GPU Wolf captures on.
The compressed data is lined up with packet boundaries, so a lost packet doesn't ruin the rest of the frame.
PyroWave is optional at build time. The Docker image builds it and includes the Vulkan runtime

Infrastructure Changes

Bitrate requested for high resolutions and framerates can get as high as 1Gbps. Some changes around bitrate limits need to be adjusted in order to properly support this codec from within Wolf.

The encoder currently rejects bitrates above 500 Mbit/s and falls back to about 15 Mbit/s, so high bitrate streams get incredibly compressed. The limit should now scale with the requested bitrate from the client
Frame pacing:
- Wolf currently sends every frame's packets at a fixed rate of 80% of 1 Gbit/s, which is 71 packets/ms with 1392-byte packets. With the 20% FEC packets added, a 4K120 frame at 679 Mbit/s is 713 packets. Sending it takes 10 ms, longer than the 8.33 ms between frames, so the stream tops out at about 100 fps.
- With this PR, the rate is the larger of the current fixed rate and 3x the stream's actual send rate (bitrate plus FEC), so a frame is sent in about a third of the time between frames. For the 679 Mbit/s example that is 220 packets/ms, so the 713-packet frame takes about 3.2 ms. Streams below about 267 Mbit/s keep the current rate. Allows for flexibility for users pushing past 1Gbps networks.
PyroWave support added as a Gstreamer plugin

Comparison

PyroWave	HEVC (H.265)

Notes

Needs a Vulkan 1.3 GPU and a fast local network.
Frames over 255 packets skip error correction. That limit already existed in Wolf, but PyroWave hits it at very high bitrates.
Testing in comparisons above were done on a Wolf host with an AMD 7900 XTX streamed to a Steam Deck over 2.5Gbps LAN.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/games-on-whales/wolf/pull/517 **Author:** [@bscubed](https://github.com/bscubed) **Created:** 9/29/2026 **Status:** 🔄 Open **Base:** `beta` ← **Head:** `pyrowave` --- ### 📝 Commits (6) - [`57a25a7`](https://github.com/games-on-whales/wolf/commit/57a25a75741c788959a02d198295881f5f036808) Add PyroWave video codec support - [`bfc35ca`](https://github.com/games-on-whales/wolf/commit/bfc35ca6f9ed670d8804edc1a4eed2c9821c6edb) Scale video pacing rate with the negotiated bitrate - [`dee1094`](https://github.com/games-on-whales/wolf/commit/dee10942a2f6395ca4c462b91006439bbd033d17) Send legacy RGB dma-bufs to the PyroWave GPU importer - [`b360fae`](https://github.com/games-on-whales/wolf/commit/b360faefa4c9a6e48513499677cd98302f350823) Split large video frames into consistent FEC blocks - [`5c8ad56`](https://github.com/games-on-whales/wolf/commit/5c8ad565e9286c4cb731c6cb34916e5f6bfa1c88) Update PyroWave to the 1.0 API - [`090b13b`](https://github.com/games-on-whales/wolf/commit/090b13be725416acf7c2334dfa33d606cab641f8) Encode PyroWave in limited range to match Aurora ### 📊 Changes **34 files changed** (+3011 additions, -76 deletions) <details> <summary>View changed files</summary> 📝 `docker/wolf.Dockerfile` (+20 -0) 📝 `src/core/CMakeLists.txt` (+7 -1) 📝 `src/moonlight-protocol/moonlight.cpp` (+10 -1) 📝 `src/moonlight-protocol/moonlight/data-structures.hpp` (+1 -0) 📝 `src/moonlight-protocol/moonlight/protocol.hpp` (+5 -1) 📝 `src/moonlight-server/CMakeLists.txt` (+18 -0) 📝 `src/moonlight-server/api/endpoints.cpp` (+3 -1) 📝 `src/moonlight-server/events/events.hpp` (+1 -0) 📝 `src/moonlight-server/events/reflectors.hpp` (+3 -0) ➕ `src/moonlight-server/gst-plugin/pyrowave/gstpyrowaveenc.cpp` (+975 -0) ➕ `src/moonlight-server/gst-plugin/pyrowave/gstpyrowaveenc.hpp` (+57 -0) ➕ `src/moonlight-server/gst-plugin/pyrowave/pyrowave_align.hpp` (+240 -0) ➕ `src/moonlight-server/gst-plugin/pyrowave/pyrowave_device.cpp` (+404 -0) ➕ `src/moonlight-server/gst-plugin/pyrowave/pyrowave_device.hpp` (+236 -0) 📝 `src/moonlight-server/gst-plugin/video.hpp` (+136 -23) 📝 `src/moonlight-server/platforms/hw.hpp` (+16 -0) 📝 `src/moonlight-server/platforms/hw_linux.cpp` (+20 -7) 📝 `src/moonlight-server/platforms/hw_unknown.cpp` (+5 -3) 📝 `src/moonlight-server/rest/endpoints.hpp` (+4 -2) 📝 `src/moonlight-server/rtsp/commands.hpp` (+22 -4) _...and 14 more files_ </details> ### 📄 Description Closes #515 This PR adds [PyroWave](https://github.com/Themaister/pyrowave) as a fourth video codec in Wolf, alongside H.264, HEVC and AV1. Other codecs save bandwidth by sending only what changed since the previous frame. That makes encoding slower, we have to wait an extra frame to compare the current frame with the previous, and one lost packet can corrupt the frames that follow. PyroWave compresses each frame on its own with a wavelet transform on the GPU. Encoding and decoding are very fast, but it needs much more bandwidth as a result. On a wired home network, that's a good trade for lower latency. ## How - Wolf will offer PyroWave only if a working encoder was found at startup. Clients that support it (only [Aurora](https://github.com/Koloses/aurora-qt) at the moment) can request it, and if negotiation is successful, the stream begins, same as any other codec - If a client asks for PyroWave and the host can't provide it, the connection is refused instead of quietly falling back to another codec. - The encoder reads frames straight from GPU memory and uses the same GPU Wolf captures on. - The compressed data is lined up with packet boundaries, so a lost packet doesn't ruin the rest of the frame. - PyroWave is optional at build time. The Docker image builds it and includes the Vulkan runtime ## Infrastructure Changes Bitrate requested for high resolutions and framerates can get as high as 1Gbps. Some changes around bitrate limits need to be adjusted in order to properly support this codec from within Wolf. - The encoder currently rejects bitrates above 500 Mbit/s and falls back to about 15 Mbit/s, so high bitrate streams get incredibly compressed. The limit should now scale with the requested bitrate from the client - Frame pacing: - Wolf currently sends every frame's packets at a fixed rate of 80% of 1 Gbit/s, which is 71 packets/ms with 1392-byte packets. With the 20% FEC packets added, a 4K120 frame at 679 Mbit/s is 713 packets. Sending it takes 10 ms, longer than the 8.33 ms between frames, so the stream tops out at about 100 fps. - With this PR, the rate is the larger of the current fixed rate and 3x the stream's actual send rate (bitrate plus FEC), so a frame is sent in about a third of the time between frames. For the 679 Mbit/s example that is 220 packets/ms, so the 713-packet frame takes about 3.2 ms. Streams below about 267 Mbit/s keep the current rate. Allows for flexibility for users pushing past 1Gbps networks. - PyroWave support added as a Gstreamer plugin ## Comparison | PyroWave | HEVC (H.265) | |---|---| | <img width="3840" height="2160" alt="Image" src="https://github.com/user-attachments/assets/e884fb08-bf3d-4160-8041-d48418c1cfbb" /> | <img width="3840" height="2160" alt="Image" src="https://github.com/user-attachments/assets/41c71973-6f3f-4843-87f8-d84572c3673b" /> | ## Notes - Needs a Vulkan 1.3 GPU and a fast local network. - Frames over 255 packets skip error correction. That limit already existed in Wolf, but PyroWave hits it at very high bitrates. - Testing in comparisons above were done on a Wolf host with an AMD 7900 XTX streamed to a Steam Deck over 2.5Gbps LAN. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-15 13:01:05 +00:00

bsakaguchi referenced this issue

2026-10-15 13:01:07 +00:00

[PR #537] fix(rest): pause the old session's pipeline on resume #509