games-on-whales/gst-wayland-display
1
0
Fork 0
mirror of https://github.com/games-on-whales/gst-wayland-display.git synced 2026-10-15 12:51:57 +00:00
Code Issues 12 Projects Packages Wiki Activity

[PR #37] feat(nv12): Vulkan NV12 + VA/Vulkan encode #36

New issue
Open
opened 2026-10-15 12:47:46 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-15 12:47:46 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/games-on-whales/gst-wayland-display/pull/37
Author: @JBailes
Created: 6/20/2026
Status: 🔄 Open

Base: stable ← Head: feat/vulkan-encode

📝 Commits (10+)

  • e7aeb99 feat(nv12): Vulkan RGBA->NV12 converter + direct VA encode
  • be09822 fix(nv12): share the VA display on the default render-node path
  • cbfdfe5 feat(nv12): add nv12 property to prefer NV12 output
  • b8a6340 feat(nv12): negotiate the encoder's NV12 modifier
  • b8d3bc0 feat(vulkan): zero-copy NV12 VulkanImage output for vulkanh264enc
  • e9e9ea8 style: cargo fmt
  • c07bce3 build: bundle the vulkanh264enc DPB-pool patch for the gst build
  • 8c6ef2c docs: document meson build recipe for the vkh264enc DPB patch
  • 36e0eba Merge pull request #41 from games-on-whales/feat/vulkan-encode-out
  • 1347da2 build: move gstreamer-rs to 0.25, bump Rust to 1.92

📊 Changes

48 files changed (+13670 additions, -235 deletions)

View changed files

📝 .devcontainer/Dockerfile (+114 -40)
📝 .github/workflows/harness.yml (+11 -10)
➕ .github/workflows/vulkan-image.yml (+67 -0)
➕ CONTRIBUTING.md (+66 -0)
📝 Cargo.lock (+129 -81)
📝 Cargo.toml (+2 -2)
📝 README.md (+55 -0)
📝 ci/README.md (+5 -1)
📝 ci/harness.sh (+9 -0)
➕ docker/vulkan.Dockerfile (+140 -0)
📝 gst-plugin-wayland-display/Cargo.toml (+3 -3)
📝 gst-plugin-wayland-display/src/lib.rs (+12 -0)
📝 gst-plugin-wayland-display/src/waylandsrc/imp.rs (+659 -24)
➕ gst-plugin-wayland-display/tests/encode.rs (+213 -0)
➕ patches/README.md (+95 -0)
➕ patches/vkh264enc-dpb-pool-in-new-sequence.patch (+32 -0)
➕ patches/vulkanh265enc.patch (+6116 -0)
📝 wayland-display-core/Cargo.toml (+20 -3)
📝 wayland-display-core/build.rs (+16 -2)
➕ wayland-display-core/resources/protocols/frog-color-management-v1.xml (+356 -0)

...and 28 more files

📄 Description

Vulkan RGBA to NV12 converter. waylanddisplaysrc feeds VA encoders an NV12 dmabuf direct, no vapostproc.

GLES renders RGBA. Vulkan imports that dmabuf (VK_EXT_external_memory_dma_buf), compute-shades BT.601 NV12, exports an NV12 dmabuf the encoder takes zero-copy. va_share binds it to the encoder's gst-va display so the encoder reuses one surface per ring slot instead of importing a fresh one each frame.

Pipeline:

waylanddisplaysrc ! vah265enc

A VA encoder directly downstream asks for NV12, so it negotiates on its own. Behind a format-agnostic sink like interpipesink (Wolf), set nv12=true so the source offers NV12 first. RGBA stays as a fallback.

Tested to EOS, 20 frames:

  • AMD 7900 XTX, vah265enc
  • Intel UHD 770, vah265lpenc
  • Nvidia 5080, converter exports NV12

Real content on Intel (SMPTE + moving ball, decoded): clean, no garbling or tearing.

Default build. CUDA path stacks on top.

Vulkan encode

Second consumer off the same converter: vulkanh264enc, zero-copy.

waylanddisplaysrc emits NV12 memory:VulkanImage on the encoder's shared GstVulkanDevice (via gst context). The encoder views the image, no copy, no dmabuf hop. Set vulkan=true.

Pipeline (Wolf, one process):

waylanddisplaysrc vulkan=true ! interpipesink
interpipesrc ! vulkanh264enc

Fan-out works: one capture, N encoders, mixed profiles. Seed the image's tracked layout to VIDEO_ENCODE_SRC and null its per-memory timeline semaphore so each encoder reads dependency-free; gate ring reuse on buffer release.

Tested on Nvidia 5080, gst 1.28.4 and 1.29.1:

  • one consumer, ~170 frames
  • two consumers, high + main, ~206 each

Needs the vkh264enc DPB patch (create_dpb_pool in new_sequence) and Vulkan headers >= 1.4.317 at build.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/games-on-whales/gst-wayland-display/pull/37 **Author:** [@JBailes](https://github.com/JBailes) **Created:** 6/20/2026 **Status:** 🔄 Open **Base:** `stable` ← **Head:** `feat/vulkan-encode` --- ### 📝 Commits (10+) - [`e7aeb99`](https://github.com/games-on-whales/gst-wayland-display/commit/e7aeb99ecfc68481525d7f1b4a9532af4304a082) feat(nv12): Vulkan RGBA->NV12 converter + direct VA encode - [`be09822`](https://github.com/games-on-whales/gst-wayland-display/commit/be098222c0a291aa971993482314b9ab9b821d01) fix(nv12): share the VA display on the default render-node path - [`cbfdfe5`](https://github.com/games-on-whales/gst-wayland-display/commit/cbfdfe5e4bd13334301c9875419fb5fd40dc43f9) feat(nv12): add `nv12` property to prefer NV12 output - [`b8a6340`](https://github.com/games-on-whales/gst-wayland-display/commit/b8a6340976d5bd2415a25d45ade116be60cf954e) feat(nv12): negotiate the encoder's NV12 modifier - [`b8d3bc0`](https://github.com/games-on-whales/gst-wayland-display/commit/b8d3bc0110767a97a1e81dcfaa48b30cd6959ca1) feat(vulkan): zero-copy NV12 VulkanImage output for vulkanh264enc - [`e9e9ea8`](https://github.com/games-on-whales/gst-wayland-display/commit/e9e9ea8b1e63b5280b3c9e827f20f8e2c3580e6e) style: cargo fmt - [`c07bce3`](https://github.com/games-on-whales/gst-wayland-display/commit/c07bce3c0f4a2781c924f53b803163bb393835b6) build: bundle the vulkanh264enc DPB-pool patch for the gst build - [`8c6ef2c`](https://github.com/games-on-whales/gst-wayland-display/commit/8c6ef2cac44f542a3c535913f5b4b49aa78c9907) docs: document meson build recipe for the vkh264enc DPB patch - [`36e0eba`](https://github.com/games-on-whales/gst-wayland-display/commit/36e0eba621606ea47c644463375e1434b375637b) Merge pull request #41 from games-on-whales/feat/vulkan-encode-out - [`1347da2`](https://github.com/games-on-whales/gst-wayland-display/commit/1347da2a54bde38048b5fa62d6279165c7ebc93d) build: move gstreamer-rs to 0.25, bump Rust to 1.92 ### 📊 Changes **48 files changed** (+13670 additions, -235 deletions) <details> <summary>View changed files</summary> 📝 `.devcontainer/Dockerfile` (+114 -40) 📝 `.github/workflows/harness.yml` (+11 -10) ➕ `.github/workflows/vulkan-image.yml` (+67 -0) ➕ `CONTRIBUTING.md` (+66 -0) 📝 `Cargo.lock` (+129 -81) 📝 `Cargo.toml` (+2 -2) 📝 `README.md` (+55 -0) 📝 `ci/README.md` (+5 -1) 📝 `ci/harness.sh` (+9 -0) ➕ `docker/vulkan.Dockerfile` (+140 -0) 📝 `gst-plugin-wayland-display/Cargo.toml` (+3 -3) 📝 `gst-plugin-wayland-display/src/lib.rs` (+12 -0) 📝 `gst-plugin-wayland-display/src/waylandsrc/imp.rs` (+659 -24) ➕ `gst-plugin-wayland-display/tests/encode.rs` (+213 -0) ➕ `patches/README.md` (+95 -0) ➕ `patches/vkh264enc-dpb-pool-in-new-sequence.patch` (+32 -0) ➕ `patches/vulkanh265enc.patch` (+6116 -0) 📝 `wayland-display-core/Cargo.toml` (+20 -3) 📝 `wayland-display-core/build.rs` (+16 -2) ➕ `wayland-display-core/resources/protocols/frog-color-management-v1.xml` (+356 -0) _...and 28 more files_ </details> ### 📄 Description Vulkan RGBA to NV12 converter. waylanddisplaysrc feeds VA encoders an NV12 dmabuf direct, no vapostproc. GLES renders RGBA. Vulkan imports that dmabuf (VK_EXT_external_memory_dma_buf), compute-shades BT.601 NV12, exports an NV12 dmabuf the encoder takes zero-copy. va_share binds it to the encoder's gst-va display so the encoder reuses one surface per ring slot instead of importing a fresh one each frame. Pipeline: ``` waylanddisplaysrc ! vah265enc ``` A VA encoder directly downstream asks for NV12, so it negotiates on its own. Behind a format-agnostic sink like interpipesink (Wolf), set `nv12=true` so the source offers NV12 first. RGBA stays as a fallback. Tested to EOS, 20 frames: - AMD 7900 XTX, vah265enc - Intel UHD 770, vah265lpenc - Nvidia 5080, converter exports NV12 Real content on Intel (SMPTE + moving ball, decoded): clean, no garbling or tearing. Default build. CUDA path stacks on top. ## Vulkan encode Second consumer off the same converter: vulkanh264enc, zero-copy. waylanddisplaysrc emits NV12 `memory:VulkanImage` on the encoder's shared GstVulkanDevice (via gst context). The encoder views the image, no copy, no dmabuf hop. Set `vulkan=true`. Pipeline (Wolf, one process): ``` waylanddisplaysrc vulkan=true ! interpipesink interpipesrc ! vulkanh264enc ``` Fan-out works: one capture, N encoders, mixed profiles. Seed the image's tracked layout to VIDEO_ENCODE_SRC and null its per-memory timeline semaphore so each encoder reads dependency-free; gate ring reuse on buffer release. Tested on Nvidia 5080, gst 1.28.4 and 1.29.1: - one consumer, ~170 frames - two consumers, high + main, ~206 each Needs the vkh264enc DPB patch (create_dpb_pool in new_sequence) and Vulkan headers >= 1.4.317 at build. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
stable
feat/vulkan-encode
fix/va-share-unsafe-ptr
chore/drop-dead-vulkan-share
fix/drop-vestigial-egl-bind
ci/pin-cargo-c
fix/cuda-pool-clear-object
fix/cuda-clear-object-type-confusion
feat/vulkan-encode-out
feat/nv12-modifier-negotiate
feat/nv12-dmabuftocuda
ci/test-harness
feat/nv12-output
feat/await-listener
fix/nvidia-block-linear
codex/output-resize-updates
No results found.
Labels
Clear labels   
enhancement

   
enhancement

   
pull-request
Mirrored from GitHub Pull Request

No labels
enhancement
enhancement
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
games-on-whales/gst-wayland-display#36
No description provided.