games-on-whales/gst-wayland-display
1
0
Fork 0
mirror of https://github.com/games-on-whales/gst-wayland-display.git synced 2026-10-15 12:51:57 +00:00
Code Issues 12 Projects Packages Wiki Activity

[PR #64] feat(input): tablet tool (pen) support via zwp_tablet_v2 #64

New issue
Open
opened 2026-10-15 12:48:04 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-15 12:48:04 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/games-on-whales/gst-wayland-display/pull/64
Author: @kmreisi
Created: 9/19/2026
Status: 🔄 Open

Base: stable ← Head: feat/tablet-tool-support

📝 Commits (1)

  • f84d782 feat(input): tablet tool (pen) support via zwp_tablet_v2

📊 Changes

6 files changed (+810 additions, -5 deletions)

View changed files

📝 wayland-display-core/src/comp/input.rs (+341 -3)
📝 wayland-display-core/src/comp/mod.rs (+7 -0)
📝 wayland-display-core/src/tests/client.rs (+77 -0)
📝 wayland-display-core/src/tests/mod.rs (+1 -0)
➕ wayland-display-core/src/tests/test_tablet.rs (+372 -0)
📝 wayland-display-core/src/wayland/handlers/seat.rs (+12 -2)

📄 Description

Summary

libinput tablet tool events were dropped on the floor: State::process_input_event()
had no TabletTool* arms and the compositor never exposed zwp_tablet_manager_v2.
A virtual pen tablet plugged into the compositor therefore did nothing at all — no
tablet events for clients that speak the protocol, and no pointer events either.

This adds zwp_tablet_v2 support, with a pointer-emulation fallback for clients
that do not implement the protocol.

Branch is based on current stable; nothing outside the tablet path is touched.

Behaviour

Tablet-aware clients (those that bound zwp_tablet_seat_v2) get the real thing:

  • proximity_in / proximity_out, motion, down / up
  • axes: pressure, distance, tilt, rotation, slider, wheel
  • stylus buttons as zwp_tablet_tool_v2.button with the raw libinput button code
  • the pen and the eraser end of a stylus are separate libinput tools, so they
    arrive as two separate zwp_tablet_tool_v2 objects with type = pen and
    type = eraser — that is how a client learns it should erase rather than draw

Such a client deliberately receives no wl_pointer motion for pen movement: it
would otherwise see every stroke twice (once as tablet motion, once as pointer
motion), which makes drawing applications misbehave. The cursor composited into the
streamed video still follows the pen — it is simply not delivered as pointer input.

Every other client (gamescope, most games, older toolkits) gets pointer
emulation, so a pen still works as a mouse:

pen emulated pointer
motion / hover wl_pointer.motion
tip down/up BTN_LEFT press/release
BTN_STYLUS BTN_RIGHT
any other stylus button BTN_MIDDLE

Emulated buttons are released when the pen leaves proximity or the tablet is
removed, so a pen that goes away mid-press cannot leave a stuck button behind.

Axes folded into other events. A libinput tablet tool event carries the axes
that changed with it — tip and proximity events included, not just axis events.
Those are forwarded rather than dropped; without that, the pressure going back to 0
on tip up never reaches the client and a hovering pen keeps reporting the pressure
of the last stroke.

Implementation notes

  • comp/mod.rs: TabletManagerState::new::<State>(dh) creates the
    zwp_tablet_manager_v2 global; a tablet_emulated_buttons: HashSet<u32> tracks
    the pointer buttons currently held on behalf of a pen so they can always be
    released.
  • wayland/handlers/seat.rs: TabletSeatHandler + delegate_tablet_manager!.
    tablet_tool_image() assigns the tool's cursor surface to the same
    cursor_state the pointer uses, so a client setting a pen cursor works like a
    client setting a mouse cursor.
  • comp/input.rs:
    • DeviceAdded / DeviceRemoved with DeviceCapability::TabletTool register and
      unregister the tablet on the seat's tablet seat; removing the last tablet clears
      the tools and releases any emulated button.
    • TabletToolAxes collects the *_has_changed() axes of any TabletToolEvent
      into one struct, so proximity / tip / axis events can all forward what they
      carry through the same send_tool_axes() helper.
    • Tablet-awareness is detected per surface, not per compositor: for the surface
      under the pen, the owning client's objects are walked
      (DisplayHandle::backend_handle().with_all_objects_for()) looking for a
      zwp_tablet_seat_v2. That is what decides between real tablet events and
      pointer emulation, and it is the reason a mixed session (a drawing app plus a
      game) behaves correctly for both.
    • Cursor handling: pen motion always moves the composited cursor. For a
      tablet-aware client that is a render-only update
      (tablet_cursor_only_motion() → set_pointer_location(), no wl_pointer
      event); for every other client it goes through the normal
      pointer_motion_absolute() path.
    • Tablet focus is resolved with Space::element_under(), the same lookup the
      pointer path uses, and passed to smithay's TabletToolHandle, which emits
      proximity_in / proximity_out on focus changes.
    • Pen coordinates are transformed with the existing touch_location_transformed()
      (a tablet tool reports absolute, normalised coordinates like a touchscreen).

Testing

ci/harness.sh lint build unit — all three phases pass, 30 tests, 1 ignored
(the GPU dmabuf test):

==> summary
  lint         PASS
  build        PASS
  unit         PASS

New unit tests in wayland-display-core/src/tests/test_tablet.rs:

  • tablet_aware_client_receives_tablet_events — a client that bound
    zwp_tablet_seat_v2 receives tablet added, tool added, proximity in, down,
    pressure (0.5 of the 0..65535 protocol range), tilt, motion, stylus button, up,
    proximity out, and pressure back to 0 on tip up; and asserts it gets no
    wl_pointer button or motion events for the pen.
  • eraser_is_reported_as_its_own_tool — flipping the stylus over announces a
    second tool with type = eraser, and both ends can draw.
  • non_tablet_client_gets_pointer_emulation — a client without the tablet protocol
    sees exactly BTN_LEFT press/release (tip) and BTN_RIGHT press/release
    (BTN_STYLUS), plus pointer motion for the pen.
  • pen_leaving_proximity_releases_emulated_buttons — a pen that leaves proximity
    while the tip is down leaves no button held.

The test client (tests/client.rs) gained enable_tablet() (bind
zwp_tablet_manager_v2 → get_tablet_seat) and collects
zwp_tablet_seat_v2 / zwp_tablet_tool_v2 events.

End to end

Verified through wolf with a synthetic
Moonlight client that replays a real pen packet sequence (hover, stroke with
pressure ramp, stylus button, tip up, out of proximity) over the ENet control
channel into a wolf session, plus a plain mouse replay as a sanity check. Two
configurations, both against this branch built into the wolf image:

  • tablet path — a GTK3 client (GDK binds zwp_tablet_manager_v2): receives the
    tablet events, with pressure and tilt, and no duplicated pointer motion.
  • pointer-emulation path — wev (verified at image build time not to reference
    zwp_tablet at all): receives the pen as pointer motion plus BTN_LEFT /
    BTN_RIGHT.

Tablet path (GTK3 client), 287 pen events + 53 mouse events replayed:

compositor: zwp_tablet_manager_v2 IS advertised
app: tablet proximity events=0 tip events=6 tablet motion=275 (nonzero pressure=153) stylus buttons=2
app: tools seen=['ERASER', 'PEN']
app: devices seen=[('POINTER', 'MOUSE', 'Wayland'), ('TABLET', 'PEN', 'Wolf'), ('TABLET', 'ERASER', 'Wolf')]
wire: tablet trace available=True proximity_in=3 proximity_out=3 pressure(0)=3 tool types=['320', '321']
app: tablet motions while the tip is up=119, of which with leftover pressure=0
app: wl_pointer motions after the first tablet event=0

[ok ] test app produced output
[ok ] tablet proximity in seen (wire)
[ok ] tablet proximity out seen (wire)
[ok ] tablet tip down/up seen
[ok ] tablet motion seen
[ok ] nonzero pressure seen
[ok ] pressure back to 0 after tip up
[ok ] stylus (barrel) button seen
[ok ] eraser tool seen
[ok ] no duplicated wl_pointer motion during pen input

RESULT: PASS

(proximity_in / proximity_out are graded on the zwp_tablet_tool_v2 wire trace
because GTK3/GDK does not deliver proximity events to a widget.)

Pointer-emulation path (wev):

app: pointer motion=328 pointer button=10 press=5 release=5
app: pointer button codes seen=['272', '273']

[ok ] test app produced output
[ok ] pointer motion seen
[ok ] pointer button press seen
[ok ] pointer button release seen
[ok ] left button (pen tip) seen
[ok ] right button (barrel) seen

RESULT: PASS

The same change set (on top of 67b1183, which is what wolf currently pins) was also
confirmed with a real stylus: Lenovo Idea Tab Pro with Moonlight Android streaming
into a wolf lobby.

The mouse sanity replay that runs before the pen sequence is unaffected in both
modes (pointer enter, 53 motions, button press/release delivered normally).

Note for wolf

This is only the compositor half. wolf needs a companion change to plug its virtual
pen tablet into the lobby compositor — the tablet has to be (re)created against the
compositor that owns the session's Wayland display, otherwise the events never reach
this code. The wolf side I tested with is on
kmreisi/wolf feature/xbox-ds4-generic-pads
(lobby hotplug of the pen tablet + pen event semantics); I can open that separately.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/games-on-whales/gst-wayland-display/pull/64 **Author:** [@kmreisi](https://github.com/kmreisi) **Created:** 9/19/2026 **Status:** 🔄 Open **Base:** `stable` ← **Head:** `feat/tablet-tool-support` --- ### 📝 Commits (1) - [`f84d782`](https://github.com/games-on-whales/gst-wayland-display/commit/f84d7826cf517dcb1585adfd0f21ef48ee4e891a) feat(input): tablet tool (pen) support via zwp_tablet_v2 ### 📊 Changes **6 files changed** (+810 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `wayland-display-core/src/comp/input.rs` (+341 -3) 📝 `wayland-display-core/src/comp/mod.rs` (+7 -0) 📝 `wayland-display-core/src/tests/client.rs` (+77 -0) 📝 `wayland-display-core/src/tests/mod.rs` (+1 -0) ➕ `wayland-display-core/src/tests/test_tablet.rs` (+372 -0) 📝 `wayland-display-core/src/wayland/handlers/seat.rs` (+12 -2) </details> ### 📄 Description ## Summary libinput tablet tool events were dropped on the floor: `State::process_input_event()` had no `TabletTool*` arms and the compositor never exposed `zwp_tablet_manager_v2`. A virtual pen tablet plugged into the compositor therefore did nothing at all — no tablet events for clients that speak the protocol, and no pointer events either. This adds `zwp_tablet_v2` support, with a pointer-emulation fallback for clients that do not implement the protocol. Branch is based on current `stable`; nothing outside the tablet path is touched. ## Behaviour **Tablet-aware clients** (those that bound `zwp_tablet_seat_v2`) get the real thing: - `proximity_in` / `proximity_out`, `motion`, `down` / `up` - axes: `pressure`, `distance`, `tilt`, `rotation`, `slider`, `wheel` - stylus buttons as `zwp_tablet_tool_v2.button` with the raw libinput button code - the pen and the eraser end of a stylus are separate libinput tools, so they arrive as two separate `zwp_tablet_tool_v2` objects with `type = pen` and `type = eraser` — that is how a client learns it should erase rather than draw Such a client deliberately receives **no `wl_pointer` motion** for pen movement: it would otherwise see every stroke twice (once as tablet motion, once as pointer motion), which makes drawing applications misbehave. The cursor composited into the streamed video still follows the pen — it is simply not delivered as pointer input. **Every other client** (gamescope, most games, older toolkits) gets pointer emulation, so a pen still works as a mouse: | pen | emulated pointer | | --- | --- | | motion / hover | `wl_pointer.motion` | | tip down/up | `BTN_LEFT` press/release | | `BTN_STYLUS` | `BTN_RIGHT` | | any other stylus button | `BTN_MIDDLE` | Emulated buttons are released when the pen leaves proximity or the tablet is removed, so a pen that goes away mid-press cannot leave a stuck button behind. **Axes folded into other events.** A libinput tablet tool event carries the axes that changed with it — tip and proximity events included, not just axis events. Those are forwarded rather than dropped; without that, the pressure going back to 0 on tip up never reaches the client and a hovering pen keeps reporting the pressure of the last stroke. ## Implementation notes - `comp/mod.rs`: `TabletManagerState::new::<State>(dh)` creates the `zwp_tablet_manager_v2` global; a `tablet_emulated_buttons: HashSet<u32>` tracks the pointer buttons currently held on behalf of a pen so they can always be released. - `wayland/handlers/seat.rs`: `TabletSeatHandler` + `delegate_tablet_manager!`. `tablet_tool_image()` assigns the tool's cursor surface to the same `cursor_state` the pointer uses, so a client setting a pen cursor works like a client setting a mouse cursor. - `comp/input.rs`: - `DeviceAdded` / `DeviceRemoved` with `DeviceCapability::TabletTool` register and unregister the tablet on the seat's tablet seat; removing the last tablet clears the tools and releases any emulated button. - `TabletToolAxes` collects the `*_has_changed()` axes of any `TabletToolEvent` into one struct, so proximity / tip / axis events can all forward what they carry through the same `send_tool_axes()` helper. - Tablet-awareness is detected per surface, not per compositor: for the surface under the pen, the owning client's objects are walked (`DisplayHandle::backend_handle().with_all_objects_for()`) looking for a `zwp_tablet_seat_v2`. That is what decides between real tablet events and pointer emulation, and it is the reason a mixed session (a drawing app plus a game) behaves correctly for both. - Cursor handling: pen motion always moves the composited cursor. For a tablet-aware client that is a render-only update (`tablet_cursor_only_motion()` → `set_pointer_location()`, no `wl_pointer` event); for every other client it goes through the normal `pointer_motion_absolute()` path. - Tablet focus is resolved with `Space::element_under()`, the same lookup the pointer path uses, and passed to smithay's `TabletToolHandle`, which emits `proximity_in` / `proximity_out` on focus changes. - Pen coordinates are transformed with the existing `touch_location_transformed()` (a tablet tool reports absolute, normalised coordinates like a touchscreen). ## Testing `ci/harness.sh lint build unit` — all three phases pass, 30 tests, 1 ignored (the GPU dmabuf test): ``` ==> summary lint PASS build PASS unit PASS ``` New unit tests in `wayland-display-core/src/tests/test_tablet.rs`: - `tablet_aware_client_receives_tablet_events` — a client that bound `zwp_tablet_seat_v2` receives tablet added, tool added, proximity in, down, pressure (0.5 of the 0..65535 protocol range), tilt, motion, stylus button, up, proximity out, and pressure back to 0 on tip up; and asserts it gets **no** `wl_pointer` button or motion events for the pen. - `eraser_is_reported_as_its_own_tool` — flipping the stylus over announces a second tool with `type = eraser`, and both ends can draw. - `non_tablet_client_gets_pointer_emulation` — a client without the tablet protocol sees exactly `BTN_LEFT` press/release (tip) and `BTN_RIGHT` press/release (`BTN_STYLUS`), plus pointer motion for the pen. - `pen_leaving_proximity_releases_emulated_buttons` — a pen that leaves proximity while the tip is down leaves no button held. The test client (`tests/client.rs`) gained `enable_tablet()` (bind `zwp_tablet_manager_v2` → `get_tablet_seat`) and collects `zwp_tablet_seat_v2` / `zwp_tablet_tool_v2` events. ### End to end Verified through [wolf](https://github.com/games-on-whales/wolf) with a synthetic Moonlight client that replays a real pen packet sequence (hover, stroke with pressure ramp, stylus button, tip up, out of proximity) over the ENet control channel into a wolf session, plus a plain mouse replay as a sanity check. Two configurations, both against this branch built into the wolf image: - **tablet path** — a GTK3 client (GDK binds `zwp_tablet_manager_v2`): receives the tablet events, with pressure and tilt, and no duplicated pointer motion. - **pointer-emulation path** — `wev` (verified at image build time not to reference `zwp_tablet` at all): receives the pen as pointer motion plus `BTN_LEFT` / `BTN_RIGHT`. Tablet path (GTK3 client), 287 pen events + 53 mouse events replayed: ``` compositor: zwp_tablet_manager_v2 IS advertised app: tablet proximity events=0 tip events=6 tablet motion=275 (nonzero pressure=153) stylus buttons=2 app: tools seen=['ERASER', 'PEN'] app: devices seen=[('POINTER', 'MOUSE', 'Wayland'), ('TABLET', 'PEN', 'Wolf'), ('TABLET', 'ERASER', 'Wolf')] wire: tablet trace available=True proximity_in=3 proximity_out=3 pressure(0)=3 tool types=['320', '321'] app: tablet motions while the tip is up=119, of which with leftover pressure=0 app: wl_pointer motions after the first tablet event=0 [ok ] test app produced output [ok ] tablet proximity in seen (wire) [ok ] tablet proximity out seen (wire) [ok ] tablet tip down/up seen [ok ] tablet motion seen [ok ] nonzero pressure seen [ok ] pressure back to 0 after tip up [ok ] stylus (barrel) button seen [ok ] eraser tool seen [ok ] no duplicated wl_pointer motion during pen input RESULT: PASS ``` (`proximity_in` / `proximity_out` are graded on the `zwp_tablet_tool_v2` wire trace because GTK3/GDK does not deliver proximity events to a widget.) Pointer-emulation path (`wev`): ``` app: pointer motion=328 pointer button=10 press=5 release=5 app: pointer button codes seen=['272', '273'] [ok ] test app produced output [ok ] pointer motion seen [ok ] pointer button press seen [ok ] pointer button release seen [ok ] left button (pen tip) seen [ok ] right button (barrel) seen RESULT: PASS ``` The same change set (on top of `67b1183`, which is what wolf currently pins) was also confirmed with a real stylus: Lenovo Idea Tab Pro with Moonlight Android streaming into a wolf lobby. The mouse sanity replay that runs before the pen sequence is unaffected in both modes (pointer enter, 53 motions, button press/release delivered normally). ## Note for wolf This is only the compositor half. wolf needs a companion change to plug its virtual pen tablet into the lobby compositor — the tablet has to be (re)created against the compositor that owns the session's Wayland display, otherwise the events never reach this code. The wolf side I tested with is on [kmreisi/wolf `feature/xbox-ds4-generic-pads`](https://github.com/kmreisi/wolf/commit/4633514) (lobby hotplug of the pen tablet + pen event semantics); I can open that separately. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
stable
feat/vulkan-encode
fix/va-share-unsafe-ptr
chore/drop-dead-vulkan-share
fix/drop-vestigial-egl-bind
ci/pin-cargo-c
fix/cuda-pool-clear-object
fix/cuda-clear-object-type-confusion
feat/vulkan-encode-out
feat/nv12-modifier-negotiate
feat/nv12-dmabuftocuda
ci/test-harness
feat/nv12-output
feat/await-listener
fix/nvidia-block-linear
codex/output-resize-updates
No results found.
Labels
Clear labels   
enhancement

   
enhancement

   
pull-request
Mirrored from GitHub Pull Request

No labels
enhancement
enhancement
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
games-on-whales/gst-wayland-display#64
No description provided.