morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2183] feat(windows): capture target-bound native window evidence #729

New issue

Open

opened 2026-10-15 18:27:45 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:27:45 +00:00

Owner

Originally created by @SoGeRiEn on GitHub (Oct 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2183

Problem

REA can capture approved native UI evidence on macOS, but the equivalent Windows workflow is unavailable. Windows reverse-engineering investigations therefore cannot retain a screenshot of the exact already-running PE application window alongside static evidence.

Proposed narrow capability

Add an optional Windows x64, passive-only MCP/CLI workflow for one explicitly selected native window:

input binds an exact PID and HWND to an explicitly selected local executable;
verify the window belongs to that PID, retain process-start identity, and digest the selected executable;
capture only the selected window/client area with a bounded PNG result;
write one new digest-addressed artifact inside a caller-selected output directory;
return target identity, screenshot dimensions/digest, capture limitations, and evidence provenance.

Hard boundary

This is deliberately not a port of a general desktop-control MCP. It must not expose full-desktop, region, foreground-window, title-substring, arbitrary script, clipboard, keyboard, mouse, configuration, or remote-control APIs. It must not launch the target, alter permissions, or send input. Screenshot capture can still contain sensitive application data, so caller-selected output retention and limitations must be explicit.

Why a separate Windows facet

The current observe_native_ui workflow is macOS/Mach-O-specific and includes optional accessibility capture plus action scenarios. A first Windows slice should be screenshot-only and truthfully report the absent UI Automation/action facets rather than claim parity. A future proposal can add separate, target-bound UIA evidence after an actual implementation and Windows verification lane exist.

Acceptance criteria

strict input/output contracts with CLI/MCP parity;
direct Win32 target binding (PID + HWND) and process-replacement detection;
no broad desktop fallback when PrintWindow/selected-window capture fails;
bounded dimensions/output and exclusive artifact creation;
unit/boundary coverage for malformed input, wrong PID/window, replaced process, capture failure, cancellation, and unavailable Windows-native artifact;
real Windows verification against a benign public/source-owned fixture and packaged artifact.

I can provide the implementation as a separate focused PR; this issue intentionally establishes the security and contract boundary first.

Originally created by @SoGeRiEn on GitHub (Oct 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2183 ## Problem REA can capture approved native UI evidence on macOS, but the equivalent Windows workflow is unavailable. Windows reverse-engineering investigations therefore cannot retain a screenshot of the exact already-running PE application window alongside static evidence. ## Proposed narrow capability Add an optional Windows x64, passive-only MCP/CLI workflow for one explicitly selected native window: - input binds an exact PID and HWND to an explicitly selected local executable; - verify the window belongs to that PID, retain process-start identity, and digest the selected executable; - capture only the selected window/client area with a bounded PNG result; - write one new digest-addressed artifact inside a caller-selected output directory; - return target identity, screenshot dimensions/digest, capture limitations, and evidence provenance. ## Hard boundary This is deliberately not a port of a general desktop-control MCP. It must not expose full-desktop, region, foreground-window, title-substring, arbitrary script, clipboard, keyboard, mouse, configuration, or remote-control APIs. It must not launch the target, alter permissions, or send input. Screenshot capture can still contain sensitive application data, so caller-selected output retention and limitations must be explicit. ## Why a separate Windows facet The current `observe_native_ui` workflow is macOS/Mach-O-specific and includes optional accessibility capture plus action scenarios. A first Windows slice should be screenshot-only and truthfully report the absent UI Automation/action facets rather than claim parity. A future proposal can add separate, target-bound UIA evidence after an actual implementation and Windows verification lane exist. ## Acceptance criteria - strict input/output contracts with CLI/MCP parity; - direct Win32 target binding (`PID + HWND`) and process-replacement detection; - no broad desktop fallback when `PrintWindow`/selected-window capture fails; - bounded dimensions/output and exclusive artifact creation; - unit/boundary coverage for malformed input, wrong PID/window, replaced process, capture failure, cancellation, and unavailable Windows-native artifact; - real Windows verification against a benign public/source-owned fixture and packaged artifact. I can provide the implementation as a separate focused PR; this issue intentionally establishes the security and contract boundary first.

bsakaguchi added the

area: runtime

P3: low

enhancement

labels

2026-10-15 18:27:45 +00:00

bsakaguchi commented

2026-10-15 18:27:46 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 12, 2026):

Design & Boundary Review

The proposal establishes a safe, bounded design for native Windows window evidence capture:

Security & Privacy Boundaries: Narrowing capture to an explicit (PID, HWND) tuple and verifying window thread process identity via GetWindowThreadProcessId avoids general desktop surveillance and respects data minimization.
Fail-Closed Guarantees: Capturing via PrintWindow (PW_RENDERFULLCONTENT) or targeted bitblt bounded strictly to client rects ensures obscured or off-screen regions do not leak unrelated desktop contents.
Artifact Ownership: Digest-addressed PNG storage matches the existing macOS Darwin UI observation contract.

@matbrgz commented on GitHub (Oct 12, 2026): ### Design & Boundary Review The proposal establishes a safe, bounded design for native Windows window evidence capture: - **Security & Privacy Boundaries:** Narrowing capture to an explicit `(PID, HWND)` tuple and verifying window thread process identity via `GetWindowThreadProcessId` avoids general desktop surveillance and respects data minimization. - **Fail-Closed Guarantees:** Capturing via `PrintWindow` (`PW_RENDERFULLCONTENT`) or targeted bitblt bounded strictly to client rects ensures obscured or off-screen regions do not leak unrelated desktop contents. - **Artifact Ownership:** Digest-addressed PNG storage matches the existing macOS Darwin UI observation contract.

bsakaguchi commented

2026-10-16 21:02:29 +00:00

Author

Owner

@noorranktools commented on GitHub (Oct 15, 2026):

Real-world validation for the "future proposal" mentioned at the end of this issue, plus one concrete gap worth flagging before that proposal gets scoped.

During a UX review of our own company's native Win32/MFC application (immediately after static Ghidra analysis of the same binary), I needed exactly what this issue defers — not just a screenshot, but the actual control tree (names, types, bounding rects, values) of a running window, since observe_native_ui is macOS-only. I built a standalone prototype (PowerShell + inline C# via Add-Type, using only System.Windows.Automation/UIAutomationClient — built into Windows, no install) that walks the UIA tree from a (PID, HWND) pair exactly like this issue's binding model, read-only, no input injection. It worked cleanly for the app's standard controls (toolbar buttons, static labels, nested panes all showed up with correct names/AutomationIds/rects).

One concrete gap any real UIA proposal will likely hit: older Win32 common controls often don't register a UIA provider at all. In my case, a SysListView32 grid (31 columns, populated rows) showed up in the UIA tree with zero children — FindAll(TreeScope.Children, ...) returned nothing, even though the control visibly had data. Same for its SysHeader32 column header. This isn't rare — SysListView32/SysHeader32/SysTreeView32 etc. are extremely common in MFC/Win32-era apps, exactly the category of target this issue's PE-binding model is aimed at.

The content was still recoverable, just not through UIA: the classic LVM_GETITEMCOUNT/LVM_GETITEMTEXTW/HDM_GETITEMCOUNT/HDM_GETITEMW window-message technique (same approach Spy++/UISpy use), which requires allocating a small buffer in the target process via VirtualAllocEx/WriteProcessMemory/ReadProcessMemory to shuttle text through — i.e. a materially different effect than pure UIA tree-walking (touches target process memory, though nothing is injected or executed). Worth deciding up front whether that fallback belongs in the same proposal/scope as plain UIA, or needs its own separate scope conversation.

Happy to share the prototype (generic Win32 automation code, no proprietary target data, no third-party deps) as a reference if someone picks this up — not offering it as a PR, just a working reference point.

@noorranktools commented on GitHub (Oct 15, 2026): Real-world validation for the "future proposal" mentioned at the end of this issue, plus one concrete gap worth flagging before that proposal gets scoped. During a UX review of our own company's native Win32/MFC application (immediately after static Ghidra analysis of the same binary), I needed exactly what this issue defers — not just a screenshot, but the actual control tree (names, types, bounding rects, values) of a running window, since `observe_native_ui` is macOS-only. I built a standalone prototype (PowerShell + inline C# via `Add-Type`, using only `System.Windows.Automation`/`UIAutomationClient` — built into Windows, no install) that walks the UIA tree from a `(PID, HWND)` pair exactly like this issue's binding model, read-only, no input injection. It worked cleanly for the app's standard controls (toolbar buttons, static labels, nested panes all showed up with correct names/AutomationIds/rects). One concrete gap any real UIA proposal will likely hit: **older Win32 common controls often don't register a UIA provider at all.** In my case, a `SysListView32` grid (31 columns, populated rows) showed up in the UIA tree with zero children — `FindAll(TreeScope.Children, ...)` returned nothing, even though the control visibly had data. Same for its `SysHeader32` column header. This isn't rare — `SysListView32`/`SysHeader32`/`SysTreeView32` etc. are extremely common in MFC/Win32-era apps, exactly the category of target this issue's PE-binding model is aimed at. The content was still recoverable, just not through UIA: the classic `LVM_GETITEMCOUNT`/`LVM_GETITEMTEXTW`/`HDM_GETITEMCOUNT`/`HDM_GETITEMW` window-message technique (same approach Spy++/UISpy use), which requires allocating a small buffer *in the target process* via `VirtualAllocEx`/`WriteProcessMemory`/`ReadProcessMemory` to shuttle text through — i.e. a materially different effect than pure UIA tree-walking (touches target process memory, though nothing is injected or executed). Worth deciding up front whether that fallback belongs in the same proposal/scope as plain UIA, or needs its own separate scope conversation. Happy to share the prototype (generic Win32 automation code, no proprietary target data, no third-party deps) as a reference if someone picks this up — not offering it as a PR, just a working reference point.

bsakaguchi referenced this issue

2026-10-16 21:15:32 +00:00

[PR #729] [MERGED] fix(mcp): retain shutdown failure diagnostics #2195

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#729

No description provided.

Rows
Columns