morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #3529] feat(reconstruction): verify visual screenshot claims #5808

New issue

Open

opened 2026-10-31 01:09:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:09:35 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3529
Author: @josenavarrojimenez-sudo
Created: 10/14/2026
Status: 🔄 Open

Base: main ← Head: feat/visual-reconstruction-claims

📝 Commits (3)

17371c0 feat(reconstruction): verify visual screenshot claims
89a5843 feat(adb): retain screen capture pixels for visual claims
9b03476 feat(adb): record the keyguard state with each screen capture

📊 Changes

19 files changed (+1377 additions, -59 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/references/evidence-workflows.md (+7 -0)
📝 docs/adb-device-analysis.md (+27 -1)
📝 scripts/lib/browser-screenshot-e2e.mjs (+1 -1)
📝 src/adb/AdbProvider.test.ts (+65 -3)
📝 src/adb/AdbProvider.ts (+34 -1)
📝 src/adb/AdbShellOutput.test.ts (+13 -1)
📝 src/adb/AdbShellOutput.ts (+12 -0)
📝 src/browser/CdpBrowserProvider.ts (+1 -1)
📝 src/cli/adbCommands.ts (+5 -0)
📝 src/contracts/adb/adbToolContracts.ts (+9 -1)
📝 src/contracts/sessionToolContracts.ts (+1 -1)
📝 src/domain/adb/adbDeviceAnalysis.ts (+36 -9)
📝 src/domain/reconstructionUnknowns.ts (+11 -5)
📝 src/domain/reconstructionVerification.ts (+61 -7)
➕ src/domain/reconstructionVerification.visual.test.ts (+645 -0)
📝 src/domain/reconstructionVerificationSchemas.ts (+63 -3)
➕ src/domain/reconstructionVisualClaims.ts (+261 -0)
📝 src/domain/webScreenshotPixels.test.ts (+40 -3)
📝 src/domain/webScreenshotPixels.ts (+85 -22)

📄 Description

Summary

Screenshot comparisons can't currently participate in reconstruction verification. verify_reconstruction accepts behavioral and structural claims only, so a compare_web_screenshots record has no typed way to support or refute a reconstruction, even though it is the natural evidence for "the rebuilt screen renders like the original".

This PR adds a visual claim kind. A visual claim cites one compare_web_screenshots record and its two source captures, and is decided against a threshold, a changed-ratio bound and an optional mask that the specification declares up front.

No issue is linked; this is a proposed enhancement within the existing reconstruction-verification contract, opened as a draft for maintainer scope review.

Problem and expected behavior

Before: a specification can't reference screenshot Evidence at all; visual parity has to be checked out of band, without provenance or unknown tracking.
After: a visual claim:
1. validates the comparison identity (compare_web_screenshots, rea.web-screenshot-diff, rea-cdp-browser v2, observed, external-service) through the same providers table the other kinds use;
2. binds each side to its declared source Evidence (capture_web_screenshot or capture_adb_screen) by artifact SHA-256 and, for ADB, dimensions;
3. recomputes the diff from the retained PNG pixels when both sources keep them inline (web captures always do; ADB captures do when taken with include_artifact: true), and requires the recomputed metrics to equal the record;
4. applies the declared channel_threshold, max_changed_ratio and optional mask (rectangular regions, a required rationale, and a max_masked_ratio), computing the decision ratio outside the mask;
5. reports residual unknowns and recommended probes like the other kinds (default probe: capture_web_screenshot).

Situation	Outcome
Recomputed ratio ≤ declared maximum	`pass`
Ratio > maximum, or dimension mismatch	`fail`
ADB capture taken with `include_artifact: true`	recomputed: `pass` / `fail`
Source is a digest-only ADB capture (no pixels in the bundle)	`unknown`
Record threshold ≠ declared threshold	`unknown`
Same capture on both sides, or a non-observed source	`unknown`
Mask excludes more than `max_masked_ratio`, or excludes everything	`unknown`
ADB source captured while the keyguard was showing (`keyguard_showing: true`)	`unknown`
An open unknown cites a source or the comparison	`unknown`
Source isn't the artifact the comparison cites	error
Inline bytes disagree with their sealed digest	error
ADB artifact digest or size differs from the capture's reported `sha256`/`bytes`	error
Recomputed metrics disagree with the record	error
Comparison identity or authority mismatch	error
Mask region outside the screenshot	error
Mask without a rationale	rejected by the schema

Change and scope

Pixel code moves to the domain. src/browser/PngVisualDiff.ts becomes src/domain/webScreenshotPixels.ts (with git mv), so the pure domain verifier can recompute comparisons without breaking module boundaries. comparePngScreenshots is unchanged; the new measurePngScreenshots(input, excludedRegions) also counts changed pixels outside the excluded regions. CdpBrowserProvider and the browser screenshot e2e script now import from the new path. No new dependencies.
New src/domain/reconstructionVisualClaims.ts holds the binding, recomputation and mask logic. reconstructionVerification.ts routes visual claims to it and reuses the shared classification, unknowns and limitations.
Schemas (reconstructionVerificationSchemas.ts):
- a visualClaimSchema (dimension pixels);
- an optional per-claim visual result block: threshold, maximum, compared/changed/masked pixels, changed ratio, and recomputed;
- a summary.visual count. It is always emitted, but optional when parsing so results recorded before this change still validate. The kind-count invariant treats it as 0 when absent.
Docs: the verify_reconstruction tool description and the skill's evidence-workflows reference.

Intentionally not included:

Derived artifacts. Crop or resize derivations between a source capture and the compared artifact aren't supported: the comparison doesn't retain bytes and the domain doesn't read disk, so the compared artifact must be exactly the source capture.
Recomputing ADB captures from their local file. The verifier never reads local_path; ADB pixels are recomputed only when the Evidence itself carries them (see below), so bundles stay portable.
A mismatch between the recomputed and recorded metrics is an error, not a limitation, because both run the same RGBA algorithm.

Opt-in ADB pixel retention (`include_artifact`)

Second commit. capture_adb_screen Evidence carried only the PNG digest, size and dimensions, so visual claims citing an ADB capture always stayed unknown.

New input include_artifact: boolean (default false; CLI --include-artifact). When true, the result also carries artifact: {sha256, bytes, media_type: "image/png", data_base64}, the same self-verifying shape capture_web_screenshot returns, so the artifact can be passed directly to compare_web_screenshots.
Same identity. The artifact uses the digest and size the capture already reports. The capture_adb_screen result schema rejects an artifact whose sha256/bytes differ from the capture, and the artifact schema rejects inline bytes that don't match their own digest.
Bounds. Captures stay limited by the existing 32 MiB screencap output limit. Base64 adds about a third (a 1080×2400 frame is typically 100–400 KB of PNG), which is why the default keeps results digest-only and unchanged. When omitted, artifact is absent, so previously recorded Evidence still parses.
Verifier. reconstructionVisualClaims uses the ADB artifact when present and recomputes like a web capture. Digest-only ADB sources stay unknown, and their limitation and the default visual probe now point at include_artifact.
Docs: the capture_adb_screen tool description and example, docs/adb-device-analysis.md, and the skill's evidence-workflows reference.
Tests: visual-claim tests for a recomputed ADB pass, a recomputed ADB fail, an artifact that isn't the reported PNG, and forged inline bytes; an AdbProvider test for default output (no artifact) and opt-in output whose artifact equals the written screen.png byte for byte. All PNGs are synthetic and built in code.
Real device: later exercised against a physical Android 17 phone over ADB (see the keyguard section below); scripts/verify-adb.mjs is unchanged.

Keyguard state on ADB captures

Third commit, from running visual claims against a real phone over long, unattended capture sessions. The device turned its screen off and locked about 30 seconds later, and screencap kept returning a valid PNG: the lock screen. Nothing in the Evidence said so, so a visual claim compared the lock screen with the reconstruction and reported an ordinary fail.

New output keyguard_showing: boolean | null on capture_adb_screen. Right after the frame, the provider reads dumpsys window displays (fixed command, same 8 MiB limit as inspect_adb_window) and parses isKeyguardShowing=. null means the window manager did not declare it; an adb failure on that read also yields null and never fails the capture, except cancellation, which propagates.
Limitation per state: showing ("the PNG likely shows the lock screen"), not showing ("a lock transition during the frame is not excluded"), or undeclared ("unknown").
Verifier: an ADB source with keyguard_showing: true adds a limitation, so the visual claim is unknown with "unlock the device and recapture", rather than a misleading fail. false and null behave as before.
Compatibility: the field is optional in the result schema, so ADB captures retained before it existed still parse.
Docs: docs/adb-device-analysis.md gains "Capture an unattended device". REA never wakes or unlocks the device. Keep-alive is the caller's job, within the caller's authorization; for example, a harmless key event sent only while the expected app is focused. Don't change the screen-timeout or lock settings, and never type a PIN. The section also explains how to render a web reconstruction at the device's exact pixel size: --force-device-scale-factor tuned until the dimensions match, safe-area insets for the system bars, and those bars declared as a masked region.
Tests: parseKeyguardShowing (CRLF, false, undeclared); an AdbProvider test whose stub declares isKeyguardShowing=true; the existing capture test now expects keyguard_showing: null when dumpsys fails; visual-claim tests for a locked source (unknown, still recomputed) and an unlocked one (pass).

Observed on the real phone (1080×2400), with the application in front:

capture_adb_screen returned keyguard_showing: false, matching isKeyguardShowing=false from a direct dumpsys;
a visual claim built from an include_artifact capture and a CDP screenshot rendered at the device scale recomputed its decision (recomputed: true). The compared pixels excluded the masked status and gesture bars.

Contract and boundary impact

Semantic owner and earliest changed stage: domain reconstruction-verification schema and verifier.
Contribution scope fit and linked maintainer decision: extends an existing read-only verifier with no target interaction; maintainer decision pending.
CLI and MCP/tool-catalog contract:
- verify_reconstruction input accepts a new claim kind;
- the output adds summary.visual and an optional per-claim visual block;
- the tool description is updated;
- capture_adb_screen input adds optional include_artifact (default false) and its output adds an optional artifact.
- capture_adb_screen output adds keyguard_showing (optional when parsing) and runs one extra read-only dumpsys window displays after the frame.
Provider, bridge, target-format, or platform compatibility: none; the CDP screenshot provider imports the moved module.
Evidence, artifact, provenance, or reconstruction contract:
- additive: existing behavioral and structural claims evaluate as before;
- comparison identity checks now read confidence and authority from the providers table, with the same values as before for the existing kinds.
Target selection, authority, upstream behavior, and cleanup details: not applicable.
Generated metadata: none committed; dist/, skills/ and the product catalog aren't part of the diff.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated:
- new src/domain/reconstructionVerification.visual.test.ts (12 tests). All fixtures are synthetic 4×4 RGBA PNGs built in code: pass, fail by ratio, fail on dimension mismatch, unknown for a digest-only ADB source, unknown for a different threshold, unknown for a reused capture, unknown for an open unknown, mask (pass with exclusion, oversized mask unknown, out-of-bounds error, blank rationale rejected), and errors for an uncited source, a tampered digest, altered metrics, and a wrong comparison identity;
- src/domain/webScreenshotPixels.test.ts (the moved diff test) gains a masked-measurement test.
Base reproduction or other evidence: on main, verify_reconstruction rejects any claim whose kind is visual at schema validation.
User-visible CLI/MCP output: a passing claim includes "visual": {"channel_threshold": 0, "max_changed_ratio": 0.1, "compared_pixels": 16, "changed_pixels": 1, "changed_ratio": 0.0625, "masked_pixels": 0, "recomputed": true}.
Remaining proof gaps: no real-browser end-to-end run of a visual claim through the MCP server; coverage is at the domain level.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npx vitest run src/domain/reconstructionVerification.visual.test.ts — 12 passed
npx vitest run src/domain/webScreenshotPixels.test.ts src/domain/reconstructionVerification* — 4 files, 110 passed
npx vitest run src/domain src/contracts src/browser tests/conformance — 239 files, 3686 passed, 3 skipped
npx vitest run on the 10 acceptance/boundary files that reference reconstruction verification, screenshot comparison or session tool contracts (after npm run build) — 82 passed, 1 failed: localToolContracts.test.ts > accepts NUL as actual terminal input… failed with cleanup_incomplete (process ownership token ESRCH on a loaded host); rerun alone it passed. It is unrelated to this change.
npm run check:fast (typecheck, lint, module boundaries) — passed
npm run verify:test-discovery — "Discovered all 1282 test files exactly once."
npm run docs:check — 5 passed
npm run build — passed
npx oxfmt --check . — all files formatted

Re-run after the include_artifact commit:

npm run test:local -- src/domain/reconstructionVerification.visual.test.ts src/adb/AdbProvider.test.ts src/domain/webScreenshotPixels.test.ts — 3 files, 97 passed (visual: 16)
npx vitest run src/domain src/contracts src/adb src/cli src/server tests/boundary/mcp/adbMcp.test.ts — 198 files, 3228 passed, 3 skipped
npm run build, npm run check:fast, npm run docs:check — passed; Prettier check on the changed files and the pre-commit oxfmt/oxlint — clean
npx vitest run tests/conformance tests/boundary/cli — 2 failures that reproduce identically on the previous commit without this change (cliSetup.test.ts > TTY setup dry run > shows the refusal…, clientProfileLifecycle.test.ts > selected client profiles lifecycle… timing out at 30 s); a third (javascriptApplicationProgress.test.ts, memory constraint under load) passed when rerun alone. Unrelated to this change.
A full-repository npx vitest run exceeded my local 28-minute limit and wasn't completed; CI is the authority for the rest of the suite.

Re-run after the keyguard commit:

npx vitest run src/adb src/domain/reconstruction — 12 files, 260 passed (visual: 18, AdbProvider: 63)
npm run compile, npx tsc --noEmit, npm run docs:check — passed; Prettier check on the changed files and the pre-commit oxfmt/oxlint — clean; oxlint on src/adb reports the same 34 pre-existing warnings as before the commit
node scripts/rea.mjs capture-adb-screen <serial> --output-directory … --format json --full-output against a physical phone — keyguard_showing: false with the matching limitation

Compatibility, safety, and release

Breaking changes or migration steps:
- none for callers: the new kind and output fields are additive;
- summary.visual is optional when parsing older results;
- src/browser/PngVisualDiff.ts is now src/domain/webScreenshotPixels.ts. It is an internal module and not a package export.
Real Hopper/Ghidra, browser, or OS coverage: none; the verifier is pure domain code.
Package or release metadata impact: none; the changelog is left to release-please.
Security, privacy, process, or containment review: read-only over the Evidence bundle; no disk, network or process access.
Limits: pixel parity under a threshold is not behavioral equivalence. A visual pass covers only the declared screens, threshold and mask; it says nothing about interaction, state or data.

Author readiness

Human owner:

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3529 **Author:** [@josenavarrojimenez-sudo](https://github.com/josenavarrojimenez-sudo) **Created:** 10/14/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/visual-reconstruction-claims` --- ### 📝 Commits (3) - [`17371c0`](https://github.com/morluto/rea/commit/17371c05a1cb24043e7a0ad0a312203f556635f9) feat(reconstruction): verify visual screenshot claims - [`89a5843`](https://github.com/morluto/rea/commit/89a5843c66102a527bf6adeeab1a994e379ac05b) feat(adb): retain screen capture pixels for visual claims - [`9b03476`](https://github.com/morluto/rea/commit/9b034762af46a4703a42564c12987bbecc511be3) feat(adb): record the keyguard state with each screen capture ### 📊 Changes **19 files changed** (+1377 additions, -59 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/references/evidence-workflows.md` (+7 -0) 📝 `docs/adb-device-analysis.md` (+27 -1) 📝 `scripts/lib/browser-screenshot-e2e.mjs` (+1 -1) 📝 `src/adb/AdbProvider.test.ts` (+65 -3) 📝 `src/adb/AdbProvider.ts` (+34 -1) 📝 `src/adb/AdbShellOutput.test.ts` (+13 -1) 📝 `src/adb/AdbShellOutput.ts` (+12 -0) 📝 `src/browser/CdpBrowserProvider.ts` (+1 -1) 📝 `src/cli/adbCommands.ts` (+5 -0) 📝 `src/contracts/adb/adbToolContracts.ts` (+9 -1) 📝 `src/contracts/sessionToolContracts.ts` (+1 -1) 📝 `src/domain/adb/adbDeviceAnalysis.ts` (+36 -9) 📝 `src/domain/reconstructionUnknowns.ts` (+11 -5) 📝 `src/domain/reconstructionVerification.ts` (+61 -7) ➕ `src/domain/reconstructionVerification.visual.test.ts` (+645 -0) 📝 `src/domain/reconstructionVerificationSchemas.ts` (+63 -3) ➕ `src/domain/reconstructionVisualClaims.ts` (+261 -0) 📝 `src/domain/webScreenshotPixels.test.ts` (+40 -3) 📝 `src/domain/webScreenshotPixels.ts` (+85 -22) </details> ### 📄 Description ## Summary Screenshot comparisons can't currently participate in reconstruction verification. `verify_reconstruction` accepts behavioral and structural claims only, so a `compare_web_screenshots` record has no typed way to support or refute a reconstruction, even though it is the natural evidence for "the rebuilt screen renders like the original". This PR adds a `visual` claim kind. A visual claim cites one `compare_web_screenshots` record and its two source captures, and is decided against a threshold, a changed-ratio bound and an optional mask that the specification declares up front. No issue is linked; this is a proposed enhancement within the existing reconstruction-verification contract, opened as a draft for maintainer scope review. ## Problem and expected behavior - **Before:** a specification can't reference screenshot Evidence at all; visual parity has to be checked out of band, without provenance or unknown tracking. - **After:** a `visual` claim: 1. validates the comparison identity (`compare_web_screenshots`, `rea.web-screenshot-diff`, `rea-cdp-browser` v2, observed, external-service) through the same providers table the other kinds use; 2. binds each side to its declared source Evidence (`capture_web_screenshot` or `capture_adb_screen`) by artifact SHA-256 and, for ADB, dimensions; 3. recomputes the diff from the retained PNG pixels when both sources keep them inline (web captures always do; ADB captures do when taken with `include_artifact: true`), and requires the recomputed metrics to equal the record; 4. applies the declared `channel_threshold`, `max_changed_ratio` and optional `mask` (rectangular regions, a required rationale, and a `max_masked_ratio`), computing the decision ratio outside the mask; 5. reports residual unknowns and recommended probes like the other kinds (default probe: `capture_web_screenshot`). | Situation | Outcome | | --- | --- | | Recomputed ratio ≤ declared maximum | `pass` | | Ratio > maximum, or dimension mismatch | `fail` | | ADB capture taken with `include_artifact: true` | recomputed: `pass` / `fail` | | Source is a digest-only ADB capture (no pixels in the bundle) | `unknown` | | Record threshold ≠ declared threshold | `unknown` | | Same capture on both sides, or a non-observed source | `unknown` | | Mask excludes more than `max_masked_ratio`, or excludes everything | `unknown` | | ADB source captured while the keyguard was showing (`keyguard_showing: true`) | `unknown` | | An open unknown cites a source or the comparison | `unknown` | | Source isn't the artifact the comparison cites | error | | Inline bytes disagree with their sealed digest | error | | ADB artifact digest or size differs from the capture's reported `sha256`/`bytes` | error | | Recomputed metrics disagree with the record | error | | Comparison identity or authority mismatch | error | | Mask region outside the screenshot | error | | Mask without a rationale | rejected by the schema | ## Change and scope - **Pixel code moves to the domain.** `src/browser/PngVisualDiff.ts` becomes `src/domain/webScreenshotPixels.ts` (with `git mv`), so the pure domain verifier can recompute comparisons without breaking module boundaries. `comparePngScreenshots` is unchanged; the new `measurePngScreenshots(input, excludedRegions)` also counts changed pixels outside the excluded regions. `CdpBrowserProvider` and the browser screenshot e2e script now import from the new path. No new dependencies. - **New `src/domain/reconstructionVisualClaims.ts`** holds the binding, recomputation and mask logic. `reconstructionVerification.ts` routes `visual` claims to it and reuses the shared classification, unknowns and limitations. - **Schemas** (`reconstructionVerificationSchemas.ts`): - a `visualClaimSchema` (dimension `pixels`); - an optional per-claim `visual` result block: threshold, maximum, compared/changed/masked pixels, changed ratio, and `recomputed`; - a `summary.visual` count. It is always emitted, but optional when parsing so results recorded before this change still validate. The kind-count invariant treats it as 0 when absent. - **Docs:** the `verify_reconstruction` tool description and the skill's evidence-workflows reference. **Intentionally not included:** - **Derived artifacts.** Crop or resize derivations between a source capture and the compared artifact aren't supported: the comparison doesn't retain bytes and the domain doesn't read disk, so the compared artifact must be exactly the source capture. - **Recomputing ADB captures from their local file.** The verifier never reads `local_path`; ADB pixels are recomputed only when the Evidence itself carries them (see below), so bundles stay portable. - A mismatch between the recomputed and recorded metrics is an error, not a limitation, because both run the same RGBA algorithm. ## Opt-in ADB pixel retention (`include_artifact`) Second commit. `capture_adb_screen` Evidence carried only the PNG digest, size and dimensions, so visual claims citing an ADB capture always stayed `unknown`. - **New input** `include_artifact: boolean` (default `false`; CLI `--include-artifact`). When `true`, the result also carries `artifact: {sha256, bytes, media_type: "image/png", data_base64}`, the same self-verifying shape `capture_web_screenshot` returns, so the artifact can be passed directly to `compare_web_screenshots`. - **Same identity.** The artifact uses the digest and size the capture already reports. The `capture_adb_screen` result schema rejects an artifact whose `sha256`/`bytes` differ from the capture, and the artifact schema rejects inline bytes that don't match their own digest. - **Bounds.** Captures stay limited by the existing 32 MiB `screencap` output limit. Base64 adds about a third (a 1080×2400 frame is typically 100–400 KB of PNG), which is why the default keeps results digest-only and unchanged. When omitted, `artifact` is absent, so previously recorded Evidence still parses. - **Verifier.** `reconstructionVisualClaims` uses the ADB artifact when present and recomputes like a web capture. Digest-only ADB sources stay `unknown`, and their limitation and the default visual probe now point at `include_artifact`. - **Docs:** the `capture_adb_screen` tool description and example, `docs/adb-device-analysis.md`, and the skill's evidence-workflows reference. - **Tests:** visual-claim tests for a recomputed ADB pass, a recomputed ADB fail, an artifact that isn't the reported PNG, and forged inline bytes; an `AdbProvider` test for default output (no `artifact`) and opt-in output whose artifact equals the written `screen.png` byte for byte. All PNGs are synthetic and built in code. - **Real device:** later exercised against a physical Android 17 phone over ADB (see the keyguard section below); `scripts/verify-adb.mjs` is unchanged. ## Keyguard state on ADB captures Third commit, from running visual claims against a real phone over long, unattended capture sessions. The device turned its screen off and locked about 30 seconds later, and `screencap` kept returning a valid PNG: the lock screen. Nothing in the Evidence said so, so a visual claim compared the lock screen with the reconstruction and reported an ordinary `fail`. - **New output** `keyguard_showing: boolean | null` on `capture_adb_screen`. Right after the frame, the provider reads `dumpsys window displays` (fixed command, same 8 MiB limit as `inspect_adb_window`) and parses `isKeyguardShowing=`. `null` means the window manager did not declare it; an `adb` failure on that read also yields `null` and never fails the capture, except cancellation, which propagates. - **Limitation per state:** showing ("the PNG likely shows the lock screen"), not showing ("a lock transition during the frame is not excluded"), or undeclared ("unknown"). - **Verifier:** an ADB source with `keyguard_showing: true` adds a limitation, so the visual claim is `unknown` with "unlock the device and recapture", rather than a misleading `fail`. `false` and `null` behave as before. - **Compatibility:** the field is optional in the result schema, so ADB captures retained before it existed still parse. - **Docs:** `docs/adb-device-analysis.md` gains "Capture an unattended device". REA never wakes or unlocks the device. Keep-alive is the caller's job, within the caller's authorization; for example, a harmless key event sent only while the expected app is focused. Don't change the screen-timeout or lock settings, and never type a PIN. The section also explains how to render a web reconstruction at the device's exact pixel size: `--force-device-scale-factor` tuned until the dimensions match, safe-area insets for the system bars, and those bars declared as a masked region. - **Tests:** `parseKeyguardShowing` (CRLF, `false`, undeclared); an `AdbProvider` test whose stub declares `isKeyguardShowing=true`; the existing capture test now expects `keyguard_showing: null` when `dumpsys` fails; visual-claim tests for a locked source (`unknown`, still recomputed) and an unlocked one (`pass`). Observed on the real phone (1080×2400), with the application in front: - `capture_adb_screen` returned `keyguard_showing: false`, matching `isKeyguardShowing=false` from a direct `dumpsys`; - a visual claim built from an `include_artifact` capture and a CDP screenshot rendered at the device scale recomputed its decision (`recomputed: true`). The compared pixels excluded the masked status and gesture bars. ## Contract and boundary impact - **Semantic owner and earliest changed stage:** domain reconstruction-verification schema and verifier. - **Contribution scope fit and linked maintainer decision:** extends an existing read-only verifier with no target interaction; maintainer decision pending. - **CLI and MCP/tool-catalog contract:** - `verify_reconstruction` input accepts a new claim kind; - the output adds `summary.visual` and an optional per-claim `visual` block; - the tool description is updated; - `capture_adb_screen` input adds optional `include_artifact` (default `false`) and its output adds an optional `artifact`. - `capture_adb_screen` output adds `keyguard_showing` (optional when parsing) and runs one extra read-only `dumpsys window displays` after the frame. - **Provider, bridge, target-format, or platform compatibility:** none; the CDP screenshot provider imports the moved module. - **Evidence, artifact, provenance, or reconstruction contract:** - additive: existing behavioral and structural claims evaluate as before; - comparison identity checks now read confidence and authority from the providers table, with the same values as before for the existing kinds. - **Target selection, authority, upstream behavior, and cleanup details:** not applicable. - **Generated metadata:** none committed; `dist/`, `skills/` and the product catalog aren't part of the diff. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - **Tests added or updated:** - new `src/domain/reconstructionVerification.visual.test.ts` (12 tests). All fixtures are synthetic 4×4 RGBA PNGs built in code: pass, fail by ratio, fail on dimension mismatch, unknown for a digest-only ADB source, unknown for a different threshold, unknown for a reused capture, unknown for an open unknown, mask (pass with exclusion, oversized mask unknown, out-of-bounds error, blank rationale rejected), and errors for an uncited source, a tampered digest, altered metrics, and a wrong comparison identity; - `src/domain/webScreenshotPixels.test.ts` (the moved diff test) gains a masked-measurement test. - **Base reproduction or other evidence:** on `main`, `verify_reconstruction` rejects any claim whose `kind` is `visual` at schema validation. - **User-visible CLI/MCP output:** a passing claim includes `"visual": {"channel_threshold": 0, "max_changed_ratio": 0.1, "compared_pixels": 16, "changed_pixels": 1, "changed_ratio": 0.0625, "masked_pixels": 0, "recomputed": true}`. - **Remaining proof gaps:** no real-browser end-to-end run of a visual claim through the MCP server; coverage is at the domain level. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npx vitest run src/domain/reconstructionVerification.visual.test.ts` — 12 passed - `npx vitest run src/domain/webScreenshotPixels.test.ts src/domain/reconstructionVerification*` — 4 files, 110 passed - `npx vitest run src/domain src/contracts src/browser tests/conformance` — 239 files, 3686 passed, 3 skipped - `npx vitest run` on the 10 acceptance/boundary files that reference reconstruction verification, screenshot comparison or session tool contracts (after `npm run build`) — 82 passed, 1 failed: `localToolContracts.test.ts > accepts NUL as actual terminal input…` failed with `cleanup_incomplete` (process ownership token `ESRCH` on a loaded host); rerun alone it passed. It is unrelated to this change. - `npm run check:fast` (typecheck, lint, module boundaries) — passed - `npm run verify:test-discovery` — "Discovered all 1282 test files exactly once." - `npm run docs:check` — 5 passed - `npm run build` — passed - `npx oxfmt --check .` — all files formatted Re-run after the `include_artifact` commit: - `npm run test:local -- src/domain/reconstructionVerification.visual.test.ts src/adb/AdbProvider.test.ts src/domain/webScreenshotPixels.test.ts` — 3 files, 97 passed (visual: 16) - `npx vitest run src/domain src/contracts src/adb src/cli src/server tests/boundary/mcp/adbMcp.test.ts` — 198 files, 3228 passed, 3 skipped - `npm run build`, `npm run check:fast`, `npm run docs:check` — passed; Prettier check on the changed files and the pre-commit `oxfmt`/`oxlint` — clean - `npx vitest run tests/conformance tests/boundary/cli` — 2 failures that reproduce identically on the previous commit without this change (`cliSetup.test.ts > TTY setup dry run > shows the refusal…`, `clientProfileLifecycle.test.ts > selected client profiles lifecycle…` timing out at 30 s); a third (`javascriptApplicationProgress.test.ts`, memory constraint under load) passed when rerun alone. Unrelated to this change. - A full-repository `npx vitest run` exceeded my local 28-minute limit and wasn't completed; CI is the authority for the rest of the suite. Re-run after the keyguard commit: - `npx vitest run src/adb src/domain/reconstruction` — 12 files, 260 passed (visual: 18, `AdbProvider`: 63) - `npm run compile`, `npx tsc --noEmit`, `npm run docs:check` — passed; Prettier check on the changed files and the pre-commit `oxfmt`/`oxlint` — clean; `oxlint` on `src/adb` reports the same 34 pre-existing warnings as before the commit - `node scripts/rea.mjs capture-adb-screen <serial> --output-directory … --format json --full-output` against a physical phone — `keyguard_showing: false` with the matching limitation ## Compatibility, safety, and release - **Breaking changes or migration steps:** - none for callers: the new kind and output fields are additive; - `summary.visual` is optional when parsing older results; - `src/browser/PngVisualDiff.ts` is now `src/domain/webScreenshotPixels.ts`. It is an internal module and not a package export. - **Real Hopper/Ghidra, browser, or OS coverage:** none; the verifier is pure domain code. - **Package or release metadata impact:** none; the changelog is left to release-please. - **Security, privacy, process, or containment review:** read-only over the Evidence bundle; no disk, network or process access. - **Limits:** pixel parity under a threshold is not behavioral equivalence. A visual `pass` covers only the declared screens, threshold and mask; it says nothing about interaction, state or data. ## Author readiness Human owner:  - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-31 01:09:35 +00:00

bsakaguchi referenced this issue

2026-11-01 21:11:00 +00:00

[GH-ISSUE #5653] [Tracker] New issue intake by area (October 2026) #3423

~~bsakaguchi referenced this issue 2026-11-01 21:11:14 +00:00~~

[GH-ISSUE #8167] [Feature] Recover individual Kuna decompiles that fail in a completed project export #4693

bsakaguchi referenced this issue

2026-11-01 21:45:15 +00:00

[PR #5818] [MERGED] feat(sqlite): make worker deadlines adjustable with recovery guidance #7080