[PR #209] [MERGED] feat: add web and Electron reverse-engineering workflows #1862

New issue

Closed

opened 2026-10-16 21:11:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:11:53 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/209
Author: @N0zoM1z0
Created: 7/14/2026
Status: ✅ Merged
Merged: 7/14/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/web-reverse-engineering-hardening

📝 Commits (2)

7e97a31 feat: add web and Electron reverse-engineering workflows
f8f49ee fix(browser): support relative source map URLs

📊 Changes

106 files changed (+11942 additions, -406 deletions)

View changed files

📝 CONTRIBUTING.md (+2 -2)
📝 README.md (+17 -15)
📝 README_ar.md (+5 -4)
📝 README_ja.md (+5 -4)
📝 README_ko.md (+5 -4)
📝 README_zh.md (+5 -4)
📝 docs/architecture.mermaid (+1 -1)
📝 docs/browser-observation.md (+50 -11)
➕ docs/electron-observation.md (+35 -0)
📝 docs/mcp-prompts.md (+1 -1)
📝 package-lock.json (+1 -3)
📝 package.json (+1 -0)
➕ scripts/fixtures/browser-verifier-site.mjs (+120 -0)
📝 scripts/rea.mjs (+1 -1)
📝 scripts/verify-package.mjs (+9 -4)
📝 scripts/verify-published-package.mjs (+2 -2)
📝 scripts/verify-real-browser.mjs (+250 -70)
📝 scripts/verify-real-hopper.mjs (+1 -1)
📝 skills/rea-analysis/SKILL.md (+19 -9)
📝 src/application/BrowserEvidence.ts (+141 -21)

...and 80 more files

📄 Description

Summary

This PR turns REA's initial point-in-time CDP observation support into a bounded website reverse-engineering workflow, while keeping browser-specific mechanics behind the existing application ports. It also adds a narrowly scoped Electron renderer provider for local file:// applications.

The public inventory grows from 70 to 78 tools:

33 direct Hopper tools
10 enhanced Hopper workflows
5 native inspection tools
2 artifact tools
8 browser tools
2 Electron tools
18 session tools

CLI and MCP expose equivalent contracts for every new operation.

What changed

Website evidence and static reconstruction

Reconciles CDP performance resources with debugger scripts into a deterministic, deduplicated inventory.
Stores approved script bodies as content-addressed evidence rather than embedding unbounded source in ordinary observations.
Adds bounded JavaScript bundle analysis for chunks, route hints, endpoint hints, vendor signals, imports, and static WebMCP registrations.
Adds explicitly approved source-map retrieval with same-origin policy checks, MIME/JSON validation, per-map and aggregate byte limits, original-source/module graphs, and generated-to-original mappings through @jridgewell/trace-mapping.
Resolves relative sourceMappingURL declarations against the debugger script URL before applying exact-origin authorization and URL redaction. This behavior is covered by the fake CDP seam and the real-Chrome verifier.
Corrects CDP initiator parsing to use the nested protocol stack shape and preserves initiator provenance in request evidence.

Runtime observation

Adds action-window browser sessions that record requests, console metadata, WebSocket message shapes, SPA navigation, same-origin reloads, redirects, failures, and policy exits on a bounded timeline.
Adds deterministic capture comparison that ignores transient request IDs while preserving unknown/incomplete dimensions.
Adds explicitly approved PNG screenshots and bounded decoded-pixel visual comparison.
Adds runtime WebMCP discovery without invoking page tools or evaluating arbitrary page code.
Adds safe response metadata capture for allowlisted headers, CSP, Link relations, sanitized href/src attributes, and agent-discovery hints.

Privacy, policy, and truthfulness

Introduces explicit completeness states: complete_within_window, policy_filtered, attach_limited, and truncated, including excluded/dropped counts where applicable.
Keeps accessibility text and console text behind independent opt-ins and byte/item limits; sensitive console text is redacted before bounded capture.
Captures JSON response schemas and WebSocket payload shapes by default instead of raw bodies/messages.
Rechecks frame URL/origin during screenshots, WebMCP discovery, observation sessions, and Electron inspection so navigation races cannot mix evidence from a different target.
Confines Electron inspection to canonical operator-approved roots and rejects renderer paths that leave those roots.

Packaging and documentation

Registers the new CLI commands, MCP tools, permission subjects, configuration fields, capability inventory, package metadata, and skill guidance.
Expands the real-browser verifier with a local HTTP/WebSocket fixture covering initiators, bundle/source-map analysis, safe metadata, SPA sessions, capture diff, and screenshot diff.
Makes verifier process/profile cleanup bounded and reliable across repeated real-Chrome runs, and emits bounded source-map diagnostics on assertion failure.
Updates the architecture and localized README inventories to the 78-tool contract.

Verification

Passed locally:

Focused new-feature suite with coverage disabled: 24 test files / 128 tests passed.
npm run check
- strict typecheck, oxlint, Prettier, build, and all tests
- 100 test files / 654 tests passed
- coverage: 81.30% statements, 70.60% branches, 83.63% functions, 83.66% lines
- browser layer coverage: 86.16% statements, 72.67% branches, 93.81% functions, 88.64% lines
npm run knip
- no unused files, exports, or dependencies
npm run jscpd
- 1.61% duplicated lines / 2.02% duplicated tokens, within the configured threshold
npm run scan:todos
- no TODO/FIXME/HACK markers
npx -y -p node@22.19.0 -p npm@11.16.0 sh -c 'node --version && npm --version && npm run verify:package'
- packaged CLI, setup plan/readback/idempotence, backups, failure recovery, skill, two MCP clients, 78 tools, 6 prompts, target lifecycle, and bounded regex bridge all passed
REA_BROWSER_EXECUTABLE=/tmp/rea-browser-verification/chrome-linux64/chrome npm run verify:browser
- passed twice consecutively with Chrome 150.0.7871.115
- each run captured 13 DOM nodes, 10 accessibility nodes, 1 script, 16 network requests, 7 console events, 12 WebSocket events, 1 analyzed bundle, 1 reconstructed source map, 6 session events, and a 7,065-byte screenshot
- verified nested CDP initiators, approved/redacted source artifacts, relative source-map resolution, original sources/module edges/mappings, JSON and WebSocket shape capture, safe metadata, SPA timelines, deterministic capture diff, PNG visual diff, and clean browser/profile shutdown

Real Hopper verification was not performed because this host does not have two distinct Hopper target binaries. npm run verify:hopper builds successfully and then reports the missing HOPPER_TARGET_PATH and HOPPER_SECOND_TARGET_PATH. This PR does not change Hopper protocol behavior.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/209 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 7/14/2026 **Status:** ✅ Merged **Merged:** 7/14/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/web-reverse-engineering-hardening` --- ### 📝 Commits (2) - [`7e97a31`](https://github.com/morluto/rea/commit/7e97a31b020ec970d6a1618100bd5f3c4ed1094c) feat: add web and Electron reverse-engineering workflows - [`f8f49ee`](https://github.com/morluto/rea/commit/f8f49ee5b44a641e1239a29662b17b641df0cf65) fix(browser): support relative source map URLs ### 📊 Changes **106 files changed** (+11942 additions, -406 deletions) <details> <summary>View changed files</summary> 📝 `CONTRIBUTING.md` (+2 -2) 📝 `README.md` (+17 -15) 📝 `README_ar.md` (+5 -4) 📝 `README_ja.md` (+5 -4) 📝 `README_ko.md` (+5 -4) 📝 `README_zh.md` (+5 -4) 📝 `docs/architecture.mermaid` (+1 -1) 📝 `docs/browser-observation.md` (+50 -11) ➕ `docs/electron-observation.md` (+35 -0) 📝 `docs/mcp-prompts.md` (+1 -1) 📝 `package-lock.json` (+1 -3) 📝 `package.json` (+1 -0) ➕ `scripts/fixtures/browser-verifier-site.mjs` (+120 -0) 📝 `scripts/rea.mjs` (+1 -1) 📝 `scripts/verify-package.mjs` (+9 -4) 📝 `scripts/verify-published-package.mjs` (+2 -2) 📝 `scripts/verify-real-browser.mjs` (+250 -70) 📝 `scripts/verify-real-hopper.mjs` (+1 -1) 📝 `skills/rea-analysis/SKILL.md` (+19 -9) 📝 `src/application/BrowserEvidence.ts` (+141 -21) _...and 80 more files_ </details> ### 📄 Description ## Summary This PR turns REA's initial point-in-time CDP observation support into a bounded website reverse-engineering workflow, while keeping browser-specific mechanics behind the existing application ports. It also adds a narrowly scoped Electron renderer provider for local `file://` applications. The public inventory grows from 70 to 78 tools: - 33 direct Hopper tools - 10 enhanced Hopper workflows - 5 native inspection tools - 2 artifact tools - 8 browser tools - 2 Electron tools - 18 session tools CLI and MCP expose equivalent contracts for every new operation. ## What changed ### Website evidence and static reconstruction - Reconciles CDP performance resources with debugger scripts into a deterministic, deduplicated inventory. - Stores approved script bodies as content-addressed evidence rather than embedding unbounded source in ordinary observations. - Adds bounded JavaScript bundle analysis for chunks, route hints, endpoint hints, vendor signals, imports, and static WebMCP registrations. - Adds explicitly approved source-map retrieval with same-origin policy checks, MIME/JSON validation, per-map and aggregate byte limits, original-source/module graphs, and generated-to-original mappings through `@jridgewell/trace-mapping`. - Resolves relative `sourceMappingURL` declarations against the debugger script URL before applying exact-origin authorization and URL redaction. This behavior is covered by the fake CDP seam and the real-Chrome verifier. - Corrects CDP initiator parsing to use the nested protocol stack shape and preserves initiator provenance in request evidence. ### Runtime observation - Adds action-window browser sessions that record requests, console metadata, WebSocket message shapes, SPA navigation, same-origin reloads, redirects, failures, and policy exits on a bounded timeline. - Adds deterministic capture comparison that ignores transient request IDs while preserving unknown/incomplete dimensions. - Adds explicitly approved PNG screenshots and bounded decoded-pixel visual comparison. - Adds runtime WebMCP discovery without invoking page tools or evaluating arbitrary page code. - Adds safe response metadata capture for allowlisted headers, CSP, Link relations, sanitized `href`/`src` attributes, and agent-discovery hints. ### Privacy, policy, and truthfulness - Introduces explicit completeness states: `complete_within_window`, `policy_filtered`, `attach_limited`, and `truncated`, including excluded/dropped counts where applicable. - Keeps accessibility text and console text behind independent opt-ins and byte/item limits; sensitive console text is redacted before bounded capture. - Captures JSON response schemas and WebSocket payload shapes by default instead of raw bodies/messages. - Rechecks frame URL/origin during screenshots, WebMCP discovery, observation sessions, and Electron inspection so navigation races cannot mix evidence from a different target. - Confines Electron inspection to canonical operator-approved roots and rejects renderer paths that leave those roots. ### Packaging and documentation - Registers the new CLI commands, MCP tools, permission subjects, configuration fields, capability inventory, package metadata, and skill guidance. - Expands the real-browser verifier with a local HTTP/WebSocket fixture covering initiators, bundle/source-map analysis, safe metadata, SPA sessions, capture diff, and screenshot diff. - Makes verifier process/profile cleanup bounded and reliable across repeated real-Chrome runs, and emits bounded source-map diagnostics on assertion failure. - Updates the architecture and localized README inventories to the 78-tool contract. ## Verification Passed locally: - Focused new-feature suite with coverage disabled: 24 test files / 128 tests passed. - `npm run check` - strict typecheck, oxlint, Prettier, build, and all tests - 100 test files / 654 tests passed - coverage: 81.30% statements, 70.60% branches, 83.63% functions, 83.66% lines - browser layer coverage: 86.16% statements, 72.67% branches, 93.81% functions, 88.64% lines - `npm run knip` - no unused files, exports, or dependencies - `npm run jscpd` - 1.61% duplicated lines / 2.02% duplicated tokens, within the configured threshold - `npm run scan:todos` - no TODO/FIXME/HACK markers - `npx -y -p node@22.19.0 -p npm@11.16.0 sh -c 'node --version && npm --version && npm run verify:package'` - packaged CLI, setup plan/readback/idempotence, backups, failure recovery, skill, two MCP clients, 78 tools, 6 prompts, target lifecycle, and bounded regex bridge all passed - `REA_BROWSER_EXECUTABLE=/tmp/rea-browser-verification/chrome-linux64/chrome npm run verify:browser` - passed twice consecutively with Chrome 150.0.7871.115 - each run captured 13 DOM nodes, 10 accessibility nodes, 1 script, 16 network requests, 7 console events, 12 WebSocket events, 1 analyzed bundle, 1 reconstructed source map, 6 session events, and a 7,065-byte screenshot - verified nested CDP initiators, approved/redacted source artifacts, relative source-map resolution, original sources/module edges/mappings, JSON and WebSocket shape capture, safe metadata, SPA timelines, deterministic capture diff, PNG visual diff, and clean browser/profile shutdown Real Hopper verification was not performed because this host does not have two distinct Hopper target binaries. `npm run verify:hopper` builds successfully and then reports the missing `HOPPER_TARGET_PATH` and `HOPPER_SECOND_TARGET_PATH`. This PR does not change Hopper protocol behavior. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:11:53 +00:00