[PR #897] [MERGED] feat(browser): trace captured website source-map locations #2329

New issue

Closed

opened 2026-10-16 21:17:21 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:17:21 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/897
Author: @N0zoM1z0
Created: 10/7/2026
Status: ✅ Merged
Merged: 10/7/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/web-source-location

📝 Commits (4)

7c90a63 refactor(browser): share captured-script acquisition boundaries
b769d7c feat(browser): trace captured website source-map locations
1100b59 test(browser): verify source-map artifacts and synchronize generated contracts
d2cdf76 fix(browser): preserve inert source-map URL context

📊 Changes

58 files changed (+3420 additions, -282 deletions)

View changed files

➕ .github/workflows/real-web-source-map.yml (+50 -0)
📝 README.md (+6 -1)
📝 README_ar.md (+1 -1)
📝 README_ja.md (+1 -1)
📝 README_ko.md (+1 -1)
📝 README_zh.md (+1 -1)
📝 docs/architecture.mermaid (+4 -0)
📝 docs/product-catalog.json (+16 -10)
📝 docs/testing.md (+22 -0)
📝 docs/verification/managed-conformance-manifest.json (+2 -2)
➕ docs/web-source-location.md (+97 -0)
📝 package.json (+2 -1)
➕ scripts/fixtures/browser-source-map-site.mjs (+77 -0)
➕ scripts/lib/browser-source-map-e2e.mjs (+193 -0)
➕ scripts/verify-browser-source-maps.mjs (+16 -0)
📝 skills/reverse-engineer-anything/SKILL.md (+2 -2)
➕ src/application/ValidateSelectedWebScript.ts (+48 -0)
📝 src/application/WebModulePorts.ts (+2 -7)
📝 src/application/WebModuleTraceService.ts (+10 -33)
➕ src/application/WebScriptArtifactPort.ts (+15 -0)

...and 38 more files

📄 Description

Summary

Add trace_web_source_location / trace-web-source-location: bind one retained capture script, one explicitly selected local map and one generated UTF-16 point to complete original-source evidence inline. This finishes the source-map portion of #851 after #879.
Reuse the locked, unchanged @jridgewell/trace-mapping 0.3.31 codec. Validate raw VLQ framing/32-bit quantities, section boundaries and section-local indexes before upstream flattening; preserve raw section/source identity, duplicate mappings and embedded content independently of URL identity.
Keep artifact acquisition, shared workflow, codec protocol and owned process lifecycle behind narrow ports/modules. Share the existing module trace's captured-script reader without changing its public behavior.
Run the trusted codec in an owned Node subprocess with a private request root, independent heap flags, one V8 worker and measured heap-limit evidence. Remove NODE_OPTIONS only from the child. Wait for output closure, preserve actual failures and reject unconfirmed cleanup with the owning operation/resources.
Add a separate conditional real Chrome/source-map CI lane with isolated esbuild 0.25.10 fixture generation. Ordinary static/unit gates acquire neither Chrome nor this compiler. Regenerate public catalogs, skill metadata and documentation.

Closes #851. Refs #774. Runtime attribution remains separate in #881.

Evidence semantics and limits

Map pairing and URL context are caller-selected; authenticity and execution remain unknown. The exact inert map context URL, including file/webpack schemes or userinfo, is preserved as local evidence and has no transport authority. No URL fetch or captured application execution occurs during tracing. Lookup returns all equal-position segments at the greatest generated position <= the selected point, including earlier lines and generated-only mappings. Missing embedded source remains unavailable; out-of-range original coordinates retain their reported location and a null offset.

Regular and inline indexed maps are supported; external section URLs are explicitly unsupported. Unknown extensions remain in raw map text. Embedded-content hashes identify UTF-8 encoding of the decoded sourcesContent text. Complete codec evidence has 4 MiB map / 32 MiB reply, 262144 row/segment and 64-level section budgets; there is no silent truncation. The 192 MiB old-generation budget is independently measured (local V8 heap limit 226492416 bytes), not an aggregate RSS claim. The codec deadline is 20 seconds; cleanup may extend it.

Sanitized MCP input

{
  "manifest_path": "/analysis/scripts/manifest.json",
  "script_index": 0,
  "source_map": {
    "path": "/analysis/app.js.map",
    "url": "https://example.test/maps/app.js.map?v=7#context"
  },
  "generated_position": { "line": 1, "column": 25 }
}

CLI: rea trace-web-source-location MANIFEST INDEX MAP_PATH MAP_URL LINE COLUMN --json.

Verification

110 tests / 14 files passed on the preceding base; after rebasing onto 0804722c and preserving the upstream JavaScript directory reorganization, 56 tests / 6 affected files passed again on the final code, covering reader/byte identity, UTF-16/CRLF, duplicate and indexed mappings, overlapping sections, local-index leakage, malformed/overflowing VLQ, output amplification, real owned codec processes, cancellation, launch/cleanup failures, existing module/process regressions, actual SDK JSON Schema 2020 and public CLI input/results.
Before-fix installed public workflow rejected an inert map context URL; the final packaged workflow preserves it, including Evidence parameters, without fetching it.
Before-fix section-local source-index regression: 1 failed / 19 passed; corrected before release. Actual upstream overflow probe returned a false generated column; raw framing validation now rejects it.
npm run check:fast, npm run format:check, npm run knip, npm run docs:check:built passed.
npm pack and isolated tarball installation passed. The tarball includes the codec entrypoint and excludes reference/private/fixture paths (1367838 bytes packed, 11798287 unpacked).
Real Chrome 153 / esbuild 0.25.10 capture → export → trace passed through the installed tarball's CLI and stdio MCP. Four public cases (CLI and MCP, each with standard and inert userinfo URL contexts) return the independently known original point (line 3, UTF-16 column 25), exact CRLF/Unicode original text and digest; no map URL fetch. Final verifier lineage had no live descendants.
Real workflow coverage is Linux x64 / Node 24.18.0. Other host source-map workflows remain unverified. No real Hopper/Ghidra verification was run; their adapters are unchanged.

Focused command:

npm run test:focused -- \
  src/domain/providerCleanupError.test.ts src/domain/webSourcePosition.test.ts \
  src/javascript/sourceMaps/TraceSourceMap.test.ts \
  src/application/WebSourceLocationService.test.ts src/application/WebModuleTraceService.test.ts \
  src/process/ProviderProcessOutput.test.ts \
  tests/boundary/javascript/sourceMapDecoder.test.ts \
  tests/boundary/mcp/webSourceLocationMcp.test.ts tests/boundary/mcp/webModuleTraceMcp.test.ts \
  tests/acceptance/webSourceLocationCli.test.ts \
  tests/boundary/filesystem/webSourceLocationArtifacts.test.ts \
  tests/boundary/filesystem/webModuleArtifacts.test.ts \
  tests/boundary/process/providerProcess.test.ts src/contracts/toolContracts.test.ts

Real packaged command (caller-supplied Chrome and isolated fixture compiler):

REA_BROWSER_EXECUTABLE=/absolute/path/to/chromium \
REA_WEB_SOURCE_MAP_COMPILER=/absolute/path/to/fixture-tools/node_modules/esbuild/lib/main.js \
node scripts/verify-browser-source-maps.mjs /absolute/path/to/installed/rea-agents/scripts/rea.mjs

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/897 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/7/2026 **Status:** ✅ Merged **Merged:** 10/7/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/web-source-location` --- ### 📝 Commits (4) - [`7c90a63`](https://github.com/morluto/rea/commit/7c90a635fe07298a9b7e96d28b1bd28820700583) refactor(browser): share captured-script acquisition boundaries - [`b769d7c`](https://github.com/morluto/rea/commit/b769d7c28807326f4ae82cbc28dc9f4d7548dc61) feat(browser): trace captured website source-map locations - [`1100b59`](https://github.com/morluto/rea/commit/1100b595f2d4ed8a75459d43c4d0b236b6d90647) test(browser): verify source-map artifacts and synchronize generated contracts - [`d2cdf76`](https://github.com/morluto/rea/commit/d2cdf76cbf36df7270efddd525a1379952eecc4f) fix(browser): preserve inert source-map URL context ### 📊 Changes **58 files changed** (+3420 additions, -282 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/real-web-source-map.yml` (+50 -0) 📝 `README.md` (+6 -1) 📝 `README_ar.md` (+1 -1) 📝 `README_ja.md` (+1 -1) 📝 `README_ko.md` (+1 -1) 📝 `README_zh.md` (+1 -1) 📝 `docs/architecture.mermaid` (+4 -0) 📝 `docs/product-catalog.json` (+16 -10) 📝 `docs/testing.md` (+22 -0) 📝 `docs/verification/managed-conformance-manifest.json` (+2 -2) ➕ `docs/web-source-location.md` (+97 -0) 📝 `package.json` (+2 -1) ➕ `scripts/fixtures/browser-source-map-site.mjs` (+77 -0) ➕ `scripts/lib/browser-source-map-e2e.mjs` (+193 -0) ➕ `scripts/verify-browser-source-maps.mjs` (+16 -0) 📝 `skills/reverse-engineer-anything/SKILL.md` (+2 -2) ➕ `src/application/ValidateSelectedWebScript.ts` (+48 -0) 📝 `src/application/WebModulePorts.ts` (+2 -7) 📝 `src/application/WebModuleTraceService.ts` (+10 -33) ➕ `src/application/WebScriptArtifactPort.ts` (+15 -0) _...and 38 more files_ </details> ### 📄 Description ## Summary - Add `trace_web_source_location` / `trace-web-source-location`: bind one retained capture script, one explicitly selected local map and one generated UTF-16 point to complete original-source evidence inline. This finishes the source-map portion of #851 after #879. - Reuse the locked, unchanged `@jridgewell/trace-mapping` 0.3.31 codec. Validate raw VLQ framing/32-bit quantities, section boundaries and section-local indexes before upstream flattening; preserve raw section/source identity, duplicate mappings and embedded content independently of URL identity. - Keep artifact acquisition, shared workflow, codec protocol and owned process lifecycle behind narrow ports/modules. Share the existing module trace's captured-script reader without changing its public behavior. - Run the trusted codec in an owned Node subprocess with a private request root, independent heap flags, one V8 worker and measured heap-limit evidence. Remove `NODE_OPTIONS` only from the child. Wait for output closure, preserve actual failures and reject unconfirmed cleanup with the owning operation/resources. - Add a separate conditional real Chrome/source-map CI lane with isolated esbuild 0.25.10 fixture generation. Ordinary static/unit gates acquire neither Chrome nor this compiler. Regenerate public catalogs, skill metadata and documentation. Closes #851. Refs #774. Runtime attribution remains separate in #881. ## Evidence semantics and limits Map pairing and URL context are caller-selected; authenticity and execution remain unknown. The exact inert map context URL, including file/webpack schemes or userinfo, is preserved as local evidence and has no transport authority. No URL fetch or captured application execution occurs during tracing. Lookup returns all equal-position segments at the greatest generated position <= the selected point, including earlier lines and generated-only mappings. Missing embedded source remains unavailable; out-of-range original coordinates retain their reported location and a null offset. Regular and inline indexed maps are supported; external section URLs are explicitly unsupported. Unknown extensions remain in raw map text. Embedded-content hashes identify UTF-8 encoding of the decoded `sourcesContent` text. Complete codec evidence has 4 MiB map / 32 MiB reply, 262144 row/segment and 64-level section budgets; there is no silent truncation. The 192 MiB old-generation budget is independently measured (local V8 heap limit 226492416 bytes), not an aggregate RSS claim. The codec deadline is 20 seconds; cleanup may extend it. ## Sanitized MCP input ```json { "manifest_path": "/analysis/scripts/manifest.json", "script_index": 0, "source_map": { "path": "/analysis/app.js.map", "url": "https://example.test/maps/app.js.map?v=7#context" }, "generated_position": { "line": 1, "column": 25 } } ``` CLI: `rea trace-web-source-location MANIFEST INDEX MAP_PATH MAP_URL LINE COLUMN --json`. ## Verification - 110 tests / 14 files passed on the preceding base; after rebasing onto `0804722c` and preserving the upstream JavaScript directory reorganization, 56 tests / 6 affected files passed again on the final code, covering reader/byte identity, UTF-16/CRLF, duplicate and indexed mappings, overlapping sections, local-index leakage, malformed/overflowing VLQ, output amplification, real owned codec processes, cancellation, launch/cleanup failures, existing module/process regressions, actual SDK JSON Schema 2020 and public CLI input/results. - Before-fix installed public workflow rejected an inert map context URL; the final packaged workflow preserves it, including Evidence parameters, without fetching it. - Before-fix section-local source-index regression: 1 failed / 19 passed; corrected before release. Actual upstream overflow probe returned a false generated column; raw framing validation now rejects it. - `npm run check:fast`, `npm run format:check`, `npm run knip`, `npm run docs:check:built` passed. - `npm pack` and isolated tarball installation passed. The tarball includes the codec entrypoint and excludes reference/private/fixture paths (1367838 bytes packed, 11798287 unpacked). - Real Chrome 153 / esbuild 0.25.10 capture → export → trace passed through the **installed tarball's CLI and stdio MCP**. Four public cases (CLI and MCP, each with standard and inert userinfo URL contexts) return the independently known original point (line 3, UTF-16 column 25), exact CRLF/Unicode original text and digest; no map URL fetch. Final verifier lineage had no live descendants. - Real workflow coverage is Linux x64 / Node 24.18.0. Other host source-map workflows remain unverified. No real Hopper/Ghidra verification was run; their adapters are unchanged. Focused command: ```bash npm run test:focused -- \ src/domain/providerCleanupError.test.ts src/domain/webSourcePosition.test.ts \ src/javascript/sourceMaps/TraceSourceMap.test.ts \ src/application/WebSourceLocationService.test.ts src/application/WebModuleTraceService.test.ts \ src/process/ProviderProcessOutput.test.ts \ tests/boundary/javascript/sourceMapDecoder.test.ts \ tests/boundary/mcp/webSourceLocationMcp.test.ts tests/boundary/mcp/webModuleTraceMcp.test.ts \ tests/acceptance/webSourceLocationCli.test.ts \ tests/boundary/filesystem/webSourceLocationArtifacts.test.ts \ tests/boundary/filesystem/webModuleArtifacts.test.ts \ tests/boundary/process/providerProcess.test.ts src/contracts/toolContracts.test.ts ``` Real packaged command (caller-supplied Chrome and isolated fixture compiler): ```bash REA_BROWSER_EXECUTABLE=/absolute/path/to/chromium \ REA_WEB_SOURCE_MAP_COMPILER=/absolute/path/to/fixture-tools/node_modules/esbuild/lib/main.js \ node scripts/verify-browser-source-maps.mjs /absolute/path/to/installed/rea-agents/scripts/rea.mjs ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:17:21 +00:00