morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5952] [Bug] Bundle analysis hides source maps excluded before fetching #3623

New issue

Closed

opened 2026-10-31 00:42:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:42:28 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5952

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The built analyze-web-bundle CLI loses the policy refusal for a source map declared by a retained same-origin script. With --fetch-source-maps, a declaration pointing outside the allowed origins produces source_maps.status: "not_requested", zero requested/processed maps, complete bundle analysis, and no source-map unknown. inspect-web-page against the same simulated page reports the actual source_maps / disallowed_origin exclusion count.

With two retained scripts, one declaring an allowed map and the other an excluded map, bundle analysis reports only the allowed map as included, complete, without the exclusion. In contrast, an allowed map URL redirecting outside the allowed origins correctly produces an unavailable collection, a policy_filtered item, partial bundle analysis, and a source-map unknown.

These results came from compiled CLI subprocesses against the repository's simulated CDP peer over real HTTP/WebSocket transport. No installed Chrome or independent MCP session was exercised.

Reproduction

Use the checkout at the revision below with its locked dependencies and built runtime. Save this diagnostic test as tests/boundary/browser/sourceMapPolicyCoverage.test.ts. Set REPRO_STATE to an existing disposable directory outside the checkout to isolate CLI home/configuration/cache/data state, then run:

node node_modules/vitest/vitest.mjs run tests/boundary/browser/sourceMapPolicyCoverage.test.ts --maxWorkers=1

The following is the complete test executed, with only its filename changed. Its assertions describe the observed defect and controls, so a passing diagnostic test is evidence of the current behavior.

import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";

const run = promisify(execFile);
it("compares missing, excluded, mixed and redirect-filtered map declarations", async () => {
  const state = process.env.REPRO_STATE;
  if (!state) throw new Error("Set REPRO_STATE to a disposable directory");
  const source = "export const observed = true;";
  for (const mode of ["none", "excluded", "mixed", "redirect"] as const) {
    const browser = await startFakeCdpBrowser({
      sourceMapBody: JSON.stringify({ version: 3, names: [], sources: ["main.js"],
        sourcesContent: ["export const original = true;"], mappings: "AAAA" }),
      sourceMapRedirect: mode === "redirect" ? "https://cdn.example.test/main.js.map" : undefined,
      commandResult: ({ method }) => method === "Debugger.getScriptSource"
        ? { scriptSource: source } : undefined,
      commandEvents: (command, origin) => {
        if (command.method !== "Debugger.enable") return undefined;
        const script = (id: string, sourceMapURL?: string) => ({
          method: "Debugger.scriptParsed",
          ...(command.sessionId === undefined ? {} : { sessionId: command.sessionId }),
          params: { scriptId: id, url: `${origin}/${id}.js`, hash: id,
            length: Buffer.byteLength(source), isModule: true, executionContextId: 1,
            scriptLanguage: "JavaScript",
            ...(sourceMapURL === undefined ? {} : { sourceMapURL }) },
        });
        return [script("main", mode === "none" ? undefined : mode === "redirect"
          ? "/app.js.map" : "https://cdn.example.test/main.js.map"),
          ...(mode === "mixed" ? [script("second", "/app.js.map")] : [])];
      },
    });
    try {
      const env = { ...process.env, HOME: `${state}/home`, XDG_CONFIG_HOME: `${state}/config`,
        XDG_CACHE_HOME: `${state}/cache`, XDG_DATA_HOME: `${state}/data` };
      const cli = async (command: string, flags: string[] = []) => {
        const { stdout, stderr } = await run(process.execPath, ["scripts/rea.mjs", command,
          browser.endpoint, "allowed-page", "--observation-ms", "0", "--json", ...flags],
          { env, timeout: 30000, maxBuffer: 4 * 1024 * 1024 });
        expect(stderr).toBe("");
        const output = JSON.parse(stdout);
        return output.normalized_result ?? output;
      };
      const inspection = await cli("inspect-web-page");
      const analysis = await cli("analyze-web-bundle", ["--fetch-source-maps"]);
      const maps = analysis.observations.source_maps;
      const exclusions = inspection.completeness.excluded.filter(
        (entry: { section: string }) => entry.section === "source_maps");
      console.log(JSON.stringify({ mode, exclusions, source_maps: { status: maps.status,
        requested: maps.requested, processed: maps.processed,
        item_statuses: maps.items.map((item: { status: string }) => item.status) },
        completeness: analysis.completeness.status, unknown_dimensions:
        analysis.unknowns.map((item: { dimension: string }) => item.dimension) }));
      expect(analysis.completeness.parse_failures).toBe(0);
      expect(analysis.capture.scripts_analyzed).toBe(mode === "mixed" ? 2 : 1);
      if (mode === "excluded" || mode === "mixed") {
        expect(exclusions).toEqual([{ section: "source_maps", reason: "disallowed_origin", count: 1 }]);
        expect(analysis.completeness.status).toBe("complete");
        expect(analysis.unknowns).toEqual([]);
        expect(maps.status).toBe(mode === "mixed" ? "included" : "not_requested");
        expect(maps.requested).toBe(mode === "mixed" ? 1 : 0);
      } else if (mode === "redirect") {
        expect(maps.status).toBe("unavailable");
        expect(maps.items[0].status).toBe("policy_filtered");
        expect(analysis.completeness.status).toBe("partial");
        expect(analysis.unknowns[0].dimension).toBe("source_maps");
      } else {
        expect(exclusions).toEqual([]);
        expect(maps.status).toBe("not_requested");
        expect(analysis.completeness.status).toBe("complete");
        expect(analysis.unknowns).toEqual([]);
      }
    } finally { await browser.close(); }
  }
}, 120000);

Remove the diagnostic test and its disposable state directory afterward. Each mode starts and closes its own simulated browser. The two CLI calls in each mode inspect and analyze equivalent repeatable captures; they do not reuse a saved inspection.

Expected behavior

When source-map fetching is selected, preserve the known policy exclusion for map declarations associated with retained scripts. Report its count/reason or a suitable unavailable/partial source-map outcome, without fetching or exposing disallowed URL details. Do not silently make a retained script with an excluded declaration indistinguishable from one with no declaration.

The source-map outcome schema provides policy_filtered map outcomes with limitations. The existing fetcher tests expect an unavailable collection with a policy-filtered item for a declared URL on another origin, without contacting it. The observation contract counts policy exclusions without retaining their details.

Keep the no-declarations control complete with no invented source-map unknown. not_requested is valid for an empty collection; this report concerns a known excluded declaration being lost before that collection is built.

Impact and scope

P2: the captured JavaScript and successfully fetched maps remain available, but callers lose the reason that original-source/map coverage is incomplete for an admitted script. The observed scope is a map declared on another origin under default target-origin scope, alone or alongside an admitted map. Real-site frequency and other exclusion reasons were not measured.

A separate inspect-web-page call demonstrably exposes the policy exclusion count, providing a workaround for recognizing the gap. This report does not request expanding origin authority or fetching an excluded URL.

REA version

6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node v26.12.0; locked Vitest v5.0.3. Compiled CLI with the repository's simulated CDP HTTP/WebSocket peer; no native analysis provider.

Target and artifact context

Synthetic same-origin JavaScript scripts. The excluded declaration points to https://cdn.example.test/main.js.map. The admitted-map control serves a small valid v3 map with one inline original source. The redirect control redirects the admitted map URL to that same excluded origin.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The executed diagnostic printed:

{"mode":"none","exclusions":[],"source_maps":{"status":"not_requested","requested":0,"processed":0,"item_statuses":[]},"completeness":"complete","unknown_dimensions":[]}
{"mode":"excluded","exclusions":[{"section":"source_maps","reason":"disallowed_origin","count":1}],"source_maps":{"status":"not_requested","requested":0,"processed":0,"item_statuses":[]},"completeness":"complete","unknown_dimensions":[]}
{"mode":"mixed","exclusions":[{"section":"source_maps","reason":"disallowed_origin","count":1}],"source_maps":{"status":"included","requested":1,"processed":1,"item_statuses":["included"]},"completeness":"complete","unknown_dimensions":[]}
{"mode":"redirect","exclusions":[],"source_maps":{"status":"unavailable","requested":1,"processed":1,"item_statuses":["policy_filtered"]},"completeness":"partial","unknown_dimensions":["source_maps"]}

All eight CLI calls succeeded. All four analyses retained and parsed the expected scripts with zero parse failures. The focused test passed. exclusions above is the source-map subset of inspect-web-page's ledger; the map status and completeness fields come from analyze-web-bundle with fetching selected.

Typed outcome or recovery expected

A policy-excluded declaration associated with a retained script should survive as explicit source-map unavailability/partial coverage or an aggregate exclusion with its reason/count. Keep allowed maps and captured source findings. An absent declaration must remain distinguishable from a policy-excluded one; keep the no-map control complete and the redirect-filtered control partial.

Analysis or suspected cause

Source-derived explanation of the executed result: capture admission clears both map URL fields after recording the policy exclusion. Request construction only forwards non-null map declarations. The provider selects fetching, but the empty-list path sees zero requests and returns not_requested.

The bundle result builder consults capture omissions only through a resource-budget-only filter. A disallowed_origin map exclusion therefore contributes neither a limitation/unknown nor partial map coverage. The mixed case loses the same exclusion while retaining the successful subset.

Additional context

The checked upstream main revision 705058f3e2df82503f1c601fc464aea7cbb626b8 has no changes to these capture, fetcher or result-projection paths relative to the tested baseline. No regression range is established. Closed issue #3217 and closed PR #3262 concern selecting source-map fetching when no map is declared; that valid empty case is preserved as a control here. Issue #829 and PR #1120 concern URL-policy vocabulary and policy-classification tests, rather than loss of the declaration outcome during bundle projection.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5952 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The built `analyze-web-bundle` CLI loses the policy refusal for a source map declared by a retained same-origin script. With `--fetch-source-maps`, a declaration pointing outside the allowed origins produces `source_maps.status: "not_requested"`, zero requested/processed maps, complete bundle analysis, and no source-map unknown. `inspect-web-page` against the same simulated page reports the actual `source_maps` / `disallowed_origin` exclusion count. With two retained scripts, one declaring an allowed map and the other an excluded map, bundle analysis reports only the allowed map as `included`, complete, without the exclusion. In contrast, an allowed map URL redirecting outside the allowed origins correctly produces an unavailable collection, a `policy_filtered` item, partial bundle analysis, and a source-map unknown. These results came from compiled CLI subprocesses against the repository's simulated CDP peer over real HTTP/WebSocket transport. No installed Chrome or independent MCP session was exercised. ### Reproduction Use the checkout at the revision below with its locked dependencies and built runtime. Save this diagnostic test as `tests/boundary/browser/sourceMapPolicyCoverage.test.ts`. Set `REPRO_STATE` to an existing disposable directory outside the checkout to isolate CLI home/configuration/cache/data state, then run: ```sh node node_modules/vitest/vitest.mjs run tests/boundary/browser/sourceMapPolicyCoverage.test.ts --maxWorkers=1 ``` The following is the complete test executed, with only its filename changed. Its assertions describe the observed defect and controls, so a passing diagnostic test is evidence of the current behavior. ```typescript import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { expect, it } from "vitest"; import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js"; const run = promisify(execFile); it("compares missing, excluded, mixed and redirect-filtered map declarations", async () => { const state = process.env.REPRO_STATE; if (!state) throw new Error("Set REPRO_STATE to a disposable directory"); const source = "export const observed = true;"; for (const mode of ["none", "excluded", "mixed", "redirect"] as const) { const browser = await startFakeCdpBrowser({ sourceMapBody: JSON.stringify({ version: 3, names: [], sources: ["main.js"], sourcesContent: ["export const original = true;"], mappings: "AAAA" }), sourceMapRedirect: mode === "redirect" ? "https://cdn.example.test/main.js.map" : undefined, commandResult: ({ method }) => method === "Debugger.getScriptSource" ? { scriptSource: source } : undefined, commandEvents: (command, origin) => { if (command.method !== "Debugger.enable") return undefined; const script = (id: string, sourceMapURL?: string) => ({ method: "Debugger.scriptParsed", ...(command.sessionId === undefined ? {} : { sessionId: command.sessionId }), params: { scriptId: id, url: `${origin}/${id}.js`, hash: id, length: Buffer.byteLength(source), isModule: true, executionContextId: 1, scriptLanguage: "JavaScript", ...(sourceMapURL === undefined ? {} : { sourceMapURL }) }, }); return [script("main", mode === "none" ? undefined : mode === "redirect" ? "/app.js.map" : "https://cdn.example.test/main.js.map"), ...(mode === "mixed" ? [script("second", "/app.js.map")] : [])]; }, }); try { const env = { ...process.env, HOME: `${state}/home`, XDG_CONFIG_HOME: `${state}/config`, XDG_CACHE_HOME: `${state}/cache`, XDG_DATA_HOME: `${state}/data` }; const cli = async (command: string, flags: string[] = []) => { const { stdout, stderr } = await run(process.execPath, ["scripts/rea.mjs", command, browser.endpoint, "allowed-page", "--observation-ms", "0", "--json", ...flags], { env, timeout: 30000, maxBuffer: 4 * 1024 * 1024 }); expect(stderr).toBe(""); const output = JSON.parse(stdout); return output.normalized_result ?? output; }; const inspection = await cli("inspect-web-page"); const analysis = await cli("analyze-web-bundle", ["--fetch-source-maps"]); const maps = analysis.observations.source_maps; const exclusions = inspection.completeness.excluded.filter( (entry: { section: string }) => entry.section === "source_maps"); console.log(JSON.stringify({ mode, exclusions, source_maps: { status: maps.status, requested: maps.requested, processed: maps.processed, item_statuses: maps.items.map((item: { status: string }) => item.status) }, completeness: analysis.completeness.status, unknown_dimensions: analysis.unknowns.map((item: { dimension: string }) => item.dimension) })); expect(analysis.completeness.parse_failures).toBe(0); expect(analysis.capture.scripts_analyzed).toBe(mode === "mixed" ? 2 : 1); if (mode === "excluded" || mode === "mixed") { expect(exclusions).toEqual([{ section: "source_maps", reason: "disallowed_origin", count: 1 }]); expect(analysis.completeness.status).toBe("complete"); expect(analysis.unknowns).toEqual([]); expect(maps.status).toBe(mode === "mixed" ? "included" : "not_requested"); expect(maps.requested).toBe(mode === "mixed" ? 1 : 0); } else if (mode === "redirect") { expect(maps.status).toBe("unavailable"); expect(maps.items[0].status).toBe("policy_filtered"); expect(analysis.completeness.status).toBe("partial"); expect(analysis.unknowns[0].dimension).toBe("source_maps"); } else { expect(exclusions).toEqual([]); expect(maps.status).toBe("not_requested"); expect(analysis.completeness.status).toBe("complete"); expect(analysis.unknowns).toEqual([]); } } finally { await browser.close(); } } }, 120000); ``` Remove the diagnostic test and its disposable state directory afterward. Each mode starts and closes its own simulated browser. The two CLI calls in each mode inspect and analyze equivalent repeatable captures; they do not reuse a saved inspection. ### Expected behavior When source-map fetching is selected, preserve the known policy exclusion for map declarations associated with retained scripts. Report its count/reason or a suitable unavailable/partial source-map outcome, without fetching or exposing disallowed URL details. Do not silently make a retained script with an excluded declaration indistinguishable from one with no declaration. The [source-map outcome schema](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/webBundleAnalysis.ts#L98-L178) provides `policy_filtered` map outcomes with limitations. The [existing fetcher tests](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/tests/boundary/browser/webSourceMapFetcher.test.ts#L434-L490) expect an unavailable collection with a policy-filtered item for a declared URL on another origin, without contacting it. The [observation contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/browser-observation.md#L368-L386) counts policy exclusions without retaining their details. Keep the no-declarations control complete with no invented source-map unknown. `not_requested` is valid for an empty collection; this report concerns a known excluded declaration being lost before that collection is built. ### Impact and scope P2: the captured JavaScript and successfully fetched maps remain available, but callers lose the reason that original-source/map coverage is incomplete for an admitted script. The observed scope is a map declared on another origin under default target-origin scope, alone or alongside an admitted map. Real-site frequency and other exclusion reasons were not measured. A separate `inspect-web-page` call demonstrably exposes the policy exclusion count, providing a workaround for recognizing the gap. This report does not request expanding origin authority or fetching an excluded URL. ### REA version 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node v26.12.0; locked Vitest v5.0.3. Compiled CLI with the repository's simulated CDP HTTP/WebSocket peer; no native analysis provider. ### Target and artifact context Synthetic same-origin JavaScript scripts. The excluded declaration points to `https://cdn.example.test/main.js.map`. The admitted-map control serves a small valid v3 map with one inline original source. The redirect control redirects the admitted map URL to that same excluded origin. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The executed diagnostic printed: ```json {"mode":"none","exclusions":[],"source_maps":{"status":"not_requested","requested":0,"processed":0,"item_statuses":[]},"completeness":"complete","unknown_dimensions":[]} {"mode":"excluded","exclusions":[{"section":"source_maps","reason":"disallowed_origin","count":1}],"source_maps":{"status":"not_requested","requested":0,"processed":0,"item_statuses":[]},"completeness":"complete","unknown_dimensions":[]} {"mode":"mixed","exclusions":[{"section":"source_maps","reason":"disallowed_origin","count":1}],"source_maps":{"status":"included","requested":1,"processed":1,"item_statuses":["included"]},"completeness":"complete","unknown_dimensions":[]} {"mode":"redirect","exclusions":[],"source_maps":{"status":"unavailable","requested":1,"processed":1,"item_statuses":["policy_filtered"]},"completeness":"partial","unknown_dimensions":["source_maps"]} ``` All eight CLI calls succeeded. All four analyses retained and parsed the expected scripts with zero parse failures. The focused test passed. `exclusions` above is the source-map subset of `inspect-web-page`'s ledger; the map status and completeness fields come from `analyze-web-bundle` with fetching selected. ### Typed outcome or recovery expected A policy-excluded declaration associated with a retained script should survive as explicit source-map unavailability/partial coverage or an aggregate exclusion with its reason/count. Keep allowed maps and captured source findings. An absent declaration must remain distinguishable from a policy-excluded one; keep the no-map control complete and the redirect-filtered control partial. ### Analysis or suspected cause Source-derived explanation of the executed result: [capture admission](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/browser/CdpCaptureEventHandlers.ts#L166-L208) clears both map URL fields after recording the policy exclusion. [Request construction](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/browser/CdpPageCaptureScripts.ts#L219-L235) only forwards non-null map declarations. The [provider](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/browser/CdpBrowserProvider.ts#L186-L218) selects fetching, but the [empty-list path](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/browser/WebSourceMapFetcher.ts#L80-L93) sees zero requests and returns `not_requested`. The [bundle result builder](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/webBundleAnalyzerResult.ts#L82-L131) consults capture omissions only through [a resource-budget-only filter](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/webBundleAnalyzerResult.ts#L327-L339). A `disallowed_origin` map exclusion therefore contributes neither a limitation/unknown nor partial map coverage. The mixed case loses the same exclusion while retaining the successful subset. ### Additional context The checked upstream main revision `705058f3e2df82503f1c601fc464aea7cbb626b8` has no changes to these capture, fetcher or result-projection paths relative to the tested baseline. No regression range is established. Closed issue #3217 and closed PR #3262 concern selecting source-map fetching when no map is declared; that valid empty case is preserved as a control here. Issue #829 and PR #1120 concern URL-policy vocabulary and policy-classification tests, rather than loss of the declaration outcome during bundle projection.

bsakaguchi closed this issue

2026-10-31 00:42:28 +00:00

bsakaguchi referenced this issue

2026-10-31 01:09:58 +00:00

[PR #3623] [CLOSED] fix(contracts): reject NUL bytes in rizin and cutter command inputs #5857