morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5878] [Bug] URL-prefix runtime mapping turns encoded slash into a false static match #3563

New issue

Closed

opened 2026-10-31 00:41:54 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:41:54 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5878

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

reconcile-javascript-runtime reports an HTTP script URL ending in child%2Fpart.js as a medium-confidence match to the static nested file child/part.js when supplied an ordinary url-prefix mapping. It emits an inferred observed_as edge even though the encoded slash is inside one URL path segment, rather than a path boundary.

This was executed through the built CLI with distinct static files child/part.js and child%2Fpart.js, and synthetic browser inspection Evidence without captured source bytes. The ordinary child/part.js URL matches the nested file too. The correctly encoded child%252Fpart.js control instead matches the literal-percent filename child%2Fpart.js.

Reproduction

Use the source revision below with its locked dependencies and compiled runtime. Save this focused test as tests/boundary/filesystem/javascript-applications/urlPrefixMappingReproduction.test.ts and run it with the repository's pinned Node version:

node node_modules/vitest/vitest.mjs run tests/boundary/filesystem/javascript-applications/urlPrefixMappingReproduction.test.ts --maxWorkers=1

The following test was executed successfully. Its assertions characterize the current incorrect behavior so all three observed cases remain visible; they are not the desired regression expectations. It uses the existing fixture cleanup helper. The static analyzer reads inert source files; synthetic observation Evidence uses production constructors and passes the reconciliation service's schema and semantic validation. The test then invokes the built public CLI for each input.

import { mkdir, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { expect, it } from 'vitest';
import { createTestTempDirectory } from '../../../fixtures/temporaryDirectory.js';
import { analyzeFixture } from '../../../fixtures/javascriptRuntimeReconciliation.js';
import { passiveNetworkCapture } from '../../../fixtures/passiveNetworkCapture.js';
import { createBrowserEvidence } from '../../../../src/application/BrowserEvidence.js';
import { inspectWebPageInputSchema } from '../../../../src/domain/browserObservation.js';
import { reconcileJavaScriptRuntimeEvidence } from '../../../../src/application/javascript/JavaScriptRuntimeReconciliationService.js';

it('shows encoded slash aliasing a distinct static nested file through the public CLI', async () => {
  const root = await createTestTempDirectory('rea-url-prefix-proof-');
  await mkdir(join(root, 'child'));
  await writeFile(join(root, 'child', 'part.js'), "export const marker = 'nested';\n");
  await writeFile(join(root, 'child%2Fpart.js'), "export const marker = 'literal';\n");
  const analysis = await analyzeFixture(root);
  for (const suffix of ['child/part.js', 'child%2Fpart.js', 'child%252Fpart.js']) {
    const inspection = passiveNetworkCapture([]);
    const origin = inspection.target.origin;
    inspection.frames = [{ frame_id: 'frame-main', parent_frame_id: null, url: origin, origin }];
    inspection.scripts = { total: 1, items: [{
      script_key: `scr_${'1'.repeat(64)}`, frame_id: 'frame-main',
      url: `${origin}/assets/${suffix}`, origin,
      cdp_hash: null, length: null, is_module: true, language: 'JavaScript',
      source_map_url: null,
      resource_reconciliation: { status: 'unmatched', reason: 'no_exact_sanitized_url' },
      source: { included: false, reason: 'not-requested' },
    }] };
    const runtime = createBrowserEvidence('inspect_web_page', inspectWebPageInputSchema.parse({
      cdp_endpoint: 'http://127.0.0.1:9222', target_id: 'page',
      allowed_origins: [origin], observation_ms: 0, include_script_sources: false,
    }), inspection, {
      id: 'rea-cdp-browser', name: 'REA Chrome DevTools Protocol observation provider', version: '2',
    });
    const input = {
      static_layers: [{ role: 'application', analysis, runtime_mappings: [{
        kind: 'url-prefix', prefix: `${origin}/assets/`, artifact_prefix: '',
      }] }], runtime_observations: [runtime],
    };
    const service = reconcileJavaScriptRuntimeEvidence(input);
    expect(service.ok).toBe(true);
    const inputPath = join(root, 'reconciliation.json');
    await writeFile(inputPath, JSON.stringify(input));
    const executed = await promisify(execFile)(process.execPath,
      ['scripts/rea.mjs', 'reconcile-javascript-runtime', inputPath, '--json'],
      { cwd: process.cwd(), timeout: 20000, maxBuffer: 4000000 });
    expect(executed.stderr).toBe('');
    const result = JSON.parse(executed.stdout).normalized_result;
    const item = result.reconciliations.find((value: any) => value.entity_kind === 'script');
    const node = result.graph.nodes.find((value: any) => value.node_id === item.static_node_id);
    const paths = node.observations.map((value: any) => value.properties.path).filter(Boolean);
    const edges = result.graph.edges.filter((value: any) => value.relation === 'observed_as');
    console.log(JSON.stringify({ suffix, status: item.status, basis: item.basis,
      confidence: item.confidence, reason: item.reason, paths, observed_as: edges.length }));
    expect(item).toMatchObject({ status: 'matched', basis: 'operator-url-mapping', confidence: 'medium' });
    expect(paths).toContain(suffix === 'child%252Fpart.js' ? 'child%2Fpart.js' : 'child/part.js');
    expect(edges).toHaveLength(1);
  }
});

No browser or Electron application was launched. The loopback endpoint is metadata in the synthetic observation and is never contacted. The supplied build and dependency installation were used without rebuilding or reinstalling.

Expected behavior

A prefix-only mapping must preserve the distinction between an encoded slash inside a URL segment and an actual artifact hierarchy separator. child%2Fpart.js must not produce a location-only match or observed_as edge to child/part.js. With no captured bytes or other supported correspondence, retain uncertainty using the existing result vocabulary. Preserve the two ordinary and literal-percent controls.

The mapping contract defines translation of an exact HTTP(S) origin and path prefix, and the matching rules require conservative correspondence. Related static URL handling explicitly rejects encoded path separators, and an existing CLI acceptance case leaves ./assets%2Fapp.js?valid=1 unresolved. Those static rules support preserving segment identity; they do not require applying every static admission rule unchanged to runtime mappings.

Impact and scope

P2: this bounded input produces a false inferred runtime/static correspondence and can mislead interpretation of which shipped asset was observed. The executed scope is a source-free browser-script observation through the CLI on one platform. Real-world frequency is unknown. The original runtime URL and static paths remain available for manual cross-checking; an automated workaround preserving the requested structured correspondence was not established.

No authorization boundary crossing, target execution, file mutation, or data loss was observed. Reconciliation derives results from supplied Evidence without contacting the browser. Worker/target variants, captured-source mismatch variants, MCP, and other platforms were not exercised.

REA version

6.7.0, source revision 52415975942a4b9c53a817915b313ccc404f280e, using the supplied compiled runtime. Fetched main at f5410bdc4c86de564ddb24f24df86c41cfdd8bec has no changes to the relevant mapping, matching, input schema, capture validation, service, CLI adapter, or reconciliation guide files since that revision.

Runtime and platform

macOS 27.0.1 arm64; Node.js v24.18.0; Vitest 5.0.3. No live browser, Electron runtime, or native analysis provider was used.

Target and artifact context

A local directory containing two distinct inert JavaScript files: child/part.js and child%2Fpart.js. One application layer is reconciled with a frame-attributed synthetic browser script using a caller-declared HTTP(S) URL-prefix mapping and no captured script source.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

One focused test passed. All three reconciliation CLI subprocesses exited successfully with empty stderr and returned these selected fields:

{"suffix":"child/part.js","status":"matched","basis":"operator-url-mapping","confidence":"medium","reason":"unique-location-match","paths":["child/part.js"],"observed_as":1}
{"suffix":"child%2Fpart.js","status":"matched","basis":"operator-url-mapping","confidence":"medium","reason":"unique-location-match","paths":["child/part.js"],"observed_as":1}
{"suffix":"child%252Fpart.js","status":"matched","basis":"operator-url-mapping","confidence":"medium","reason":"unique-location-match","paths":["child%2Fpart.js"],"observed_as":1}

The middle row is the failure. The final row demonstrates that the correctly encoded URL for a literal-percent filename already works; the defect is specifically the additional false correspondence created by decoding %2F into a hierarchy separator.

Typed outcome or recovery expected

Do not return a location-only matched item or an observed_as edge from child%2Fpart.js to child/part.js under this prefix-only mapping. Retain the original runtime URL and uncertainty. Do not introduce a new result status merely for this case. No patch or restoration was tested.

Analysis or suspected cause

Source-derived explanation: pathBelowUrlPrefix calls decodeURIComponent on the entire pathname suffix. This turns %2F into / before the suffix reaches logical artifact-path interpretation. mappedRuntimePaths supplies the resulting path to matching; the unique-location branch then creates the medium-confidence match. The CLI adapter reaches the same production reconciliation service used by the focused input-validation check.

A fix needs to preserve URL segment identity before a decoded byte can become an artifact separator. A blanket rejection of encoded backslashes would change a separate existing contract: the runtime URL-prefix test deliberately maps %5C to a literal POSIX backslash filename. This report concerns %2F only.

Additional context

Related #5872 is a different defect: Electron file-target admission rejects an otherwise valid local file URL because its query or fragment contains an encoded separator. This report accepts HTTP(S) runtime Evidence and creates a false static correspondence by decoding an encoded separator in the pathname. The owning functions, failure stages, results, and required corrections differ.

Closed #3316 concerns captured-digest mismatch precedence, and #3192 concerns absence states for ambiguous candidates; neither addresses this pathname translation. #4269 and PR #4323 concern historical ESM dependency resolution. Merged PR #4941 preserves native runtime path bytes and application ownership but leaves this URL-prefix decoder unchanged. Static HTML suffix reports #2827/PR #2833 and encoded-separator handling in PRs #1287/#1295 affect a separate resolver. A last known good version and installed-package behavior were not established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5878 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `reconcile-javascript-runtime` reports an HTTP script URL ending in `child%2Fpart.js` as a medium-confidence match to the static nested file `child/part.js` when supplied an ordinary `url-prefix` mapping. It emits an inferred `observed_as` edge even though the encoded slash is inside one URL path segment, rather than a path boundary. This was executed through the built CLI with distinct static files `child/part.js` and `child%2Fpart.js`, and synthetic browser inspection Evidence without captured source bytes. The ordinary `child/part.js` URL matches the nested file too. The correctly encoded `child%252Fpart.js` control instead matches the literal-percent filename `child%2Fpart.js`. ### Reproduction Use the source revision below with its locked dependencies and compiled runtime. Save this focused test as `tests/boundary/filesystem/javascript-applications/urlPrefixMappingReproduction.test.ts` and run it with the repository's pinned Node version: ```sh node node_modules/vitest/vitest.mjs run tests/boundary/filesystem/javascript-applications/urlPrefixMappingReproduction.test.ts --maxWorkers=1 ``` The following test was executed successfully. Its assertions characterize the current incorrect behavior so all three observed cases remain visible; they are not the desired regression expectations. It uses the existing fixture cleanup helper. The static analyzer reads inert source files; synthetic observation Evidence uses production constructors and passes the reconciliation service's schema and semantic validation. The test then invokes the built public CLI for each input. ```ts import { mkdir, writeFile } from 'node:fs/promises'; import { join } from 'node:path'; import { execFile } from 'node:child_process'; import { promisify } from 'node:util'; import { expect, it } from 'vitest'; import { createTestTempDirectory } from '../../../fixtures/temporaryDirectory.js'; import { analyzeFixture } from '../../../fixtures/javascriptRuntimeReconciliation.js'; import { passiveNetworkCapture } from '../../../fixtures/passiveNetworkCapture.js'; import { createBrowserEvidence } from '../../../../src/application/BrowserEvidence.js'; import { inspectWebPageInputSchema } from '../../../../src/domain/browserObservation.js'; import { reconcileJavaScriptRuntimeEvidence } from '../../../../src/application/javascript/JavaScriptRuntimeReconciliationService.js'; it('shows encoded slash aliasing a distinct static nested file through the public CLI', async () => { const root = await createTestTempDirectory('rea-url-prefix-proof-'); await mkdir(join(root, 'child')); await writeFile(join(root, 'child', 'part.js'), "export const marker = 'nested';\n"); await writeFile(join(root, 'child%2Fpart.js'), "export const marker = 'literal';\n"); const analysis = await analyzeFixture(root); for (const suffix of ['child/part.js', 'child%2Fpart.js', 'child%252Fpart.js']) { const inspection = passiveNetworkCapture([]); const origin = inspection.target.origin; inspection.frames = [{ frame_id: 'frame-main', parent_frame_id: null, url: origin, origin }]; inspection.scripts = { total: 1, items: [{ script_key: `scr_${'1'.repeat(64)}`, frame_id: 'frame-main', url: `${origin}/assets/${suffix}`, origin, cdp_hash: null, length: null, is_module: true, language: 'JavaScript', source_map_url: null, resource_reconciliation: { status: 'unmatched', reason: 'no_exact_sanitized_url' }, source: { included: false, reason: 'not-requested' }, }] }; const runtime = createBrowserEvidence('inspect_web_page', inspectWebPageInputSchema.parse({ cdp_endpoint: 'http://127.0.0.1:9222', target_id: 'page', allowed_origins: [origin], observation_ms: 0, include_script_sources: false, }), inspection, { id: 'rea-cdp-browser', name: 'REA Chrome DevTools Protocol observation provider', version: '2', }); const input = { static_layers: [{ role: 'application', analysis, runtime_mappings: [{ kind: 'url-prefix', prefix: `${origin}/assets/`, artifact_prefix: '', }] }], runtime_observations: [runtime], }; const service = reconcileJavaScriptRuntimeEvidence(input); expect(service.ok).toBe(true); const inputPath = join(root, 'reconciliation.json'); await writeFile(inputPath, JSON.stringify(input)); const executed = await promisify(execFile)(process.execPath, ['scripts/rea.mjs', 'reconcile-javascript-runtime', inputPath, '--json'], { cwd: process.cwd(), timeout: 20000, maxBuffer: 4000000 }); expect(executed.stderr).toBe(''); const result = JSON.parse(executed.stdout).normalized_result; const item = result.reconciliations.find((value: any) => value.entity_kind === 'script'); const node = result.graph.nodes.find((value: any) => value.node_id === item.static_node_id); const paths = node.observations.map((value: any) => value.properties.path).filter(Boolean); const edges = result.graph.edges.filter((value: any) => value.relation === 'observed_as'); console.log(JSON.stringify({ suffix, status: item.status, basis: item.basis, confidence: item.confidence, reason: item.reason, paths, observed_as: edges.length })); expect(item).toMatchObject({ status: 'matched', basis: 'operator-url-mapping', confidence: 'medium' }); expect(paths).toContain(suffix === 'child%252Fpart.js' ? 'child%2Fpart.js' : 'child/part.js'); expect(edges).toHaveLength(1); } }); ``` No browser or Electron application was launched. The loopback endpoint is metadata in the synthetic observation and is never contacted. The supplied build and dependency installation were used without rebuilding or reinstalling. ### Expected behavior A prefix-only mapping must preserve the distinction between an encoded slash inside a URL segment and an actual artifact hierarchy separator. `child%2Fpart.js` must not produce a location-only match or `observed_as` edge to `child/part.js`. With no captured bytes or other supported correspondence, retain uncertainty using the existing result vocabulary. Preserve the two ordinary and literal-percent controls. The [mapping contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/javascript-runtime-reconciliation.md#L57-L63) defines translation of an exact HTTP(S) origin and path prefix, and the [matching rules](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/javascript-runtime-reconciliation.md#L87-L109) require conservative correspondence. Related static URL handling explicitly [rejects encoded path separators](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/javascript-artifact-reconstruction.md#L727-L735), and an existing [CLI acceptance case](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/tests/acceptance/applications/javascriptHtmlUrlSuffixesCli.test.ts#L55-L59) leaves `./assets%2Fapp.js?valid=1` unresolved. Those static rules support preserving segment identity; they do not require applying every static admission rule unchanged to runtime mappings. ### Impact and scope P2: this bounded input produces a false inferred runtime/static correspondence and can mislead interpretation of which shipped asset was observed. The executed scope is a source-free browser-script observation through the CLI on one platform. Real-world frequency is unknown. The original runtime URL and static paths remain available for manual cross-checking; an automated workaround preserving the requested structured correspondence was not established. No authorization boundary crossing, target execution, file mutation, or data loss was observed. Reconciliation derives results from supplied Evidence without contacting the browser. Worker/target variants, captured-source mismatch variants, MCP, and other platforms were not exercised. ### REA version 6.7.0, source revision `52415975942a4b9c53a817915b313ccc404f280e`, using the supplied compiled runtime. Fetched main at `f5410bdc4c86de564ddb24f24df86c41cfdd8bec` has no changes to the relevant mapping, matching, input schema, capture validation, service, CLI adapter, or reconciliation guide files since that revision. ### Runtime and platform macOS 27.0.1 arm64; Node.js v24.18.0; Vitest 5.0.3. No live browser, Electron runtime, or native analysis provider was used. ### Target and artifact context A local directory containing two distinct inert JavaScript files: `child/part.js` and `child%2Fpart.js`. One application layer is reconciled with a frame-attributed synthetic browser script using a caller-declared HTTP(S) URL-prefix mapping and no captured script source. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence One focused test passed. All three reconciliation CLI subprocesses exited successfully with empty stderr and returned these selected fields: ```json {"suffix":"child/part.js","status":"matched","basis":"operator-url-mapping","confidence":"medium","reason":"unique-location-match","paths":["child/part.js"],"observed_as":1} {"suffix":"child%2Fpart.js","status":"matched","basis":"operator-url-mapping","confidence":"medium","reason":"unique-location-match","paths":["child/part.js"],"observed_as":1} {"suffix":"child%252Fpart.js","status":"matched","basis":"operator-url-mapping","confidence":"medium","reason":"unique-location-match","paths":["child%2Fpart.js"],"observed_as":1} ``` The middle row is the failure. The final row demonstrates that the correctly encoded URL for a literal-percent filename already works; the defect is specifically the additional false correspondence created by decoding `%2F` into a hierarchy separator. ### Typed outcome or recovery expected Do not return a location-only `matched` item or an `observed_as` edge from `child%2Fpart.js` to `child/part.js` under this prefix-only mapping. Retain the original runtime URL and uncertainty. Do not introduce a new result status merely for this case. No patch or restoration was tested. ### Analysis or suspected cause Source-derived explanation: [pathBelowUrlPrefix](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/javascript/javascriptRuntimeStaticCandidates.ts#L394-L418) calls `decodeURIComponent` on the entire pathname suffix. This turns `%2F` into `/` before the suffix reaches logical artifact-path interpretation. [mappedRuntimePaths](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/javascript/javascriptRuntimeStaticCandidates.ts#L61-L107) supplies the resulting path to matching; [the unique-location branch](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/javascript/javascriptRuntimeReconciliationMatching.ts#L83-L99) then creates the medium-confidence match. The [CLI adapter](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/cli/electronCommands.ts#L167-L196) reaches the same production reconciliation service used by the focused input-validation check. A fix needs to preserve URL segment identity before a decoded byte can become an artifact separator. A blanket rejection of encoded backslashes would change a separate existing contract: the [runtime URL-prefix test](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/tests/boundary/filesystem/javascript-applications/javascriptRuntimeReconciliation.test.ts#L498-L552) deliberately maps `%5C` to a literal POSIX backslash filename. This report concerns `%2F` only. ### Additional context Related #5872 is a different defect: Electron file-target admission rejects an otherwise valid local file URL because its query or fragment contains an encoded separator. This report accepts HTTP(S) runtime Evidence and creates a false static correspondence by decoding an encoded separator in the pathname. The owning functions, failure stages, results, and required corrections differ. Closed #3316 concerns captured-digest mismatch precedence, and #3192 concerns absence states for ambiguous candidates; neither addresses this pathname translation. #4269 and PR #4323 concern historical ESM dependency resolution. Merged PR #4941 preserves native runtime path bytes and application ownership but leaves this URL-prefix decoder unchanged. Static HTML suffix reports #2827/PR #2833 and encoded-separator handling in PRs #1287/#1295 affect a separate resolver. A last known good version and installed-package behavior were not established.

bsakaguchi closed this issue

2026-10-31 00:41:54 +00:00

bsakaguchi referenced this issue

2026-10-31 01:09:44 +00:00

[PR #3563] [CLOSED] fix(comparison): keep unexamined Swift literals unknown #5828