morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6616] [Bug] valid encoded dots in package targets are rejected #4001

New issue

Closed

opened 2026-10-31 00:45:56 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:45:56 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6616

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The public analyze-javascript-application command completes successfully but rejects valid ESM package paths containing an encoded dot inside a filename. With node_modules/pkg/x.y.js present, a legacy pkg/x%2Ey.js import is rejected with a null resolved path, while the plain pkg/x.y.js spelling resolves. An exact exports key selecting ./x%2Ey.js and an exact imports alias selecting ./local%2Efile.mjs have the same failure. Node's resolve-only controls identify the existing regular files for all three cases without executing any fixture module.

The unresolved declarations remain in the graph; their target mappings are missing. Both the syntax import record and the module-link import record carry the rejection. A relative ./local%2Efile.mjs control resolves correctly.

Reproduction

Run this block from a prepared, built REA checkout root with the supported Node executable on PATH. It creates two independent inert fixtures, isolates HOME/config/cache/tmp for child processes, resolves requests with Node without importing them, and invokes the real public CLI. The block was executed as written; both CLI calls exited 0, with zero parse failures and no integrity contradictions.

python3 - <<'PYTHON'
import json, os, pathlib, subprocess, tempfile

oracle = '''import {fileURLToPath} from "node:url";
import {relative} from "node:path";
import {statSync} from "node:fs";
const result = JSON.parse(process.argv[1]).map(specifier => {
  const target = fileURLToPath(import.meta.resolve(specifier));
  return {specifier, path: relative(process.cwd(), target), regular: statSync(target).isFile()};
});
console.log(JSON.stringify(result));
'''
cases = [
    ("legacy", {"name": "pkg", "type": "module"},
     ["pkg/x%2Ey.js", "pkg/x.y.js", "./local%2Efile.mjs"]),
    ("exports", {"name": "pkg", "type": "module", "exports": {
        "./encoded": "./x%2Ey.js", "./plain": "./x.y.js"}},
     ["pkg/encoded", "pkg/plain", "#encoded"]),
]
with tempfile.TemporaryDirectory(prefix="package-dot-") as directory:
    base = pathlib.Path(directory).resolve()
    home = base / "home"
    home.mkdir()
    env = {"PATH": os.environ["PATH"], "HOME": str(home),
           "XDG_CONFIG_HOME": str(home / "config"),
           "XDG_CACHE_HOME": str(home / "cache"), "TMPDIR": str(base)}
    for name, metadata, specifiers in cases:
        root = base / name
        package = root / "node_modules" / "pkg"
        package.mkdir(parents=True)
        (root / "package.json").write_text(json.dumps({"name": "fixture", "type": "module",
            "main": "entry.mjs", "imports": {"#encoded": "./local%2Efile.mjs"}}))
        (root / "entry.mjs").write_text("".join("import " + json.dumps(s) + ";\n" for s in specifiers))
        (root / "local.file.mjs").write_text("export const value = 1;\n")
        (package / "package.json").write_text(json.dumps(metadata))
        (package / "x.y.js").write_text("export const value = 42;\n")
        control = subprocess.run(["node", "--input-type=module", "-e", oracle, json.dumps(specifiers)],
                                 cwd=root, env=env, capture_output=True, text=True, timeout=15)
        assert control.returncode == 0, control.stderr
        print(name, "Node resolve-only:", control.stdout.strip())
        analysis = subprocess.run(["node", "scripts/rea.mjs", "analyze-javascript-application",
                                   str(root), "--format", "json"], env=env,
                                  capture_output=True, text=True, timeout=90)
        assert analysis.returncode == 0, analysis.stderr
        result = json.loads(analysis.stdout)["normalized_result"]
        assert result["statistics"]["parse_failures"] == 0
        assert result["integrity_contradictions"] == []
        rows = sorted({(p["specifier"], p["resolution_status"], p.get("resolved_path"))
            for e in result["graph"]["edges"] for p in [e["properties"]]
            if p.get("specifier") in specifiers})
        print(name, "REA exit:", analysis.returncode, "rows:", json.dumps(rows))
PYTHON

The reduced run printed these REA rows, ordered as [specifier, resolution_status, resolved_path]:

[ ["./local%2Efile.mjs", "resolved", "local.file.mjs"], ["pkg/x%2Ey.js", "rejected", null], ["pkg/x.y.js", "resolved", "node_modules/pkg/x.y.js"] ]
[ ["#encoded", "rejected", null], ["pkg/encoded", "rejected", null], ["pkg/plain", "resolved", "node_modules/pkg/x.y.js"] ]

Expected behavior

Resolve the legacy encoded package subpath and the selected exports target to node_modules/pkg/x.y.js, and the imports alias to local.file.mjs, while retaining each original declared specifier. The documented package URL contract applies one percent-decoding pass before exact inventoried-file lookup. A dot inside x.y.js is not a whole . or .. segment and does not escape the package root. Exact package-map keys remain literal lookup identities; decoding belongs to the selected target rather than to key matching.

Encoded separators, forbidden whole package-target segments, and artifact-root escapes must retain their current refusal behavior. A selected file that is absent should retain the exact-file lookup failure rather than be rejected merely for the encoded dot; a selected target's decoding or lookup failure must not select a later array entry.

Impact and scope

This is a bounded incorrect dependency-resolution result for supported local artifacts. Rejected mappings can mislead an analyst about an import's target even though the file is inventoried. The independently executed legacy subpath, exports target, and imports alias all fail; plain package targets and relative ESM URL paths pass. No downstream feature-trace result, severe workflow failure, security-boundary crossing, or real-world frequency was established. NOT_ENOUGH_INFO: prevalence in actual applications and a practical workaround preserving the original structured graph have not been established.

REA version

The source-built public CLI reports 6.7.0; tested source revision is bcd1f95e688a2ac020a9d13f9c85e0ce321cb6b6. The latest stable registry package is also 6.7.0. Its downloaded published runtime reproduced the same legacy, exports, and imports-alias failures through its public launcher using the prepared source checkout's dependency tree. That is published-runtime coverage, not a clean npm-installed package test. The relevant source differs from the release tag, so the release-runtime result was checked separately rather than inferred from the manifest version.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1 arm64; prepared locked dependencies. No package installation or build was run. Node controls resolve paths and check regular-file existence; they never import or execute the fixture modules. Analysis reads the fixtures as static artifacts.

Target and artifact context

Self-authored extracted local JavaScript application directories with an ESM entry, package metadata, and an inventoried node_modules/pkg dependency. No archive, symlink, provider engine, external service, or private target is needed for the reproduction.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Expected source: docs/javascript-artifact-reconstruction.md:745-778 documents exact imports aliases, one decoding pass for legacy ESM package subpaths and selected exports targets, forbidden whole segments, and no post-selection array fallback. The Node v24.22.1 ESM resolution specification, especially PACKAGE_RESOLVE and PACKAGE_TARGET_RESOLVE, distinguishes invalid package names and whole forbidden target segments from valid encoded filename punctuation.
Failure source: the executed public CLI fixture above; src/application/javascript/JavaScriptArtifactPathResolution.ts:1096-1134 rejects legacy subpaths before decoding, and its selected exports/imports finalizer at 1188-1234 rejects the encoded pathname before decoding. The admission at 557-574 consults src/domain/artifactPathSyntax.ts:31-34, which rejects %2e anywhere. Package-key selection and valid target admission precede that refusal at 1800-1940.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The reduced Node controls returned regular:true and the decoded paths for every requested module, including pkg/x%2Ey.js, pkg/encoded, and #encoded. Both CLI calls exited 0; stderr contained progress records, with no cleanup failure. Each encoded package-path rejection includes:

Encoded dot or separator bytes are rejected before artifact path resolution.

For the selected exports target the limitation additionally identifies the package metadata, map kind, and raw "./x%2Ey.js" target; the imports alias identifies its raw "./local%2Efile.mjs" target. A broader independently executed control fixture also established:

A selected ./x%252Ey.js target resolves to the literal x%2Ey.js file, preserving exactly one decoding pass.
An invalid ./%2e%2e/outside.js target is rejected by both Node and REA; an array containing it can select its later valid fallback.
A valid ./x%2Ey.js first array target remains rejected in REA instead of resolving its decoded file; it does not select the later fallback.
A missing plain first array target reports not-found and does not select its later existing fallback. Node resolution identifies that missing pathname, and the independent existence check reports false; this does not demonstrate successful loading.
A key containing %2E matches literally and resolves its plain target. An unexported query-bearing key stays unresolved, while an explicitly exported query-bearing key resolves.

The existing resolver source-test file passed all 291 tests. That result does not cover the failing fixture or establish restoration.

Typed outcome or recovery expected

Return resolution_status: "resolved" with the decoded inventoried path for the existing valid targets, preserving the declared specifier and package-map provenance. For an absent decoded file, retain an unresolved exact-file outcome and its reason. Keep invalid-target selection distinct from a selected target's file lookup.

Analysis or suspected cause

The observed extra refusal is consistent with applying canonical artifact-path admission to URL text before decoding. The dedicated package-target guards already accept this non-traversal filename, but the later generic admission treats its %2E as forbidden. The source trace explains the result; no repair or alternate implementation was tested.

Additional context

Closed #2960 concerns relative module specifiers and explicitly excludes package exports from its reported scope. Its maintainer closure describes restored relative ESM decoding. Independent current controls confirm both an encoded filename dot and a confined encoded parent segment work through relative imports, while these package paths fail.

Open #4269 concerns the separate historical import-reference-source graph selecting a literal escaped filename. Its public entrypoint, returned graph, and relative-file identity closure differ; changing that historical projection does not establish restoration of this package-target rejection.

Closed #1883 concerns stripping query/fragment text before exports-key lookup. Current exact-key positive and negative controls pass while decoding the selected encoded-dot target fails. The closure signals are independent even if an implementation shares URL handling.

NOT_ENOUGH_INFO: a last-known-good version and alternate-platform behavior were not established. No source edits or fix were made.

Restoration check

Rerun the unchanged reproduction and require the three failing specifiers to resolve to the same paths as Node's regular-file controls; retain the passing plain and relative controls. Keep the raw exports/imports keys exact. Preserve the once-only %252E control, rejection of forbidden whole segments and encoded separators, artifact-root confinement, valid-target array selection, and the absence of post-selection fallback when a selected file is missing. No restoration run has been performed because no fix was made.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Valid ESM package paths deterministically lose their resolved target mapping in the static application graph while their declarations and target files remain available. This is a consequential bounded result error; severe harm or a general workflow failure was not demonstrated.
Workaround status: unverified
Workaround basis: Plain and relative spellings work as controls, but changing the original artifact changes the evidence. Inspecting an inventoried file manually does not restore the structured relationship; no graph-preserving workaround was verified.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6616 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior The public `analyze-javascript-application` command completes successfully but rejects valid ESM package paths containing an encoded dot inside a filename. With `node_modules/pkg/x.y.js` present, a legacy `pkg/x%2Ey.js` import is `rejected` with a null resolved path, while the plain `pkg/x.y.js` spelling resolves. An exact exports key selecting `./x%2Ey.js` and an exact imports alias selecting `./local%2Efile.mjs` have the same failure. Node's resolve-only controls identify the existing regular files for all three cases without executing any fixture module. The unresolved declarations remain in the graph; their target mappings are missing. Both the syntax import record and the module-link import record carry the rejection. A relative `./local%2Efile.mjs` control resolves correctly. ## Reproduction Run this block from a prepared, built REA checkout root with the supported Node executable on PATH. It creates two independent inert fixtures, isolates HOME/config/cache/tmp for child processes, resolves requests with Node without importing them, and invokes the real public CLI. The block was executed as written; both CLI calls exited 0, with zero parse failures and no integrity contradictions. ```sh python3 - <<'PYTHON' import json, os, pathlib, subprocess, tempfile oracle = '''import {fileURLToPath} from "node:url"; import {relative} from "node:path"; import {statSync} from "node:fs"; const result = JSON.parse(process.argv[1]).map(specifier => { const target = fileURLToPath(import.meta.resolve(specifier)); return {specifier, path: relative(process.cwd(), target), regular: statSync(target).isFile()}; }); console.log(JSON.stringify(result)); ''' cases = [ ("legacy", {"name": "pkg", "type": "module"}, ["pkg/x%2Ey.js", "pkg/x.y.js", "./local%2Efile.mjs"]), ("exports", {"name": "pkg", "type": "module", "exports": { "./encoded": "./x%2Ey.js", "./plain": "./x.y.js"}}, ["pkg/encoded", "pkg/plain", "#encoded"]), ] with tempfile.TemporaryDirectory(prefix="package-dot-") as directory: base = pathlib.Path(directory).resolve() home = base / "home" home.mkdir() env = {"PATH": os.environ["PATH"], "HOME": str(home), "XDG_CONFIG_HOME": str(home / "config"), "XDG_CACHE_HOME": str(home / "cache"), "TMPDIR": str(base)} for name, metadata, specifiers in cases: root = base / name package = root / "node_modules" / "pkg" package.mkdir(parents=True) (root / "package.json").write_text(json.dumps({"name": "fixture", "type": "module", "main": "entry.mjs", "imports": {"#encoded": "./local%2Efile.mjs"}})) (root / "entry.mjs").write_text("".join("import " + json.dumps(s) + ";\n" for s in specifiers)) (root / "local.file.mjs").write_text("export const value = 1;\n") (package / "package.json").write_text(json.dumps(metadata)) (package / "x.y.js").write_text("export const value = 42;\n") control = subprocess.run(["node", "--input-type=module", "-e", oracle, json.dumps(specifiers)], cwd=root, env=env, capture_output=True, text=True, timeout=15) assert control.returncode == 0, control.stderr print(name, "Node resolve-only:", control.stdout.strip()) analysis = subprocess.run(["node", "scripts/rea.mjs", "analyze-javascript-application", str(root), "--format", "json"], env=env, capture_output=True, text=True, timeout=90) assert analysis.returncode == 0, analysis.stderr result = json.loads(analysis.stdout)["normalized_result"] assert result["statistics"]["parse_failures"] == 0 assert result["integrity_contradictions"] == [] rows = sorted({(p["specifier"], p["resolution_status"], p.get("resolved_path")) for e in result["graph"]["edges"] for p in [e["properties"]] if p.get("specifier") in specifiers}) print(name, "REA exit:", analysis.returncode, "rows:", json.dumps(rows)) PYTHON ``` The reduced run printed these REA rows, ordered as `[specifier, resolution_status, resolved_path]`: ```json [ ["./local%2Efile.mjs", "resolved", "local.file.mjs"], ["pkg/x%2Ey.js", "rejected", null], ["pkg/x.y.js", "resolved", "node_modules/pkg/x.y.js"] ] [ ["#encoded", "rejected", null], ["pkg/encoded", "rejected", null], ["pkg/plain", "resolved", "node_modules/pkg/x.y.js"] ] ``` ## Expected behavior Resolve the legacy encoded package subpath and the selected exports target to `node_modules/pkg/x.y.js`, and the imports alias to `local.file.mjs`, while retaining each original declared specifier. The documented package URL contract applies one percent-decoding pass before exact inventoried-file lookup. A dot inside `x.y.js` is not a whole `.` or `..` segment and does not escape the package root. Exact package-map keys remain literal lookup identities; decoding belongs to the selected target rather than to key matching. Encoded separators, forbidden whole package-target segments, and artifact-root escapes must retain their current refusal behavior. A selected file that is absent should retain the exact-file lookup failure rather than be rejected merely for the encoded dot; a selected target's decoding or lookup failure must not select a later array entry. ## Impact and scope This is a bounded incorrect dependency-resolution result for supported local artifacts. Rejected mappings can mislead an analyst about an import's target even though the file is inventoried. The independently executed legacy subpath, exports target, and imports alias all fail; plain package targets and relative ESM URL paths pass. No downstream feature-trace result, severe workflow failure, security-boundary crossing, or real-world frequency was established. NOT_ENOUGH_INFO: prevalence in actual applications and a practical workaround preserving the original structured graph have not been established. ## REA version The source-built public CLI reports `6.7.0`; tested source revision is `bcd1f95e688a2ac020a9d13f9c85e0ce321cb6b6`. The latest stable registry package is also `6.7.0`. Its downloaded published runtime reproduced the same legacy, exports, and imports-alias failures through its public launcher using the prepared source checkout's dependency tree. That is published-runtime coverage, not a clean npm-installed package test. The relevant source differs from the release tag, so the release-runtime result was checked separately rather than inferred from the manifest version. ## Runtime and platform Node.js v24.22.1; macOS 27.0.1 arm64; prepared locked dependencies. No package installation or build was run. Node controls resolve paths and check regular-file existence; they never import or execute the fixture modules. Analysis reads the fixtures as static artifacts. ## Target and artifact context Self-authored extracted local JavaScript application directories with an ESM entry, package metadata, and an inventoried `node_modules/pkg` dependency. No archive, symlink, provider engine, external service, or private target is needed for the reproduction. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence - Expected source: `docs/javascript-artifact-reconstruction.md:745-778` documents exact imports aliases, one decoding pass for legacy ESM package subpaths and selected exports targets, forbidden whole segments, and no post-selection array fallback. The [Node v24.22.1 ESM resolution specification](https://nodejs.org/download/release/v24.22.1/docs/api/esm.html#resolution-algorithm-specification), especially PACKAGE_RESOLVE and PACKAGE_TARGET_RESOLVE, distinguishes invalid package names and whole forbidden target segments from valid encoded filename punctuation. - Failure source: the executed public CLI fixture above; `src/application/javascript/JavaScriptArtifactPathResolution.ts:1096-1134` rejects legacy subpaths before decoding, and its selected exports/imports finalizer at `1188-1234` rejects the encoded pathname before decoding. The admission at `557-574` consults `src/domain/artifactPathSyntax.ts:31-34`, which rejects `%2e` anywhere. Package-key selection and valid target admission precede that refusal at `1800-1940`. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The reduced Node controls returned `regular:true` and the decoded paths for every requested module, including `pkg/x%2Ey.js`, `pkg/encoded`, and `#encoded`. Both CLI calls exited 0; stderr contained progress records, with no cleanup failure. Each encoded package-path rejection includes: ```text Encoded dot or separator bytes are rejected before artifact path resolution. ``` For the selected exports target the limitation additionally identifies the package metadata, map kind, and raw `"./x%2Ey.js"` target; the imports alias identifies its raw `"./local%2Efile.mjs"` target. A broader independently executed control fixture also established: - A selected `./x%252Ey.js` target resolves to the literal `x%2Ey.js` file, preserving exactly one decoding pass. - An invalid `./%2e%2e/outside.js` target is rejected by both Node and REA; an array containing it can select its later valid fallback. - A valid `./x%2Ey.js` first array target remains rejected in REA instead of resolving its decoded file; it does not select the later fallback. - A missing plain first array target reports `not-found` and does not select its later existing fallback. Node resolution identifies that missing pathname, and the independent existence check reports false; this does not demonstrate successful loading. - A key containing `%2E` matches literally and resolves its plain target. An unexported query-bearing key stays unresolved, while an explicitly exported query-bearing key resolves. The existing resolver source-test file passed all 291 tests. That result does not cover the failing fixture or establish restoration. ## Typed outcome or recovery expected Return `resolution_status: "resolved"` with the decoded inventoried path for the existing valid targets, preserving the declared specifier and package-map provenance. For an absent decoded file, retain an unresolved exact-file outcome and its reason. Keep invalid-target selection distinct from a selected target's file lookup. ## Analysis or suspected cause The observed extra refusal is consistent with applying canonical artifact-path admission to URL text before decoding. The dedicated package-target guards already accept this non-traversal filename, but the later generic admission treats its `%2E` as forbidden. The source trace explains the result; no repair or alternate implementation was tested. ## Additional context Closed [#2960](https://github.com/morluto/rea/issues/2960) concerns relative module specifiers and explicitly excludes package exports from its reported scope. Its maintainer closure describes restored relative ESM decoding. Independent current controls confirm both an encoded filename dot and a confined encoded parent segment work through relative imports, while these package paths fail. Open [#4269](https://github.com/morluto/rea/issues/4269) concerns the separate historical `import-reference-source` graph selecting a literal escaped filename. Its public entrypoint, returned graph, and relative-file identity closure differ; changing that historical projection does not establish restoration of this package-target rejection. Closed [#1883](https://github.com/morluto/rea/issues/1883) concerns stripping query/fragment text before exports-key lookup. Current exact-key positive and negative controls pass while decoding the selected encoded-dot target fails. The closure signals are independent even if an implementation shares URL handling. NOT_ENOUGH_INFO: a last-known-good version and alternate-platform behavior were not established. No source edits or fix were made. ## Restoration check Rerun the unchanged reproduction and require the three failing specifiers to resolve to the same paths as Node's regular-file controls; retain the passing plain and relative controls. Keep the raw exports/imports keys exact. Preserve the once-only `%252E` control, rejection of forbidden whole segments and encoded separators, artifact-root confinement, valid-target array selection, and the absence of post-selection fallback when a selected file is missing. No restoration run has been performed because no fix was made. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Valid ESM package paths deterministically lose their resolved target mapping in the static application graph while their declarations and target files remain available. This is a consequential bounded result error; severe harm or a general workflow failure was not demonstrated. - Workaround status: unverified - Workaround basis: Plain and relative spellings work as controls, but changing the original artifact changes the evidence. Inspecting an inventoried file manually does not restore the structured relationship; no graph-preserving workaround was verified.

bsakaguchi closed this issue

2026-10-31 00:45:56 +00:00

bsakaguchi commented

2026-10-31 00:45:58 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Resolved by github.com/morluto/rea@75daf0c7df. Selected package URL targets are decoded once after exact key selection; the public CLI and stdio MCP fixture checks legacy/exports/imports encoded dots, double encoding, missing targets and invalid traversal without loading dependencies. Validation: tests/acceptance/applications/javascriptModuleResolutionSemantics.test.ts passed in the recorded round2/3 execution; this review did not rerun tests. Coverage is limited to the cited control; no broader real-provider, Windows or performance claim.

@morluto commented on GitHub (Oct 23, 2026): Resolved by https://github.com/morluto/rea/commit/75daf0c7df108f04ab41f34aec1a31d4a7ceb322. Selected package URL targets are decoded once after exact key selection; the public CLI and stdio MCP fixture checks legacy/exports/imports encoded dots, double encoding, missing targets and invalid traversal without loading dependencies. Validation: `tests/acceptance/applications/javascriptModuleResolutionSemantics.test.ts` passed in the recorded round2/3 execution; this review did not rerun tests. Coverage is limited to the cited control; no broader real-provider, Windows or performance claim.

bsakaguchi referenced this issue

2026-10-31 00:46:08 +00:00

[GH-ISSUE #6639] [Bug] WASM glue drops valid URL literals with line separators in queries or fragments #4021

bsakaguchi referenced this issue

2026-10-31 01:11:46 +00:00

[PR #4149] [CLOSED] fix(wasm): retain glue literals parsed as JavaScript directives #6089