morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3916] [Bug] query-bearing ESM imports miss inventoried modules #1526

New issue

Closed

opened 2026-10-16 21:06:40 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-16 21:06:40 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3916

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

import-reference-source treats the query and fragment in a relative ESM request as filesystem-name bytes. It leaves the dependency unresolved despite an inventoried dep.mjs. Node resolves the same request to that pathname and retains the suffix as URL components. This is a static REA finding; REA was not executed.

The statically derived relationship is {"from_path":"main.mjs","to":"dep.mjs?revision=1#first","kind":"imports","resolution":"unresolved","parse_state":"parsed"}. This is not captured REA output. Static tracing also retains hashes for both files. The independently executed Node control established that the selected pathname is a regular file.

Reproduction

These CLI steps are proposed and unexecuted. Use the source baseline under Runtime or environment, normal valid REA configuration, and an otherwise empty, stable, readable directory on macOS or Linux. Files must be regular UTF-8 files, without symlinks, .gitignore, caller exclusions, or configured reference secret patterns. No enclosing Git repository or provider is required.

mkdir source
printf 'import "./dep.mjs?revision=1#first";\n' > source/main.mjs
printf 'export const value = 1;\n' > source/dep.mjs
REA_REFERENCE_SECRET_PATTERNS_JSON='[]' rea import-reference-source "$PWD/source" --json > graph.json

Inspect the main.mjs import relationship. During the initial investigation, a disposable fixture with these exact bytes and one independent resolve-only Node control ran. A separate control.mjs in the source directory's parent called import.meta.resolve('./source/dep.mjs?revision=1#first'), converted the URL using fileURLToPath, and checked statSync(...).isFile(). Its retained result was source-relative pathname dep.mjs, search ?revision=1, hash #first, and regular-file state true. Node never imported or executed the fixture modules. No experimental minimization or REA reproduction ran; CLI output and exit status are unavailable.

Expected behavior

The parsed import in main.mjs should target inventoried dep.mjs with kind: "imports" and resolution: "internal". Explicit .mjs mode establishes ESM applicability. The graph's file identity should represent the existing file selected by that request; URL query and fragment components must not become pathname bytes.

Impact and scope

Valid query-bearing local ESM requests produce incomplete historical dependency relationships while source inventory remains available. Broader frequency is unknown. The Node control aids manual interpretation, but no practical workaround restoring the saved graph without changing exact historical source bytes was verified.

REA version

Source baseline: rea-agents-6.5.0, revision 3b79af28e80785afce9f5998d8f03289402a1847. The initial investigation inspected the relevant main diff at cf0a38ad9774ee9acff3164052688f7a58a1c72c. Node controls used Node.js v26.10.0, Darwin arm64. REA runtime and last known good version are unknown. Linux applicability rests on source premises, not an executed control there.

Runtime and platform

REA runtime reproduction was not run. Runtime and platform details for any smaller premise checks are recorded in the version/environment field above. Real-provider versions and end-to-end output are NOT_ENOUGH_INFO because no provider was launched.

Target and artifact context

Local historical ESM source tree containing main.mjs and dep.mjs; query-bearing relative import; reference-source dependency graph; no provider requirement.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Expected source: the CLI contract exposes historical relationships. src/domain/referenceSourceImportParsing.ts:129-142,315 explicitly parses ESM imports; tests/boundary/filesystem/referenceSourceDynamicImports.test.ts:38-49 expects internal edges to present local targets. The Node control establishes this request's loader meaning.
Failure source: the reference resolver joins and POSIX-normalizes the complete specifier, then probes it and appended extensions/indexes. src/domain/referenceSourcePathIdentity.ts:19-32 finds none of those candidate strings among the two real paths.
Evidence provenance: static
Local verification: static-proof
Reproduction completeness: complete

REA runtime reproduction was not run. The primary violation evidence is static; the initial investigator separately observed the Node resolve-only results above. Public reachability and admission premises follow.

The shared public source path begins at CLI registration and projection. It validates the directory-string argument, defaults to full detail, parses configuration, calls runReferenceSourceImport, passes the root and policy into importReferenceSource, and returns the graph. src/config/environment.ts:106-121 and src/config/parseConfig.ts:94-96 accept the shown empty secret policy.

src/application/ReferenceSourceImportPolicy.ts:49-166 validates the directory and creates matchers. Defaults at src/application/ReferenceSourceImportTypes.ts:79-89 exclude neither file. src/application/ReferenceSourceImport.ts:201-220 reads the tree and VCS metadata; ReferenceSourceVcsAdapter.ts:37-49 permits kind: none. src/reference/ReferenceSourceReader.ts:46-51 and src/reference/ReferenceSourceReaderValidate.ts:31-32 require available no-follow opens, limiting the proposed trigger to the supported hosts above. Stable regular files pass src/reference/ReferenceSourceReaderEntries.ts:192-277, and unchanged descriptor contents become read entries at src/reference/ReferenceSourceReaderFile.ts:183-197,230-275.

ReferenceSourceImport.ts:239-247 creates the lookup from successfully read regular-file paths. src/domain/referenceSourceClassification.ts:46-50 classifies .mjs as JavaScript; src/domain/referenceSourceImportParsing.ts:64-65 admits it. src/domain/javascript/javascriptSourceParser.ts:40-51,65-75 selects module parsing. The plain syntax introduces no recoverable parsing issue. referenceSourceImportParsing.ts:129-142 takes the declaration string unchanged, lines 67-69 retain it, and lines 72-80 classify ./... as internal.

appendFile at ReferenceSourceImportEntries.ts:133-170 hashes files and resolves internal relationships. Normalization yields dep.mjs?revision=1#first; no file/index candidate equals dep.mjs. TypeScriptEmittedSourceCandidates.ts:22-24 supplies no substitutions for .mjs. ReferenceSourceImport.ts:249-292 sorts, deduplicates, and validates the graph. src/domain/referenceSourceGraph.ts:119-125,297-323 permits the nonempty unresolved target; ReferenceSourceImport.ts:96-124 produces a partial inventory rather than rejection. The reader records its pathname-race limitation, so this report does not assert complete inventory coverage. No intervening guard repairs the target before projection.

Typed outcome or recovery expected

Proposed, unexecuted checks: the exact fixture must yield an internal edge to dep.mjs and retain both original hashes. Ordinary import "./dep.mjs" must retain the same target. On a POSIX filesystem, a CommonJS request to a regular file with literal ? or # in its filename must retain filesystem-name semantics. This prevents unconditional suffix stripping and restores a contract independent of extensionless JSON fallback.

Additional context

The main source inspection found that ReferenceSourceImportEntries.ts:182-218 still POSIX-joins the complete request. Its package-main/index changes cannot repair this flat explicit-extension case. File/directory precedence in #3895/#3381 and package-main selection in #3896/#3383 have different triggers. Related closed application-analyzer reports #1883, #2021, #3214, and #3307 concern analyze-javascript-application, other request shapes, or other URL rules.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Complete static reachability establishes a valid existing ESM dependency reported unresolved in optional historical analysis; inventoried source facts remain available, and no severe or universal workflow failure is established.
Workaround status: unverified
Workaround basis: The observed resolver control identifies the pathname, but no method restoring the saved graph without altering historical source bytes was tested.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3916 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior `import-reference-source` treats the query and fragment in a relative ESM request as filesystem-name bytes. It leaves the dependency unresolved despite an inventoried `dep.mjs`. Node resolves the same request to that pathname and retains the suffix as URL components. This is a static REA finding; REA was not executed. The statically derived relationship is `{"from_path":"main.mjs","to":"dep.mjs?revision=1#first","kind":"imports","resolution":"unresolved","parse_state":"parsed"}`. This is not captured REA output. Static tracing also retains hashes for both files. The independently executed Node control established that the selected pathname is a regular file. ## Reproduction These CLI steps are proposed and unexecuted. Use the source baseline under Runtime or environment, normal valid REA configuration, and an otherwise empty, stable, readable directory on macOS or Linux. Files must be regular UTF-8 files, without symlinks, `.gitignore`, caller exclusions, or configured reference secret patterns. No enclosing Git repository or provider is required. ```sh mkdir source printf 'import "./dep.mjs?revision=1#first";\n' > source/main.mjs printf 'export const value = 1;\n' > source/dep.mjs REA_REFERENCE_SECRET_PATTERNS_JSON='[]' rea import-reference-source "$PWD/source" --json > graph.json ``` Inspect the `main.mjs` import relationship. During the initial investigation, a disposable fixture with these exact bytes and one independent resolve-only Node control ran. A separate `control.mjs` in the source directory's parent called `import.meta.resolve('./source/dep.mjs?revision=1#first')`, converted the URL using `fileURLToPath`, and checked `statSync(...).isFile()`. Its retained result was source-relative pathname `dep.mjs`, search `?revision=1`, hash `#first`, and regular-file state `true`. Node never imported or executed the fixture modules. No experimental minimization or REA reproduction ran; CLI output and exit status are unavailable. ## Expected behavior The parsed import in `main.mjs` should target inventoried `dep.mjs` with `kind: "imports"` and `resolution: "internal"`. Explicit `.mjs` mode establishes ESM applicability. The graph's file identity should represent the existing file selected by that request; URL query and fragment components must not become pathname bytes. ## Impact and scope Valid query-bearing local ESM requests produce incomplete historical dependency relationships while source inventory remains available. Broader frequency is unknown. The Node control aids manual interpretation, but no practical workaround restoring the saved graph without changing exact historical source bytes was verified. ## REA version Source baseline: `rea-agents-6.5.0`, revision `3b79af28e80785afce9f5998d8f03289402a1847`. The initial investigation inspected the relevant main diff at `cf0a38ad9774ee9acff3164052688f7a58a1c72c`. Node controls used Node.js `v26.10.0`, Darwin arm64. REA runtime and last known good version are unknown. Linux applicability rests on source premises, not an executed control there. ## Runtime and platform REA runtime reproduction was not run. Runtime and platform details for any smaller premise checks are recorded in the version/environment field above. Real-provider versions and end-to-end output are NOT_ENOUGH_INFO because no provider was launched. ## Target and artifact context Local historical ESM source tree containing `main.mjs` and `dep.mjs`; query-bearing relative import; reference-source dependency graph; no provider requirement. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence - Expected source: the [CLI contract](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/docs/cli.md#L227-L257) exposes historical relationships. `src/domain/referenceSourceImportParsing.ts:129-142,315` explicitly parses ESM imports; `tests/boundary/filesystem/referenceSourceDynamicImports.test.ts:38-49` expects internal edges to present local targets. The Node control establishes this request's loader meaning. - Failure source: the [reference resolver](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/src/application/ReferenceSourceImportEntries.ts#L97-L170) joins and POSIX-normalizes the complete specifier, then probes it and appended extensions/indexes. `src/domain/referenceSourcePathIdentity.ts:19-32` finds none of those candidate strings among the two real paths. - Evidence provenance: static - Local verification: static-proof - Reproduction completeness: complete REA runtime reproduction was not run. The primary violation evidence is static; the initial investigator separately observed the Node resolve-only results above. Public reachability and admission premises follow. The shared public source path begins at [CLI registration and projection](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/src/cli/utilityCommands.ts#L233-L395). It validates the directory-string argument, defaults to full detail, parses configuration, calls `runReferenceSourceImport`, passes the root and policy into `importReferenceSource`, and returns the graph. `src/config/environment.ts:106-121` and `src/config/parseConfig.ts:94-96` accept the shown empty secret policy. `src/application/ReferenceSourceImportPolicy.ts:49-166` validates the directory and creates matchers. Defaults at `src/application/ReferenceSourceImportTypes.ts:79-89` exclude neither file. `src/application/ReferenceSourceImport.ts:201-220` reads the tree and VCS metadata; `ReferenceSourceVcsAdapter.ts:37-49` permits `kind: none`. `src/reference/ReferenceSourceReader.ts:46-51` and `src/reference/ReferenceSourceReaderValidate.ts:31-32` require available no-follow opens, limiting the proposed trigger to the supported hosts above. Stable regular files pass `src/reference/ReferenceSourceReaderEntries.ts:192-277`, and unchanged descriptor contents become read entries at `src/reference/ReferenceSourceReaderFile.ts:183-197,230-275`. `ReferenceSourceImport.ts:239-247` creates the lookup from successfully read regular-file paths. `src/domain/referenceSourceClassification.ts:46-50` classifies `.mjs` as JavaScript; `src/domain/referenceSourceImportParsing.ts:64-65` admits it. `src/domain/javascript/javascriptSourceParser.ts:40-51,65-75` selects module parsing. The plain syntax introduces no recoverable parsing issue. `referenceSourceImportParsing.ts:129-142` takes the declaration string unchanged, lines 67-69 retain it, and lines 72-80 classify `./...` as internal. `appendFile` at `ReferenceSourceImportEntries.ts:133-170` hashes files and resolves internal relationships. Normalization yields `dep.mjs?revision=1#first`; no file/index candidate equals `dep.mjs`. `TypeScriptEmittedSourceCandidates.ts:22-24` supplies no substitutions for `.mjs`. `ReferenceSourceImport.ts:249-292` sorts, deduplicates, and validates the graph. `src/domain/referenceSourceGraph.ts:119-125,297-323` permits the nonempty unresolved target; `ReferenceSourceImport.ts:96-124` produces a partial inventory rather than rejection. The reader records its pathname-race limitation, so this report does not assert complete inventory coverage. No intervening guard repairs the target before projection. ## Typed outcome or recovery expected Proposed, unexecuted checks: the exact fixture must yield an internal edge to `dep.mjs` and retain both original hashes. Ordinary `import "./dep.mjs"` must retain the same target. On a POSIX filesystem, a CommonJS request to a regular file with literal `?` or `#` in its filename must retain filesystem-name semantics. This prevents unconditional suffix stripping and restores a contract independent of extensionless JSON fallback. ## Additional context The main source inspection found that `ReferenceSourceImportEntries.ts:182-218` still POSIX-joins the complete request. Its package-main/index changes cannot repair this flat explicit-extension case. File/directory precedence in #3895/#3381 and package-main selection in #3896/#3383 have different triggers. Related closed application-analyzer reports #1883, #2021, #3214, and #3307 concern `analyze-javascript-application`, other request shapes, or other URL rules. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Complete static reachability establishes a valid existing ESM dependency reported unresolved in optional historical analysis; inventoried source facts remain available, and no severe or universal workflow failure is established. - Workaround status: unverified - Workaround basis: The observed resolver control identifies the pathname, but no method restoring the saved graph without altering historical source bytes was tested.

bsakaguchi commented

2026-10-16 21:06:42 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Confirmed: Node resolves the request to the pathname and keeps ?query#fragment as URL components, so treating them as filesystem-name bytes is a resolution bug, not a spec mismatch. Resolving against the pathname alone (with the suffix retained as URL metadata) would match Node semantics while keeping the resolved file discoverable.

@XinwangTao commented on GitHub (Oct 15, 2026): Confirmed: Node resolves the request to the pathname and keeps ?query#fragment as URL components, so treating them as filesystem-name bytes is a resolution bug, not a spec mismatch. Resolving against the pathname alone (with the suffix retained as URL metadata) would match Node semantics while keeping the resolved file discoverable.

bsakaguchi commented

2026-10-16 21:06:43 +00:00

Author

Owner

@yankikucuk commented on GitHub (Oct 15, 2026):

Working on this. Ran your fixture through rea import-reference-source on a built checkout: main.mjs -> dep.mjs?revision=1#first stayed unresolved, and the Node import.meta.resolve control splits it into pathname dep.mjs, search ?revision=1, hash #first, regular file. The resolver now strips the URL suffix from ESM imports requests (static and dynamic) before candidate lookup, so the edge resolves to dep.mjs as internal; CommonJS require keeps those characters as name bytes. Draft PR follows shortly.

@yankikucuk commented on GitHub (Oct 15, 2026): Working on this. Ran your fixture through `rea import-reference-source` on a built checkout: `main.mjs -> dep.mjs?revision=1#first` stayed unresolved, and the Node `import.meta.resolve` control splits it into pathname `dep.mjs`, search `?revision=1`, hash `#first`, regular file. The resolver now strips the URL suffix from ESM `imports` requests (static and dynamic) before candidate lookup, so the edge resolves to `dep.mjs` as `internal`; CommonJS `require` keeps those characters as name bytes. Draft PR follows shortly.

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:11:52 +00:00

[PR #1483] [MERGED] feat(process): report bounded live capture progress #2921

bsakaguchi referenced this issue

2026-10-17 21:12:05 +00:00

[PR #1526] [MERGED] test(javascript): verify escaped receivers through CLI and MCP #2943

bsakaguchi closed this issue

2026-10-31 00:31:15 +00:00

bsakaguchi commented

2026-10-31 00:31:16 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Resolved by github.com/morluto/rea@f2e5fd148a. Historical ESM resolution separates URL query/fragment from inventoried pathname and preserves original source_specifier; versioned URL controls retain the existing hashed module. Validation: tests/boundary/cli/referenceSourceModuleSemantics.test.ts passed in the recorded round2/3 execution; this review did not rerun tests. Coverage is limited to the cited control; no broader real-provider, Windows or performance claim.

@morluto commented on GitHub (Oct 23, 2026): Resolved by https://github.com/morluto/rea/commit/f2e5fd148a85fa737302dd194a33172c4561093e. Historical ESM resolution separates URL query/fragment from inventoried pathname and preserves original source_specifier; versioned URL controls retain the existing hashed module. Validation: `tests/boundary/cli/referenceSourceModuleSemantics.test.ts` passed in the recorded round2/3 execution; this review did not rerun tests. Coverage is limited to the cited control; no broader real-provider, Windows or performance claim.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1526

No description provided.

Rows
Columns