[GH-ISSUE #7393] [Bug] Selecting one source makes version comparison mark an identical excluded file as changed #4401

New issue

Closed

opened 2026-10-31 00:49:30 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:49:30 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7393

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Comparing two analyses of the same two-file JavaScript directory reports one application change when the second analysis selects only main.js. The excluded lib.js asset matches by exact content digest and retains the same node ID, but its comparison item says changed with properties and relationships dimensions. The item's only limitation concerns unresolved relationship identity; it does not explain that lib.js was not analyzed on the right. The right source observation explicitly says outside-analysis-scope and not-examined. Overall comparison coverage remains partial, so the loss is in the item classification and explanation rather than all coverage metadata.

Reproduction

Use a fresh clone checked out at the SHA below and Node v24.22.1 on PATH. Install the locked dependencies with npm ci. The following is the source-backed CLI procedure I ran: it loads the unchanged production CLI and real source worker without building dist. The scratch launcher changes only source-module resolution, including the worker entry point; it does not replace analysis or comparison logic. The packaged CLI wrapper was not exercised.

WORK=$(mktemp -d)
mkdir -p "$WORK/app" "$WORK/home"
export HOME="$WORK/home" REA_LOG_LEVEL=silent
printf 'export function shared() {\n  return 1;\n}\n' > "$WORK/app/lib.js"
printf 'export const name = "one";\n' > "$WORK/app/main.js"
cat > "$WORK/source-cli.mjs" <<'JS'
import { writeFile } from 'node:fs/promises';
import { registerHooks } from 'node:module';
import { pathToFileURL } from 'node:url';
const root = pathToFileURL(`${process.cwd()}/`);
const worker = new URL('./worker.mjs', import.meta.url);
const loader = new URL('scripts/lib/verifier-source-loader.mjs', root);
await writeFile(worker, `import { importVerifierSource } from ${JSON.stringify(loader.href)};\nawait importVerifierSource('./src/composition/javascriptAnalysisWorkerProcess.ts', ${JSON.stringify(root.href)});\n`);
registerHooks({ resolve(specifier, context, nextResolve) {
  return specifier === '#javascript-analysis-worker'
    ? { url: worker.href, shortCircuit: true }
    : nextResolve(specifier, context);
} });
const { importVerifierSource } = await import(loader);
const { createCli, closeCliJavaScriptResources } = await importVerifierSource('./src/cli.ts', root);
try {
  await createCli(process.env).serve(process.argv.slice(2), {
    stdout: output => process.stdout.write(output),
    exit: code => { process.exitCode = code; }
  });
} finally { await closeCliJavaScriptResources(); }
JS
node "$WORK/source-cli.mjs" analyze-javascript-application "$WORK/app" --json > "$WORK/complete.json"
node "$WORK/source-cli.mjs" analyze-javascript-application "$WORK/app" --source-path main.js --json > "$WORK/selected.json"
node --input-type=module - "$WORK" <<'JS'
import { readFileSync, writeFileSync } from 'node:fs';
const work = process.argv[2];
const read = name => JSON.parse(readFileSync(`${work}/${name}.json`, 'utf8'));
writeFileSync(`${work}/input.json`, JSON.stringify({ left: read('complete'), right: read('selected') }));
JS
node "$WORK/source-cli.mjs" compare-application-versions "$WORK/input.json" --json > "$WORK/result.json"
node --input-type=module - "$WORK/result.json" <<'JS'
import { readFileSync } from 'node:fs';
const result = JSON.parse(readFileSync(process.argv[2], 'utf8')).normalized_result;
console.log(JSON.stringify({ summary: result.summary, changed: result.items.filter(item => item.status === 'changed') }, null, 2));
JS

Run the commands from the clone root. The source loader may print Node's experimental TypeScript-transformation warning on stderr. All three source-backed CLI commands completed successfully in my run, and a separate direct production-service replay produced the same affected item.

Expected behavior

An unexamined source should retain its selection uncertainty instead of becoming a definite application-property change solely because its analysis metadata differs. Expected source: AGENTS.md:19 requires consumers to preserve uncertainty from its owning producer, and AGENTS.md:21 keeps unresolved applicability unknown. The producer explicitly marks excluded JavaScript analysis not examined in src/application/javascript/JavaScriptArtifactGraphStructure.ts:662. The comparison documentation permits definite property differences under partial coverage in docs/javascript-application-workflows.md:353; this input contains an analysis-availability difference, without evidence that the excluded file's application properties changed. A conservative item should explain the unexamined scope and avoid asserting a definite application change from those markers alone.

Impact and scope

P2: this is a consequential false-positive version-comparison result for mixed source scopes, bounded by retained source Evidence and a practical workaround. In this fixture, comparing the full analysis against an explicit selection of both main.js and lib.js produces seven unchanged items and zero changes; lib.js is unchanged with no differing dimensions. Reanalyzing both versions with complete source coverage avoids this failure on the tested fixture. I did not test all file kinds, resource-limited analyses, or the packaged runtime. No files are modified by comparison, and the test executes no target code or network requests.

REA version

6.8.0; baseline f8f7ac96de

Runtime and platform

Node v24.22.1; macOS, Darwin 27.0.0, arm64

Target and artifact context

An inert local directory containing two small JavaScript ES modules, analyzed once completely and once with only main.js selected. No native provider or live target is involved.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The source-backed compare-application-versions command returned this summary:

{"unchanged":0,"added":0,"removed":0,"changed":1,"unknown":7}

The sole changed item describes lib.js:

{
  "node_kind": "javascript-asset",
  "status": "changed",
  "dimensions": ["properties", "relationships"],
  "match": {"status": "matched", "basis": "exact-content-digest", "confidence": "exact"},
  "limitations": ["Relationship target identity remains unresolved; equal node kinds do not establish cross-version alignment."]
}

Both asset identities have SHA-256 87ece0408f9afc809b55ea9859b3e5539c9aa138352fafb670c1034261b85580. The right asset's observation has text_status outside-analysis-scope, parse_status null, and syntax/examination status not-examined. Its unexamined_reason says the source is outside caller-selected source_paths. The right graph is partial, and comparison coverage retains right_graph_status partial.

Failure source: src/domain/javascript/javascriptApplicationVersionItems.ts:506 compares observation properties, and src/domain/javascript/javascriptApplicationVersionItems.ts:517 leaves analysis-status fields in that comparison. The uncertainty guard at src/domain/javascript/javascriptApplicationVersionItems.ts:437 checks the observation's Evidence state; this asset's bytes remain observed. The branch at src/domain/javascript/javascriptApplicationVersionItems.ts:160 consequently treats the analysis-status difference as an observed property change. Coverage is appended to item dimensions only for unknown items at src/domain/javascript/javascriptApplicationVersionItems.ts:177. Supported CLI registration calls the authenticated service in src/cli/applicationCommands.ts:51, and the service validates the supplied Evidence before comparison in src/application/javascript/JavaScriptApplicationWorkflowService.ts:182.

Additional context

References

#5534 (closed) fixed the same fixture's unselected scope placeholders being reported as added or removed. This report covers the remaining definite changed classification of an excluded, byte-identical asset; whether the two share a cause is not established.

The exact replacement item dimensions are a fix decision; this report establishes the false definite-change classification, not a prescribed implementation.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7393 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Comparing two analyses of the same two-file JavaScript directory reports one application change when the second analysis selects only main.js. The excluded lib.js asset matches by exact content digest and retains the same node ID, but its comparison item says changed with properties and relationships dimensions. The item's only limitation concerns unresolved relationship identity; it does not explain that lib.js was not analyzed on the right. The right source observation explicitly says outside-analysis-scope and not-examined. Overall comparison coverage remains partial, so the loss is in the item classification and explanation rather than all coverage metadata. ### Reproduction Use a fresh clone checked out at the SHA below and Node v24.22.1 on PATH. Install the locked dependencies with npm ci. The following is the source-backed CLI procedure I ran: it loads the unchanged production CLI and real source worker without building dist. The scratch launcher changes only source-module resolution, including the worker entry point; it does not replace analysis or comparison logic. The packaged CLI wrapper was not exercised. ```sh WORK=$(mktemp -d) mkdir -p "$WORK/app" "$WORK/home" export HOME="$WORK/home" REA_LOG_LEVEL=silent printf 'export function shared() {\n return 1;\n}\n' > "$WORK/app/lib.js" printf 'export const name = "one";\n' > "$WORK/app/main.js" cat > "$WORK/source-cli.mjs" <<'JS' import { writeFile } from 'node:fs/promises'; import { registerHooks } from 'node:module'; import { pathToFileURL } from 'node:url'; const root = pathToFileURL(`${process.cwd()}/`); const worker = new URL('./worker.mjs', import.meta.url); const loader = new URL('scripts/lib/verifier-source-loader.mjs', root); await writeFile(worker, `import { importVerifierSource } from ${JSON.stringify(loader.href)};\nawait importVerifierSource('./src/composition/javascriptAnalysisWorkerProcess.ts', ${JSON.stringify(root.href)});\n`); registerHooks({ resolve(specifier, context, nextResolve) { return specifier === '#javascript-analysis-worker' ? { url: worker.href, shortCircuit: true } : nextResolve(specifier, context); } }); const { importVerifierSource } = await import(loader); const { createCli, closeCliJavaScriptResources } = await importVerifierSource('./src/cli.ts', root); try { await createCli(process.env).serve(process.argv.slice(2), { stdout: output => process.stdout.write(output), exit: code => { process.exitCode = code; } }); } finally { await closeCliJavaScriptResources(); } JS node "$WORK/source-cli.mjs" analyze-javascript-application "$WORK/app" --json > "$WORK/complete.json" node "$WORK/source-cli.mjs" analyze-javascript-application "$WORK/app" --source-path main.js --json > "$WORK/selected.json" node --input-type=module - "$WORK" <<'JS' import { readFileSync, writeFileSync } from 'node:fs'; const work = process.argv[2]; const read = name => JSON.parse(readFileSync(`${work}/${name}.json`, 'utf8')); writeFileSync(`${work}/input.json`, JSON.stringify({ left: read('complete'), right: read('selected') })); JS node "$WORK/source-cli.mjs" compare-application-versions "$WORK/input.json" --json > "$WORK/result.json" node --input-type=module - "$WORK/result.json" <<'JS' import { readFileSync } from 'node:fs'; const result = JSON.parse(readFileSync(process.argv[2], 'utf8')).normalized_result; console.log(JSON.stringify({ summary: result.summary, changed: result.items.filter(item => item.status === 'changed') }, null, 2)); JS ``` Run the commands from the clone root. The source loader may print Node's experimental TypeScript-transformation warning on stderr. All three source-backed CLI commands completed successfully in my run, and a separate direct production-service replay produced the same affected item. ### Expected behavior An unexamined source should retain its selection uncertainty instead of becoming a definite application-property change solely because its analysis metadata differs. Expected source: AGENTS.md:19 requires consumers to preserve uncertainty from its owning producer, and AGENTS.md:21 keeps unresolved applicability unknown. The producer explicitly marks excluded JavaScript analysis not examined in src/application/javascript/JavaScriptArtifactGraphStructure.ts:662. The comparison documentation permits definite property differences under partial coverage in docs/javascript-application-workflows.md:353; this input contains an analysis-availability difference, without evidence that the excluded file's application properties changed. A conservative item should explain the unexamined scope and avoid asserting a definite application change from those markers alone. ### Impact and scope P2: this is a consequential false-positive version-comparison result for mixed source scopes, bounded by retained source Evidence and a practical workaround. In this fixture, comparing the full analysis against an explicit selection of both main.js and lib.js produces seven unchanged items and zero changes; lib.js is unchanged with no differing dimensions. Reanalyzing both versions with complete source coverage avoids this failure on the tested fixture. I did not test all file kinds, resource-limited analyses, or the packaged runtime. No files are modified by comparison, and the test executes no target code or network requests. ### REA version 6.8.0; baseline f8f7ac96de8a66a4ab9a23d87e04779bfcd11045 ### Runtime and platform Node v24.22.1; macOS, Darwin 27.0.0, arm64 ### Target and artifact context An inert local directory containing two small JavaScript ES modules, analyzed once completely and once with only main.js selected. No native provider or live target is involved. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The source-backed compare-application-versions command returned this summary: ```json {"unchanged":0,"added":0,"removed":0,"changed":1,"unknown":7} ``` The sole changed item describes lib.js: ```json { "node_kind": "javascript-asset", "status": "changed", "dimensions": ["properties", "relationships"], "match": {"status": "matched", "basis": "exact-content-digest", "confidence": "exact"}, "limitations": ["Relationship target identity remains unresolved; equal node kinds do not establish cross-version alignment."] } ``` Both asset identities have SHA-256 87ece0408f9afc809b55ea9859b3e5539c9aa138352fafb670c1034261b85580. The right asset's observation has text_status outside-analysis-scope, parse_status null, and syntax/examination status not-examined. Its unexamined_reason says the source is outside caller-selected source_paths. The right graph is partial, and comparison coverage retains right_graph_status partial. Failure source: src/domain/javascript/javascriptApplicationVersionItems.ts:506 compares observation properties, and src/domain/javascript/javascriptApplicationVersionItems.ts:517 leaves analysis-status fields in that comparison. The uncertainty guard at src/domain/javascript/javascriptApplicationVersionItems.ts:437 checks the observation's Evidence state; this asset's bytes remain observed. The branch at src/domain/javascript/javascriptApplicationVersionItems.ts:160 consequently treats the analysis-status difference as an observed property change. Coverage is appended to item dimensions only for unknown items at src/domain/javascript/javascriptApplicationVersionItems.ts:177. Supported CLI registration calls the authenticated service in src/cli/applicationCommands.ts:51, and the service validates the supplied Evidence before comparison in src/application/javascript/JavaScriptApplicationWorkflowService.ts:182. ### Additional context ## References - #5534 (closed) fixed the same fixture's unselected scope placeholders being reported as added or removed. This report covers the remaining definite `changed` classification of an excluded, byte-identical asset; whether the two share a cause is not established. The exact replacement item dimensions are a fix decision; this report establishes the false definite-change classification, not a prescribed implementation.

bsakaguchi closed this issue

2026-10-31 00:49:30 +00:00

bsakaguchi referenced this issue

2026-10-31 01:12:44 +00:00

[PR #4401] [CLOSED] fix(reference): retain static template require dependencies #6214

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425