[GH-ISSUE #3956] [Bug] Duplicate ZIP paths inflate the unchanged artifact comparison count #1561

New issue

Closed

opened 2026-10-16 21:07:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:07:11 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3956

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Observed in the initial run and again when replaying this draft's reproduction program, for two duplicate-fixture comparisons and two unique-name controls in total. Both CLI inspections exited successfully, and both MCP calls returned successful Evidence. Both inventories declared complete container-entries coverage. The duplicate inventory retained ., neighbor.txt, x.txt, and x.txt, with distinct occurrence IDs for the repeated path.

The duplicate comparison returned:

{
  "status": "unknown",
  "summary": {
    "unchanged": 3,
    "added": 0,
    "removed": 0,
    "changed": 0,
    "unknown": 1,
    "contradiction": 0
  }
}

Its only change row classified x.txt as unknown with the availability dimension and null occurrence and artifact IDs. Its limitation said that duplicate logical paths cannot be paired across inventories and comparisons for those paths are unknown. The unique-name control retained ., neighbor.txt, x.txt, and y.txt, returned status: "unchanged", and reported unchanged: 4, unknown: 0, and no changes.

Reproduction

Start from the stated baseline with installed dependencies and its built runtime, and use the stated Node.js version. Run the following program from the repository root. It creates a duplicate-name ZIP and a unique-name control, inventories each through the public CLI, and passes the unchanged nested inventory Evidence to the registered MCP comparison tool. It isolates the child processes' home directory and removes its disposable files when finished.

node --input-type=module <<'JS'
import { createRequire } from 'node:module';
import { pathToFileURL } from 'node:url';
import { mkdirSync, writeFileSync, mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { resolve, join } from 'node:path';
import { spawnSync } from 'node:child_process';
const require = createRequire(resolve('package.json'));
const { Client } = await import(pathToFileURL(require.resolve('@modelcontextprotocol/client')));
const { StdioClientTransport } = await import(pathToFileURL(require.resolve('@modelcontextprotocol/client/stdio')));
const { ZipWriter, TextReader, Uint8ArrayWriter } = await import(pathToFileURL(require.resolve('@zip.js/zip.js')));
const root = mkdtempSync(join(tmpdir(), 'rea-artifact-comparison-'));
mkdirSync(join(root, 'home'), {recursive:true});
const env = {PATH:process.env.PATH, HOME:join(root,'home'), REA_LOG_LEVEL:'silent'};
const transport = new StdioClientTransport({command:process.execPath,args:[resolve('scripts/rea.mjs'),'mcp'],env,stderr:'pipe'});
const client = new Client({name:'artifact-comparison-probe',version:'1'});
try {
  await client.connect(transport);
  for (const duplicate of [true,false]) {
    const writer = new ZipWriter(new Uint8ArrayWriter());
    for (const [name, text] of [['x.txt','first'],['y.txt','second'],['neighbor.txt','neighbor']]) await writer.add(name,new TextReader(text));
    const bytes = Buffer.from(await writer.close());
    if (duplicate) for (const [sig,size] of [[Buffer.from([0x50,0x4b,0x03,0x04]),30],[Buffer.from([0x50,0x4b,0x01,0x02]),46]]) {
      const first = bytes.indexOf(sig), second = bytes.indexOf(sig,first+4);
      if (bytes.subarray(second+size,second+size+5).toString() !== 'y.txt') throw Error('ZIP fixture mismatch');
      bytes.write('x.txt', second+size);
    }
    const path = join(root,duplicate?'duplicate.zip':'unique.zip');
    writeFileSync(path,bytes);
    const run = spawnSync(process.execPath,['scripts/rea.mjs','inspect-artifact',path,'--json'],{env,encoding:'utf8',maxBuffer:10*1024*1024});
    if (run.status !== 0) throw Error(JSON.stringify({stage:'inspect',status:run.status,stdout:run.stdout,stderr:run.stderr}));
    const inspection = JSON.parse(run.stdout);
    const evidence = inspection.normalized_result.substeps.find(s=>s.evidence.operation==='inventory_artifact').evidence;
    const reply = await client.callTool({name:'compare_artifacts',arguments:{left:evidence,right:evidence}});
    if (reply.isError || !reply.structuredContent) throw Error(JSON.stringify(reply));
    console.log(JSON.stringify({duplicate, paths:evidence.normalized_result.occurrences.map(o=>o.logical_path), comparison:reply.structuredContent.normalized_result}, null, 2));
  }
} finally {
  await client.close();
  await transport.close();
  rmSync(root, {recursive:true, force:true});
}
JS

For the duplicate case, inspect the four occurrence paths, the unknown change at x.txt, and the unchanged count of three. Compare the control, which returns four unchanged entries and no changes.

Expected behavior

The comparison must preserve unknown pairing for duplicate logical paths throughout its result. An occurrence on a path classified as unknown must not contribute to an affirmative unchanged count. For this fixture, only . and neighbor.txt qualify, so summary.unchanged should be two. This does not require a particular path-versus-occurrence aggregation policy for summary.unknown.

Impact and scope

Duplicate-name ZIP inputs produce a consequential incorrect comparison summary. The observed scope is bounded to the numerical counts; the returned status and path-level change still identify the unresolved comparison. No data loss or execution effect was observed.

Workaround: The observed result exposes status: "unknown", an unknown change at the repeated path, and an explicit pairing limitation. Callers can use those fields to keep the path unresolved instead of trusting the unchanged count; callers needing counts can reconcile those paths against the supplied inventory occurrences.

REA version

rea 6.5.0 at source commit 4692bb3330; current-main source checked at 22ec8bdb2f

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Authentic inventory Evidence produced by inspect-artifact for a ZIP with x.txt twice plus neighbor.txt, compared unchanged against itself through MCP.

Interface

MCP

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

src/contracts/sessionToolContracts.ts:105-108 accepts nested inventory Evidence and requires comparison gaps to remain unknown; docs/mcp-contracts.md:24-33 requires complete comparable observations for equality. src/domain/artifactComparison.ts:166-170,230-243 classifies duplicate paths as unknown but emits one row per path; lines 485-499 subtract row counts from occurrence counts. Public CLI inspection and production stdio MCP comparison reproduced unchanged: 3 for four occurrences with two unresolved x.txt entries. The unique-name control returned unchanged: 4 with no changes.

Typed outcome or recovery expected

Keep x.txt unknown, exclude both unresolved occurrences from unchanged, and retain unchanged: 4 for the unique-name control.

Analysis or suspected cause

Source-derived: the summary mixes a path-level unknown count with an occurrence-level total. Only . and neighbor.txt are outside the unresolved path, so unchanged must be 2. This report does not prescribe whether summary.unknown counts paths or occurrences.

Additional context

Related closed reports cover different contracts: #3152 added duplicate-member inventory support; #1844 concerns a changed root counted twice in partial inventories; #2173 concerns complete plist inventories classified unknown. This report concerns summary accounting after successful duplicate-member inventory.

Freshness check: the failing source is unchanged on current upstream main at 22ec8bdb2f. The only change among the cited files is unrelated CLI output-flag normalization. Runtime observations above are from the stated baseline; current-main persistence is source-derived.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3956 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Observed in the initial run and again when replaying this draft's reproduction program, for two duplicate-fixture comparisons and two unique-name controls in total. Both CLI inspections exited successfully, and both MCP calls returned successful Evidence. Both inventories declared complete `container-entries` coverage. The duplicate inventory retained `.`, `neighbor.txt`, `x.txt`, and `x.txt`, with distinct occurrence IDs for the repeated path. The duplicate comparison returned: ```json { "status": "unknown", "summary": { "unchanged": 3, "added": 0, "removed": 0, "changed": 0, "unknown": 1, "contradiction": 0 } } ``` Its only change row classified `x.txt` as `unknown` with the `availability` dimension and null occurrence and artifact IDs. Its limitation said that duplicate logical paths cannot be paired across inventories and comparisons for those paths are unknown. The unique-name control retained `.`, `neighbor.txt`, `x.txt`, and `y.txt`, returned `status: "unchanged"`, and reported `unchanged: 4`, `unknown: 0`, and no changes. ### Reproduction 1. Start from the stated baseline with installed dependencies and its built runtime, and use the stated Node.js version. Run the following program from the repository root. It creates a duplicate-name ZIP and a unique-name control, inventories each through the public CLI, and passes the unchanged nested inventory Evidence to the registered MCP comparison tool. It isolates the child processes' home directory and removes its disposable files when finished. ```sh node --input-type=module <<'JS' import { createRequire } from 'node:module'; import { pathToFileURL } from 'node:url'; import { mkdirSync, writeFileSync, mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { resolve, join } from 'node:path'; import { spawnSync } from 'node:child_process'; const require = createRequire(resolve('package.json')); const { Client } = await import(pathToFileURL(require.resolve('@modelcontextprotocol/client'))); const { StdioClientTransport } = await import(pathToFileURL(require.resolve('@modelcontextprotocol/client/stdio'))); const { ZipWriter, TextReader, Uint8ArrayWriter } = await import(pathToFileURL(require.resolve('@zip.js/zip.js'))); const root = mkdtempSync(join(tmpdir(), 'rea-artifact-comparison-')); mkdirSync(join(root, 'home'), {recursive:true}); const env = {PATH:process.env.PATH, HOME:join(root,'home'), REA_LOG_LEVEL:'silent'}; const transport = new StdioClientTransport({command:process.execPath,args:[resolve('scripts/rea.mjs'),'mcp'],env,stderr:'pipe'}); const client = new Client({name:'artifact-comparison-probe',version:'1'}); try { await client.connect(transport); for (const duplicate of [true,false]) { const writer = new ZipWriter(new Uint8ArrayWriter()); for (const [name, text] of [['x.txt','first'],['y.txt','second'],['neighbor.txt','neighbor']]) await writer.add(name,new TextReader(text)); const bytes = Buffer.from(await writer.close()); if (duplicate) for (const [sig,size] of [[Buffer.from([0x50,0x4b,0x03,0x04]),30],[Buffer.from([0x50,0x4b,0x01,0x02]),46]]) { const first = bytes.indexOf(sig), second = bytes.indexOf(sig,first+4); if (bytes.subarray(second+size,second+size+5).toString() !== 'y.txt') throw Error('ZIP fixture mismatch'); bytes.write('x.txt', second+size); } const path = join(root,duplicate?'duplicate.zip':'unique.zip'); writeFileSync(path,bytes); const run = spawnSync(process.execPath,['scripts/rea.mjs','inspect-artifact',path,'--json'],{env,encoding:'utf8',maxBuffer:10*1024*1024}); if (run.status !== 0) throw Error(JSON.stringify({stage:'inspect',status:run.status,stdout:run.stdout,stderr:run.stderr})); const inspection = JSON.parse(run.stdout); const evidence = inspection.normalized_result.substeps.find(s=>s.evidence.operation==='inventory_artifact').evidence; const reply = await client.callTool({name:'compare_artifacts',arguments:{left:evidence,right:evidence}}); if (reply.isError || !reply.structuredContent) throw Error(JSON.stringify(reply)); console.log(JSON.stringify({duplicate, paths:evidence.normalized_result.occurrences.map(o=>o.logical_path), comparison:reply.structuredContent.normalized_result}, null, 2)); } } finally { await client.close(); await transport.close(); rmSync(root, {recursive:true, force:true}); } JS ``` 2. For the duplicate case, inspect the four occurrence paths, the unknown change at `x.txt`, and the unchanged count of three. Compare the control, which returns four unchanged entries and no changes. ### Expected behavior The comparison must preserve unknown pairing for duplicate logical paths throughout its result. An occurrence on a path classified as unknown must not contribute to an affirmative unchanged count. For this fixture, only `.` and `neighbor.txt` qualify, so `summary.unchanged` should be two. This does not require a particular path-versus-occurrence aggregation policy for `summary.unknown`. ### Impact and scope Duplicate-name ZIP inputs produce a consequential incorrect comparison summary. The observed scope is bounded to the numerical counts; the returned status and path-level change still identify the unresolved comparison. No data loss or execution effect was observed. Workaround: The observed result exposes `status: "unknown"`, an unknown change at the repeated path, and an explicit pairing limitation. Callers can use those fields to keep the path unresolved instead of trusting the unchanged count; callers needing counts can reconcile those paths against the supplied inventory occurrences. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860; current-main source checked at 22ec8bdb2f7d4626587f23b1eed4a207df600386 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Authentic inventory Evidence produced by inspect-artifact for a ZIP with x.txt twice plus neighbor.txt, compared unchanged against itself through MCP. ### Interface MCP ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence `src/contracts/sessionToolContracts.ts:105-108` accepts nested inventory Evidence and requires comparison gaps to remain unknown; `docs/mcp-contracts.md:24-33` requires complete comparable observations for equality. `src/domain/artifactComparison.ts:166-170,230-243` classifies duplicate paths as unknown but emits one row per path; lines 485-499 subtract row counts from occurrence counts. Public CLI inspection and production stdio MCP comparison reproduced unchanged: 3 for four occurrences with two unresolved x.txt entries. The unique-name control returned unchanged: 4 with no changes. ### Typed outcome or recovery expected Keep x.txt unknown, exclude both unresolved occurrences from unchanged, and retain unchanged: 4 for the unique-name control. ### Analysis or suspected cause Source-derived: the summary mixes a path-level unknown count with an occurrence-level total. Only . and neighbor.txt are outside the unresolved path, so unchanged must be 2. This report does not prescribe whether summary.unknown counts paths or occurrences. ### Additional context Related closed reports cover different contracts: #3152 added duplicate-member inventory support; #1844 concerns a changed root counted twice in partial inventories; #2173 concerns complete plist inventories classified unknown. This report concerns summary accounting after successful duplicate-member inventory. Freshness check: the failing source is unchanged on current upstream main at 22ec8bdb2f7d4626587f23b1eed4a207df600386. The only change among the cited files is unrelated CLI output-flag normalization. Runtime observations above are from the stated baseline; current-main persistence is source-derived.

bsakaguchi closed this issue

2026-10-16 21:07:11 +00:00

bsakaguchi referenced this issue

2026-10-17 21:04:14 +00:00

[GH-ISSUE #4625] [Bug] OMP path guidance does not distinguish PI_CONFIG_DIR's home-relative base #2406

bsakaguchi referenced this issue

2026-10-17 21:12:24 +00:00

[PR #1562] [MERGED] fix(setup): reject invalid OMP profiles before fallback writes #2973