morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3171] [Bug] The documented artifact comparison handoff fails after summary inspection #1109

New issue

Closed

opened 2026-10-15 18:31:08 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:08 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3171

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

Two successful inspect_artifact calls with detail: "summary" return Evidence that compare_artifacts rejects when passed unchanged as left and right, despite the documented direct handoff. The production MCP stdio server returned isError: true, invalid_request, and Artifact comparison requires inspect_artifact Evidence for two local ZIP archives in one connection.

The summaries have operation: "inspect_analysis_view" and normalized_result.parent_operation: "inspect_artifact". Their complete parent inspections remain retained. Reading the left parent's occurrence page after opening the right archive succeeded and returned . and a.txt. Reopening both archives and inspecting with detail omitted produced complete Evidence that comparison accepted, returning status: "changed", a changed root, and an added b.txt.

The live comparison tool description requests complete inventory Evidence. The defect is its disagreement with the documentation's unqualified direct-handoff promise for a supported summary delivery option.

Reproduction

Start from a clean checkout at commit 58300885207133422af235e0f2b10658cf958b03 with the documented Node.js toolchain, npm, and Python 3 available. Install locked dependencies and build the runtime. The verifier used an already prepared checkout and did not rerun these two preparation commands.

npm ci
npm run build:cached

From the repository root, create an isolated local fixture directory and configuration directories. Run the remaining commands in this same shell.

mkdir -p .artifact-summary-repro/home/config .artifact-summary-repro/home/cache .artifact-summary-repro/home/data

Create two local ZIP archives. Each contains a.txt; the second also contains b.txt.

python3 - <<'PY'
import zipfile
from pathlib import Path
root = Path('.artifact-summary-repro')
for name, entries in [('one.zip', {'a.txt': b'same'}), ('two.zip', {'a.txt': b'same', 'b.txt': b'new '})]:
    with zipfile.ZipFile(root / name, 'w') as archive:
        for path, content in entries.items():
            archive.writestr(path, content)
PY

Use the checkout's locked MCP client SDK to call the production stdio server. This script passes each complete structuredContent object returned by summary inspection unchanged to the comparison tool. It then checks retained access and the complete-delivery workaround in the same connection.

node --input-type=module <<'JS'
import { Client } from '@modelcontextprotocol/client';
import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
import { resolve } from 'node:path';

const root = resolve('.artifact-summary-repro');
const client = new Client({ name: 'artifact-summary-repro', version: '1' });
const transport = new StdioClientTransport({
  command: process.execPath,
  args: [resolve('scripts/rea.mjs'), '--mcp'],
  env: {
    PATH: process.env.PATH,
    HOME: resolve(root, 'home'),
    XDG_CONFIG_HOME: resolve(root, 'home/config'),
    XDG_CACHE_HOME: resolve(root, 'home/cache'),
    XDG_DATA_HOME: resolve(root, 'home/data'),
    REA_LOG_LEVEL: 'silent',
    REA_ANALYSIS_PROVIDER: 'auto',
    HOPPER_LAUNCHER_PATH: resolve(root, 'unconfigured-hopper'),
  },
  stderr: 'pipe',
});
async function call(name, args = {}, allowError = false) {
  const result = await client.callTool({ name, arguments: args });
  if (result.isError && !allowError) throw new Error(JSON.stringify(result));
  return result;
}
async function inspectBoth(args) {
  const records = [];
  for (const file of ['one.zip', 'two.zip']) {
    await call('open_binary', { path: resolve(root, file) });
    records.push((await call('inspect_artifact', args)).structuredContent);
  }
  return records;
}
try {
  await client.connect(transport);
  const summaries = await inspectBoth({ detail: 'summary' });
  console.log('summary operation:', summaries[0].operation);
  console.log('summary parent operation:', summaries[0].normalized_result.parent_operation);
  const failed = await call('compare_artifacts', {
    left: summaries[0], right: summaries[1],
  }, true);
  console.log('summary comparison:', JSON.stringify(failed));
  const page = await call('inspect_analysis_view', {
    source: {
      kind: 'retained-evidence',
      evidence_id: summaries[0].normalized_result.parent_evidence_id,
    },
    view: { kind: 'inventory', collection: 'occurrences', offset: 0, limit: 2 },
  });
  console.log('retained paths:', page.structuredContent.normalized_result.items.map(x => x.logical_path));
  const complete = await inspectBoth({});
  const control = await call('compare_artifacts', {
    left: complete[0], right: complete[1],
  });
  const result = control.structuredContent.normalized_result;
  console.log('complete comparison:', result.status);
  console.log('complete changes:', result.changes.map(x => [x.logical_path, x.classification]));
} finally {
  await client.close();
  await transport.close();
}
JS

Expected behavior

docs/mcp-contracts.md:18-22 says, "Pass the returned Evidence directly to a compatible comparison tool" and specifically names the inspect_artifact to compare_artifacts handoff. docs/mcp-contracts.md:195-207 documents detail: "summary" for artifact inspection, explains that it returns derived inspect_analysis_view Evidence, and promises retention of the complete inspection and atomic inventory. Neither passage states a complete-only exception to the handoff.

Callers need a consistent working contract. If the direct-handoff promise remains unqualified, passing the two unchanged summary records should yield the changed comparison and added b.txt. If comparison intentionally requires complete Evidence, each handoff instruction should state that restriction and describe a supported working path.

Impact and scope

P2. Following the documented handoff blocks this MCP comparison attempt when inspection uses supported summary delivery. The reproduced complete-Evidence workaround preserves the expected comparison, and retained parent access succeeds. No lost Evidence, incorrect computed comparison, severe harm, or security boundary crossing was observed. The verification covers two small local ZIP fixtures in one production stdio session. Large-artifact response sizes were not measured.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node.js v26.10.0; macOS. The verifier record does not state the OS version or architecture.

Target and artifact context

Two local ZIP archives. one.zip contains a.txt with bytes same; two.zip contains that identical member plus b.txt with bytes new . The production artifact provider handled both in one MCP connection with isolated configuration directories. No deep analysis engine, fake provider, or external network API was needed for the executed verification.

Interface

MCP

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Summary Evidence operation: inspect_analysis_view
Summary parent operation: inspect_artifact
Summary comparison: isError=true
Error code: invalid_request
Error category: invalid_input
Error issue: Artifact comparison requires inspect_artifact Evidence
Retained left parent occurrence paths: ., a.txt
Complete comparison status: changed
Complete comparison changes: changed root; added b.txt

Typed outcome or recovery expected

Preserve the successful complete-record comparison, readable retained parent, summary parent identity, and required inventory coverage. If summary Evidence is supported as a comparison input, resolve it through the supported retained parent and return the expected changed comparison. If complete Evidence is required, make that restriction consistent across the documented handoff and give callers the working recovery. Reopening each archive, inspecting with detail omitted, and comparing the complete returned records succeeded for these fixtures. Do not infer equality from absent inventories or discard coverage to accept the input.

Analysis or suspected cause

Source-derived at the stated baseline. src/server/registerArtifactComparisonTool.ts:33-39 parses the supplied Evidence and passes it to compareArtifacts without resolving the retained parent. src/domain/artifactComparison.ts:103-104 parses both inventory inputs. src/domain/artifactInventoryEvidence.ts:58-69 accepts inventory_artifact or inspect_artifact operations and rejects the summary's inspect_analysis_view operation. The operation guard fails before an inspection substeps read; missing summary substeps is not the immediate rejecting condition. src/contracts/sessionToolContracts.ts:106-109 advertises the narrower complete-inventory requirement, which the verifier also read from the live tool catalog.

Additional context

Related reports are #551, #723, and #1050. This differs because valid returned summary Evidence is rejected by its operation during artifact comparison, whereas those reports concern missing inline normalized results, representation costs, or selected-view delivery.

Related: #3220 reports a separate failure in the same area; the two have independent fixes.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3171 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area MCP or agent integration ### Observed behavior Two successful `inspect_artifact` calls with `detail: "summary"` return Evidence that `compare_artifacts` rejects when passed unchanged as `left` and `right`, despite the documented direct handoff. The production MCP stdio server returned `isError: true`, `invalid_request`, and `Artifact comparison requires inspect_artifact Evidence` for two local ZIP archives in one connection. The summaries have `operation: "inspect_analysis_view"` and `normalized_result.parent_operation: "inspect_artifact"`. Their complete parent inspections remain retained. Reading the left parent's occurrence page after opening the right archive succeeded and returned `.` and `a.txt`. Reopening both archives and inspecting with detail omitted produced complete Evidence that comparison accepted, returning `status: "changed"`, a changed root, and an added `b.txt`. The live comparison tool description requests complete inventory Evidence. The defect is its disagreement with the documentation's unqualified direct-handoff promise for a supported summary delivery option. ### Reproduction 1. Start from a clean checkout at commit `58300885207133422af235e0f2b10658cf958b03` with the documented Node.js toolchain, npm, and Python 3 available. Install locked dependencies and build the runtime. The verifier used an already prepared checkout and did not rerun these two preparation commands. ```sh npm ci npm run build:cached ``` 2. From the repository root, create an isolated local fixture directory and configuration directories. Run the remaining commands in this same shell. ```sh mkdir -p .artifact-summary-repro/home/config .artifact-summary-repro/home/cache .artifact-summary-repro/home/data ``` 3. Create two local ZIP archives. Each contains `a.txt`; the second also contains `b.txt`. ```sh python3 - <<'PY' import zipfile from pathlib import Path root = Path('.artifact-summary-repro') for name, entries in [('one.zip', {'a.txt': b'same'}), ('two.zip', {'a.txt': b'same', 'b.txt': b'new '})]: with zipfile.ZipFile(root / name, 'w') as archive: for path, content in entries.items(): archive.writestr(path, content) PY ``` 4. Use the checkout's locked MCP client SDK to call the production stdio server. This script passes each complete `structuredContent` object returned by summary inspection unchanged to the comparison tool. It then checks retained access and the complete-delivery workaround in the same connection. ```sh node --input-type=module <<'JS' import { Client } from '@modelcontextprotocol/client'; import { StdioClientTransport } from '@modelcontextprotocol/client/stdio'; import { resolve } from 'node:path'; const root = resolve('.artifact-summary-repro'); const client = new Client({ name: 'artifact-summary-repro', version: '1' }); const transport = new StdioClientTransport({ command: process.execPath, args: [resolve('scripts/rea.mjs'), '--mcp'], env: { PATH: process.env.PATH, HOME: resolve(root, 'home'), XDG_CONFIG_HOME: resolve(root, 'home/config'), XDG_CACHE_HOME: resolve(root, 'home/cache'), XDG_DATA_HOME: resolve(root, 'home/data'), REA_LOG_LEVEL: 'silent', REA_ANALYSIS_PROVIDER: 'auto', HOPPER_LAUNCHER_PATH: resolve(root, 'unconfigured-hopper'), }, stderr: 'pipe', }); async function call(name, args = {}, allowError = false) { const result = await client.callTool({ name, arguments: args }); if (result.isError && !allowError) throw new Error(JSON.stringify(result)); return result; } async function inspectBoth(args) { const records = []; for (const file of ['one.zip', 'two.zip']) { await call('open_binary', { path: resolve(root, file) }); records.push((await call('inspect_artifact', args)).structuredContent); } return records; } try { await client.connect(transport); const summaries = await inspectBoth({ detail: 'summary' }); console.log('summary operation:', summaries[0].operation); console.log('summary parent operation:', summaries[0].normalized_result.parent_operation); const failed = await call('compare_artifacts', { left: summaries[0], right: summaries[1], }, true); console.log('summary comparison:', JSON.stringify(failed)); const page = await call('inspect_analysis_view', { source: { kind: 'retained-evidence', evidence_id: summaries[0].normalized_result.parent_evidence_id, }, view: { kind: 'inventory', collection: 'occurrences', offset: 0, limit: 2 }, }); console.log('retained paths:', page.structuredContent.normalized_result.items.map(x => x.logical_path)); const complete = await inspectBoth({}); const control = await call('compare_artifacts', { left: complete[0], right: complete[1], }); const result = control.structuredContent.normalized_result; console.log('complete comparison:', result.status); console.log('complete changes:', result.changes.map(x => [x.logical_path, x.classification])); } finally { await client.close(); await transport.close(); } JS ``` ### Expected behavior `docs/mcp-contracts.md:18-22` says, "Pass the returned Evidence directly to a compatible comparison tool" and specifically names the `inspect_artifact` to `compare_artifacts` handoff. `docs/mcp-contracts.md:195-207` documents `detail: "summary"` for artifact inspection, explains that it returns derived `inspect_analysis_view` Evidence, and promises retention of the complete inspection and atomic inventory. Neither passage states a complete-only exception to the handoff. Callers need a consistent working contract. If the direct-handoff promise remains unqualified, passing the two unchanged summary records should yield the changed comparison and added `b.txt`. If comparison intentionally requires complete Evidence, each handoff instruction should state that restriction and describe a supported working path. ### Impact and scope P2. Following the documented handoff blocks this MCP comparison attempt when inspection uses supported summary delivery. The reproduced complete-Evidence workaround preserves the expected comparison, and retained parent access succeeds. No lost Evidence, incorrect computed comparison, severe harm, or security boundary crossing was observed. The verification covers two small local ZIP fixtures in one production stdio session. Large-artifact response sizes were not measured. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node.js v26.10.0; macOS. The verifier record does not state the OS version or architecture. ### Target and artifact context Two local ZIP archives. `one.zip` contains `a.txt` with bytes `same`; `two.zip` contains that identical member plus `b.txt` with bytes `new `. The production artifact provider handled both in one MCP connection with isolated configuration directories. No deep analysis engine, fake provider, or external network API was needed for the executed verification. ### Interface MCP ### Earliest failure stage Request parsing or validation ### Relevant output or evidence ```text Summary Evidence operation: inspect_analysis_view Summary parent operation: inspect_artifact Summary comparison: isError=true Error code: invalid_request Error category: invalid_input Error issue: Artifact comparison requires inspect_artifact Evidence Retained left parent occurrence paths: ., a.txt Complete comparison status: changed Complete comparison changes: changed root; added b.txt ``` ### Typed outcome or recovery expected Preserve the successful complete-record comparison, readable retained parent, summary parent identity, and required inventory coverage. If summary Evidence is supported as a comparison input, resolve it through the supported retained parent and return the expected changed comparison. If complete Evidence is required, make that restriction consistent across the documented handoff and give callers the working recovery. Reopening each archive, inspecting with detail omitted, and comparing the complete returned records succeeded for these fixtures. Do not infer equality from absent inventories or discard coverage to accept the input. ### Analysis or suspected cause Source-derived at the stated baseline. `src/server/registerArtifactComparisonTool.ts:33-39` parses the supplied Evidence and passes it to `compareArtifacts` without resolving the retained parent. `src/domain/artifactComparison.ts:103-104` parses both inventory inputs. `src/domain/artifactInventoryEvidence.ts:58-69` accepts `inventory_artifact` or `inspect_artifact` operations and rejects the summary's `inspect_analysis_view` operation. The operation guard fails before an inspection `substeps` read; missing summary substeps is not the immediate rejecting condition. `src/contracts/sessionToolContracts.ts:106-109` advertises the narrower complete-inventory requirement, which the verifier also read from the live tool catalog. ### Additional context Related reports are [#551](https://github.com/morluto/rea/issues/551), [#723](https://github.com/morluto/rea/issues/723), and [#1050](https://github.com/morluto/rea/issues/1050). This differs because valid returned summary Evidence is rejected by its operation during artifact comparison, whereas those reports concern missing inline normalized results, representation costs, or selected-view delivery. Related: #3220 reports a separate failure in the same area; the two have independent fixes.

bsakaguchi

2026-10-15 18:31:08 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:09 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Delivered on main in fffb07419.

Artifact comparison could not consume an analysis summary handoff. Summary resolution now requires the exact retained parent and canonical reprojection equality; the shared inventory owner authenticates every page, subject and relationship.

Validation: Public MCP summary-to-comparison handoff, tampered provenance controls, and display-only identity preservation. Full exact-revision qualification passed.

@morluto commented on GitHub (Oct 14, 2026): Delivered on main in [fffb07419](https://github.com/morluto/rea/commit/fffb0741936067530d1ce9777461cd53e87398c2). Artifact comparison could not consume an analysis summary handoff. Summary resolution now requires the exact retained parent and canonical reprojection equality; the shared inventory owner authenticates every page, subject and relationship. Validation: Public MCP summary-to-comparison handoff, tampered provenance controls, and display-only identity preservation. Full exact-revision [qualification](https://github.com/morluto/rea/actions/runs/38629313022) passed.

bsakaguchi referenced this issue

2026-10-16 21:02:34 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:08:42 +00:00

[PR #1109] [MERGED] fix(browser): compose nested source-map column offsets #2657