[GH-ISSUE #1364] [Bug] MCP version comparison drops input refinements and returns unreadable_output #473

New issue

Closed

opened 2026-10-15 18:25:21 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:25:21 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 9, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1364

MCP version comparison accepts the same application Evidence for both roles, then returns execution_failure / unreadable_output with generic retry/doctor advice. CLI correctly rejects that request as invalid_request, identifying right and requiring distinct Evidence.

Reproduced against main 37fcea78cecbeeadd869e9a3d77dd133ed264a0a, compiled with the unrelated CLI JSON follow-up. Actual stdio MCP analysis of a tiny inert JavaScript directory succeeds, then this retained request reaches the comparison producer:

{
  "name": "compare_application_versions",
  "arguments": {
    "left": { "kind": "retained-evidence", "evidence_id": "<the analysis Evidence ID>" },
    "right": { "kind": "retained-evidence", "evidence_id": "<the same Evidence ID>" }
  }
}

A complete real Notion ASAR workflow encounters the same error after successful analysis and retained summary/module views. Application code was parsed as text, never executed. Following tracing remains usable.

Root cause: compareApplicationVersionsInputSchema already requires distinct application Evidence and unique native observations on each side. The MCP reference-aware input path does not preserve those refinements and calls the validated application entry point. With identical inputs, producer provenance deduplicates to one ID, then the ordinary two-source output schema rejects it.

Expected behavior:

Apply the existing distinct-input and per-side unique-native-observation rules consistently to inline, retained and mixed MCP representations.
Reject invalid input before computation/publication with invalid_request, the affected field and actionable reason, matching CLI behavior.
Keep valid distinct-input comparison, full provenance, coverage and unknowns unchanged.
Exercise real producer data through CLI and SDK stdio MCP, including malformed requests followed by valid comparison/tracing and a working connection.

This is an input-validation/CLI–MCP parity defect, not an added self-comparison capability or a transport framing defect. The current graph contract also excludes empty graphs; that initially suspected adjacent case is outside the valid input domain. Related tracking: #1065.

Originally created by @N0zoM1z0 on GitHub (Oct 9, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1364 MCP version comparison accepts the same application Evidence for both roles, then returns `execution_failure / unreadable_output` with generic retry/doctor advice. CLI correctly rejects that request as `invalid_request`, identifying `right` and requiring distinct Evidence. Reproduced against main `37fcea78cecbeeadd869e9a3d77dd133ed264a0a`, compiled with the unrelated CLI JSON follow-up. Actual stdio MCP analysis of a tiny inert JavaScript directory succeeds, then this retained request reaches the comparison producer: ```json { "name": "compare_application_versions", "arguments": { "left": { "kind": "retained-evidence", "evidence_id": "<the analysis Evidence ID>" }, "right": { "kind": "retained-evidence", "evidence_id": "<the same Evidence ID>" } } } ``` A complete real Notion ASAR workflow encounters the same error after successful analysis and retained summary/module views. Application code was parsed as text, never executed. Following tracing remains usable. Root cause: `compareApplicationVersionsInputSchema` already requires distinct application Evidence and unique native observations on each side. The MCP reference-aware input path does not preserve those refinements and calls the validated application entry point. With identical inputs, producer provenance deduplicates to one ID, then the ordinary two-source output schema rejects it. Expected behavior: - Apply the existing distinct-input and per-side unique-native-observation rules consistently to inline, retained and mixed MCP representations. - Reject invalid input before computation/publication with `invalid_request`, the affected field and actionable reason, matching CLI behavior. - Keep valid distinct-input comparison, full provenance, coverage and unknowns unchanged. - Exercise real producer data through CLI and SDK stdio MCP, including malformed requests followed by valid comparison/tracing and a working connection. This is an input-validation/CLI–MCP parity defect, not an added self-comparison capability or a transport framing defect. The current graph contract also excludes empty graphs; that initially suspected adjacent case is outside the valid input domain. Related tracking: #1065.

bsakaguchi

2026-10-15 18:25:21 +00:00

closed this issue
added the
area: cli

area: mcp

triage

area: domain

bug

P2: medium

area: tests
labels

bsakaguchi commented

2026-10-15 18:25:21 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 9, 2026):

TL;DR: The released 6.2.0 has this bug too. Comparing a record with itself fails there as well. But 6.2.0 calls it a bad input you can retry. Its details point at the same evidence_links minimum of 2.

I tested the published rea-agents@6.2.0 over stdio MCP. I used two tiny inert JS folders, A and B, each with one package.json and one index.js.

`compare_application_versions` call	6.2.0 result
A vs A, both `retained-evidence` with the same ID	`invalid_request`
A vs A, both inline (the same Evidence record)	`invalid_request`
A vs B, retained	ok, 2 `evidence_links`
A vs B, inline	ok, 2 `evidence_links`

So 6.2.0 blames the caller's input and says a retry may help. But the tool's input schema has no items or top-level evidence_links. Those paths are output fields, so a retry can't fix it. That fits your root cause. On main you saw execution_failure / unreadable_output instead. So the error class changed after 6.2.0; I didn't look for the commit that changed it.

Raw output

Both self-comparisons return this (issues for items 1–5 trimmed):

{"code": "invalid_request", "category": "invalid_input", "retryable": true,
 "remediation": {"action": "Correct the listed arguments and retry."},
 "details": {"operation": "compare_application_versions", "issues": [
  {"path": ["items", 0, "evidence_links"], "reason": "out_of_range", "minimum": 2},
  {"path": ["items", 6, "evidence_links"], "reason": "out_of_range", "minimum": 2},
  {"path": ["evidence_links"], "reason": "out_of_range", "minimum": 2}]}}

Test setup

Box: Linux
Node: 22.22.2

🤖 Generated with Claude Code (Opus 5.5) 🦊

@CassieFox23 commented on GitHub (Oct 9, 2026): **TL;DR:** The released 6.2.0 has this bug too. Comparing a record with itself fails there as well. But 6.2.0 calls it a bad input you can retry. Its details point at the same `evidence_links` minimum of 2. I tested the published `rea-agents@6.2.0` over stdio MCP. I used two tiny inert JS folders, A and B, each with one `package.json` and one `index.js`. | `compare_application_versions` call | 6.2.0 result | |---|---| | A vs A, both `retained-evidence` with the same ID | `invalid_request` | | A vs A, both inline (the same Evidence record) | `invalid_request` | | A vs B, retained | ok, 2 `evidence_links` | | A vs B, inline | ok, 2 `evidence_links` | So 6.2.0 blames the caller's input and says a retry may help. But the tool's input schema has no `items` or top-level `evidence_links`. Those paths are output fields, so a retry can't fix it. That fits your root cause. On main you saw `execution_failure / unreadable_output` instead. So the error class changed after 6.2.0; I didn't look for the commit that changed it. <details> <summary>Raw output</summary> Both self-comparisons return this (issues for items 1–5 trimmed): ```json {"code": "invalid_request", "category": "invalid_input", "retryable": true, "remediation": {"action": "Correct the listed arguments and retry."}, "details": {"operation": "compare_application_versions", "issues": [ {"path": ["items", 0, "evidence_links"], "reason": "out_of_range", "minimum": 2}, {"path": ["items", 6, "evidence_links"], "reason": "out_of_range", "minimum": 2}, {"path": ["evidence_links"], "reason": "out_of_range", "minimum": 2}]}} ``` </details> <details> <summary>Test setup</summary> - Box: Linux - Node: 22.22.2 </details> 🤖 Generated with Claude Code (Opus 5.5) 🦊

bsakaguchi referenced this issue

2026-10-16 21:13:25 +00:00

[PR #473] [MERGED] refactor(domain): encode comparison and lifecycle states exactly #2019

bsakaguchi referenced this issue

2026-10-16 21:13:27 +00:00

[PR #474] [MERGED] refactor(test): keep coverage at behavioral boundaries #2022