morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1261] [Bug] Error results carry a non-conforming structuredContent, so strict MCP clients discard every REA error #455

New issue

Closed

opened 2026-10-15 18:25:12 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:25:12 +00:00

Owner

Originally created by @liuying-cell on GitHub (Oct 9, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1261

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

Every tool advertises an outputSchema that describes only the success envelope, but on a tool execution error REA returns a payload of a different shape in structuredContent.

	evidence-family tools	lifecycle-family tools
advertised `required`	`["result","evidence_id","evidence"]`	`["result"]`
advertised `additionalProperties`	`false`	`false`
advertised root `anyOf`	absent	absent
actual error `structuredContent`	`{"error":{...}}`	`{"error":{...}}`

The error object is {"code","category","message","retryable","remediation","details"} and arrives with isError: true.

Because additionalProperties is false and the success keys are required, every error result fails its own advertised schema. A client that validates structured results against the advertised schema rejects the whole result and surfaces a schema-validation complaint instead of the actual error. The real error is still reachable by parsing content[0].text — where REA already duplicates the same JSON — but the structuredContent path is unusable on strict clients.

The schema and the returned payload come from the same tools/list / tools/call connection, so this is not a client-side stale-catalog problem. Verified on four tools spanning both schema families; all four fail:

tool	advertised `required`	`additionalProperties`	error keys	verdict
`analyze_javascript_application`	`result, evidence_id, evidence`	`false`	`error`	FAIL
`inspect_artifact`	(evidence family)	`false`	`error`	FAIL
`list_documents`	`result, evidence_id, evidence`	`false`	`error`	FAIL
`open_binary`	`result`	`false`	`error`	FAIL

Reproduction

A stdio probe that touches no client-side validation at all — it compares the tool result against the outputSchema REA itself advertises in tools/list:

// repro.mjs   node repro.mjs rea-agents@6.1.0 list_documents '{}'
import { spawn } from 'node:child_process';

const [pkg = 'rea-agents@6.1.0', tool = 'list_documents', raw = '{}'] = process.argv.slice(2);
const child = spawn('npx', ['-y', pkg, 'mcp'], { stdio: ['pipe', 'pipe', 'inherit'], shell: true });

let buf = '', id = 0;
const waiting = new Map();
child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => {
  buf += chunk;
  let i;
  while ((i = buf.indexOf('\n')) >= 0) {
    const line = buf.slice(0, i).trim();
    buf = buf.slice(i + 1);
    if (!line) continue;
    let m;
    try { m = JSON.parse(line); } catch { continue; }
    if (m.id !== undefined && waiting.has(m.id)) { waiting.get(m.id)(m); waiting.delete(m.id); }
  }
});

const rpc = (method, params, notify) => new Promise((resolve) => {
  const msg = { jsonrpc: '2.0', method, ...(params !== undefined ? { params } : {}) };
  if (notify) { child.stdin.write(JSON.stringify(msg) + '\n'); return resolve(null); }
  msg.id = ++id;
  waiting.set(msg.id, resolve);
  child.stdin.write(JSON.stringify(msg) + '\n');
});

await rpc('initialize', {
  protocolVersion: '2025-11-25',
  capabilities: {},
  clientInfo: { name: 'probe', version: '1.0.0' },
});
await rpc('notifications/initialized', undefined, true);

const list = await rpc('tools/list', {});
const def = list.result.tools.find((t) => t.name === tool);
const schema = def.outputSchema;
console.log('advertised outputSchema =', JSON.stringify({
  required: schema.required,
  properties: Object.keys(schema.properties),
  additionalProperties: schema.additionalProperties,
  hasAnyOf: Boolean(schema.anyOf),
}));

const call = await rpc('tools/call', { name: tool, arguments: JSON.parse(raw) });
const res = call.result;
console.log('isError           =', res.isError);
console.log('structuredContent =', JSON.stringify(res.structuredContent));

const props = Object.keys(schema.properties ?? {});
const extra = Object.keys(res.structuredContent ?? {}).filter((k) => !props.includes(k));
const missing = (schema.required ?? []).filter((k) => !(k in (res.structuredContent ?? {})));
console.log('extra (not in properties) =', extra.join(', ') || '(none)');
console.log('missing required          =', missing.join(', ') || '(none)');
console.log(extra.length || missing.length ? '>>> FAIL' : '>>> PASS');

child.kill();
process.exit(0);

Observed:

$ node repro.mjs rea-agents@6.1.0 list_documents '{}'
advertised outputSchema = {"required":["result","evidence_id","evidence"],
                           "properties":["result","evidence_id","evidence"],
                           "additionalProperties":false,"hasAnyOf":false}
isError           = true
structuredContent = {"error":{"code":"target_unavailable","category":"unavailable",
                     "message":"No app is open. Ask the user which app to investigate, then call
                     open_binary with its local path.","retryable":false,
                     "remediation":{"action":"No app is open. …"}}}
extra (not in properties) = error
missing required          = result, evidence_id, evidence
>>> FAIL

$ node repro.mjs rea-agents@6.1.0 open_binary '{"path":"D:/__definitely_missing__.bin"}'
advertised outputSchema = {"required":["result"],"properties":["result"],
                           "additionalProperties":false,"hasAnyOf":false}
isError           = true
structuredContent = {"error":{"code":"target_unavailable","category":"unavailable",
                     "message":"Cannot open artifact: target does not exist …","retryable":false}}
extra (not in properties) = error
missing required          = result
>>> FAIL

Client-side confirmation: calling any of the above from a client that validates structuredContent against outputSchema produces only

Structured content does not match the tool's output schema

with the actual code, message, and remediation discarded.

Expected behavior

Per MCP Tools → Output Schema: "If an output schema is provided: Servers MUST provide structured results that conform to this schema. Clients SHOULD validate structured results against this schema."

Two representations would conform:

Omit structuredContent on error results, returning content (the JSON text, already present) plus isError: true. This is exactly what the specification's own tool execution error example does, and it loses nothing.
Advertise the full surface — make the advertised outputSchema cover both branches (success envelope ∪ {error: analysisErrorProjectionSchema}). Care is needed because a root anyOf is rejected by some clients, which is why flattenRootUnion exists for input schemas (see #1052).

The current combination — structuredContent present but not conforming — is the one option the spec does not allow.

Impact and scope

Scope: every tool. All 138 publish an outputSchema; four were verified across both schema families, all failing. The error payload shape is uniform.
What is not affected: Evidence integrity, the CLI surface, and the content[0].text channel. Nothing is lost from REA's own record — this is a representation mismatch, not data loss.
Compatibility: any client implementing the "SHOULD validate" guidance hides all REA errors behind one indistinguishable message. A caller cannot tell "you must call open_binary first" from "your analysis finished, but the response exceeded the budget — export the retained evidence".
Concrete harm: the second case is the one addressed in #1053 / #1050. REA correctly reports resource_constraint with the message "The operation completed, but its complete MCP response exceeds the stdio response budget… export retained evidence to consume it". On a strict client the caller is told the call failed and never learns that (a) the analysis actually completed, or (b) export_evidence_bundle is the recovery — so completed work appears lost. This is the failure mode I hit while analyzing an Electron ASAR.
Recovery/next tool: currently unreachable through the structured channel; the caller must string-parse content[0].text.
Reproduced identically on rea-agents@6.0.0, so this is not a recent regression.

REA version

6.1.0

Runtime and platform

Node v22.22.2; Windows 11 x64; stdio MCP, protocolVersion: 2025-11-25; no decompiler provider installed

Target and artifact context

obsidian.asar, 23,342,614 bytes (Obsidian 1.8.9 desktop, Electron) — used to reproduce the resource_constraint case; default_app.asar (109 KB, Electron stock skeleton); and a deliberately non-existent path for open_binary

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

Error payload as it arrives today (verbatim, structuredContent):

{
  "error": {
    "code": "resource_constraint",
    "category": "resource_constraint",
    "message": "The operation completed, but its complete MCP response exceeds the stdio response budget. The analysis result was not truncated; export retained evidence to consume it, or use complete CLI JSON output.",
    "retryable": false,
    "remediation": {
      "action": "Export retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. Tools that accept retained-evidence references can inspect compatible records. The connection remains usable."
    },
    "details": {
      "operation": "analyze_javascript_application",
      "resource": "transport",
      "reason": "The operation completed, but its complete MCP response exceeds the stdio response budget. …"
    }
  }
}

The same JSON is present in content[0].text.

Typed outcome or recovery expected

A caller should receive either (a) no structuredContent plus isError: true, or (b) a structuredContent that validates against the advertised outputSchema — so the typed code, remediation, and details survive strict clients. resource_constraint in particular should remain actionable, since it signals completed work awaiting export, not a failure.

Analysis or suspected cause

Source-derived hypothesis (not verified against maintainer intent):

dist/contracts/toolSchemaMetadata.js L46 — toolOutputSchemaWithMetadata advertises contract.outputSchema unchanged; the output projection applies no advertise transform, so nothing reconciles the two envelopes.
dist/contracts/toolOutputSchemaPrimitives.js L13–17 — evidenceResultOf is z.strictObject({ result, evidence_id, evidence }); L20 — lifecycleResultOf is z.object({ result }). Zod's output projection emits additionalProperties: false for both.
dist/contracts/errorSchemas.js L31 — analysisErrorProjectionSchema, a complete z.discriminatedUnion("code", …) documented as "shared by every CLI and MCP error surface", already exists; it is simply never unioned into (or advertised alongside) the tool's outputSchema. The success and error surfaces therefore advertise one shape and return two.

An alternative reading is that clients are expected to skip validation when isError is true. The specification states no such exemption, and its own tool-execution-error example omits structuredContent entirely — so under either reading, emitting a present-but-non-conforming structuredContent is the one outcome the spec rules out. Which of the two fixes is intended is a maintainer call; reporting the mismatch is the point of this issue.

Additional context

Related issues reviewed before filing — none covers this:

#100 (Complete actionable output schemas, structured results, and typed error details) — concerns error detail fidelity (which fields survive redaction). The shipped list confirms "Tool registrations publish outputSchema" and "Successful values are validated before return"; the advertised schema's coverage of the error branch is not addressed.
#1052 (Three MCP tool input schemas use a root anyOf…) — the input-side root-union problem. This report is the output-side analogue and is not fixed by flattenRootUnion, which only runs on inputs.
#1053 / #1050 — large-result transport and caller-selected views. #1053 is the resource_constraint behavior referenced above; this report does not dispute that fix, only the visibility of its error envelope on strict clients.
#746 — Obsidian 1.12.7 24 MB ASAR (this run used Obsidian 1.8.9 and completed successfully, exit 0).
#723 (open) — design discussion on result/schema representation; noted because its boundary clause already requires "validate schemas after SDK conversion".

A regression test that would have caught this: for every registered tool, assert that a synthetic error result placed in structuredContent validates against the outputSchema advertised for that tool in tools/list.

Originally created by @liuying-cell on GitHub (Oct 9, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1261 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area MCP or agent integration ### Observed behavior Every tool advertises an `outputSchema` that describes **only the success envelope**, but on a tool execution error REA returns a payload of a different shape in `structuredContent`. | | evidence-family tools | lifecycle-family tools | |---|---|---| | advertised `required` | `["result","evidence_id","evidence"]` | `["result"]` | | advertised `additionalProperties` | `false` | `false` | | advertised root `anyOf` | absent | absent | | actual error `structuredContent` | `{"error":{...}}` | `{"error":{...}}` | The error object is `{"code","category","message","retryable","remediation","details"}` and arrives with `isError: true`. Because `additionalProperties` is `false` and the success keys are `required`, **every error result fails its own advertised schema**. A client that validates structured results against the advertised schema rejects the whole result and surfaces a schema-validation complaint instead of the actual error. The real error is still reachable by parsing `content[0].text` — where REA already duplicates the same JSON — but the `structuredContent` path is unusable on strict clients. The schema and the returned payload come from the same `tools/list` / `tools/call` connection, so this is not a client-side stale-catalog problem. Verified on four tools spanning both schema families; all four fail: | tool | advertised `required` | `additionalProperties` | error keys | verdict | |---|---|---|---|---| | `analyze_javascript_application` | `result, evidence_id, evidence` | `false` | `error` | FAIL | | `inspect_artifact` | (evidence family) | `false` | `error` | FAIL | | `list_documents` | `result, evidence_id, evidence` | `false` | `error` | FAIL | | `open_binary` | `result` | `false` | `error` | FAIL | ### Reproduction A stdio probe that touches no client-side validation at all — it compares the tool result against the `outputSchema` REA itself advertises in `tools/list`: ```js // repro.mjs node repro.mjs rea-agents@6.1.0 list_documents '{}' import { spawn } from 'node:child_process'; const [pkg = 'rea-agents@6.1.0', tool = 'list_documents', raw = '{}'] = process.argv.slice(2); const child = spawn('npx', ['-y', pkg, 'mcp'], { stdio: ['pipe', 'pipe', 'inherit'], shell: true }); let buf = '', id = 0; const waiting = new Map(); child.stdout.setEncoding('utf8'); child.stdout.on('data', (chunk) => { buf += chunk; let i; while ((i = buf.indexOf('\n')) >= 0) { const line = buf.slice(0, i).trim(); buf = buf.slice(i + 1); if (!line) continue; let m; try { m = JSON.parse(line); } catch { continue; } if (m.id !== undefined && waiting.has(m.id)) { waiting.get(m.id)(m); waiting.delete(m.id); } } }); const rpc = (method, params, notify) => new Promise((resolve) => { const msg = { jsonrpc: '2.0', method, ...(params !== undefined ? { params } : {}) }; if (notify) { child.stdin.write(JSON.stringify(msg) + '\n'); return resolve(null); } msg.id = ++id; waiting.set(msg.id, resolve); child.stdin.write(JSON.stringify(msg) + '\n'); }); await rpc('initialize', { protocolVersion: '2025-11-25', capabilities: {}, clientInfo: { name: 'probe', version: '1.0.0' }, }); await rpc('notifications/initialized', undefined, true); const list = await rpc('tools/list', {}); const def = list.result.tools.find((t) => t.name === tool); const schema = def.outputSchema; console.log('advertised outputSchema =', JSON.stringify({ required: schema.required, properties: Object.keys(schema.properties), additionalProperties: schema.additionalProperties, hasAnyOf: Boolean(schema.anyOf), })); const call = await rpc('tools/call', { name: tool, arguments: JSON.parse(raw) }); const res = call.result; console.log('isError =', res.isError); console.log('structuredContent =', JSON.stringify(res.structuredContent)); const props = Object.keys(schema.properties ?? {}); const extra = Object.keys(res.structuredContent ?? {}).filter((k) => !props.includes(k)); const missing = (schema.required ?? []).filter((k) => !(k in (res.structuredContent ?? {}))); console.log('extra (not in properties) =', extra.join(', ') || '(none)'); console.log('missing required =', missing.join(', ') || '(none)'); console.log(extra.length || missing.length ? '>>> FAIL' : '>>> PASS'); child.kill(); process.exit(0); ``` Observed: ``` $ node repro.mjs rea-agents@6.1.0 list_documents '{}' advertised outputSchema = {"required":["result","evidence_id","evidence"], "properties":["result","evidence_id","evidence"], "additionalProperties":false,"hasAnyOf":false} isError = true structuredContent = {"error":{"code":"target_unavailable","category":"unavailable", "message":"No app is open. Ask the user which app to investigate, then call open_binary with its local path.","retryable":false, "remediation":{"action":"No app is open. …"}}} extra (not in properties) = error missing required = result, evidence_id, evidence >>> FAIL $ node repro.mjs rea-agents@6.1.0 open_binary '{"path":"D:/__definitely_missing__.bin"}' advertised outputSchema = {"required":["result"],"properties":["result"], "additionalProperties":false,"hasAnyOf":false} isError = true structuredContent = {"error":{"code":"target_unavailable","category":"unavailable", "message":"Cannot open artifact: target does not exist …","retryable":false}} extra (not in properties) = error missing required = result >>> FAIL ``` Client-side confirmation: calling any of the above from a client that validates `structuredContent` against `outputSchema` produces only ``` Structured content does not match the tool's output schema ``` with the actual `code`, `message`, and `remediation` discarded. ### Expected behavior Per MCP Tools → *Output Schema*: *"If an output schema is provided: Servers **MUST** provide structured results that conform to this schema. Clients **SHOULD** validate structured results against this schema."* Two representations would conform: 1. **Omit `structuredContent` on error results**, returning `content` (the JSON text, already present) plus `isError: true`. This is exactly what the specification's own *tool execution error* example does, and it loses nothing. 2. **Advertise the full surface** — make the advertised `outputSchema` cover both branches (success envelope ∪ `{error: analysisErrorProjectionSchema}`). Care is needed because a root `anyOf` is rejected by some clients, which is why `flattenRootUnion` exists for input schemas (see #1052). The current combination — `structuredContent` present but not conforming — is the one option the spec does not allow. ### Impact and scope - **Scope:** every tool. All 138 publish an `outputSchema`; four were verified across both schema families, all failing. The error payload shape is uniform. - **What is not affected:** Evidence integrity, the CLI surface, and the `content[0].text` channel. Nothing is lost from REA's own record — this is a representation mismatch, not data loss. - **Compatibility:** any client implementing the "SHOULD validate" guidance hides **all** REA errors behind one indistinguishable message. A caller cannot tell *"you must call `open_binary` first"* from *"your analysis finished, but the response exceeded the budget — export the retained evidence"*. - **Concrete harm:** the second case is the one addressed in #1053 / #1050. REA correctly reports `resource_constraint` with the message *"The operation completed, but its complete MCP response exceeds the stdio response budget… export retained evidence to consume it"*. On a strict client the caller is told the call failed and never learns that (a) the analysis actually completed, or (b) `export_evidence_bundle` is the recovery — so completed work appears lost. This is the failure mode I hit while analyzing an Electron ASAR. - **Recovery/next tool:** currently unreachable through the structured channel; the caller must string-parse `content[0].text`. - Reproduced identically on `rea-agents@6.0.0`, so this is not a recent regression. ### REA version 6.1.0 ### Runtime and platform Node v22.22.2; Windows 11 x64; stdio MCP, `protocolVersion: 2025-11-25`; no decompiler provider installed ### Target and artifact context `obsidian.asar`, 23,342,614 bytes (Obsidian 1.8.9 desktop, Electron) — used to reproduce the `resource_constraint` case; `default_app.asar` (109 KB, Electron stock skeleton); and a deliberately non-existent path for `open_binary` ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence Error payload as it arrives today (verbatim, `structuredContent`): ```json { "error": { "code": "resource_constraint", "category": "resource_constraint", "message": "The operation completed, but its complete MCP response exceeds the stdio response budget. The analysis result was not truncated; export retained evidence to consume it, or use complete CLI JSON output.", "retryable": false, "remediation": { "action": "Export retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. Tools that accept retained-evidence references can inspect compatible records. The connection remains usable." }, "details": { "operation": "analyze_javascript_application", "resource": "transport", "reason": "The operation completed, but its complete MCP response exceeds the stdio response budget. …" } } } ``` The same JSON is present in `content[0].text`. ### Typed outcome or recovery expected A caller should receive either (a) no `structuredContent` plus `isError: true`, or (b) a `structuredContent` that validates against the advertised `outputSchema` — so the typed `code`, `remediation`, and `details` survive strict clients. `resource_constraint` in particular should remain actionable, since it signals *completed work awaiting export*, not a failure. ### Analysis or suspected cause Source-derived hypothesis (not verified against maintainer intent): - `dist/contracts/toolSchemaMetadata.js` L46 — `toolOutputSchemaWithMetadata` advertises `contract.outputSchema` unchanged; the `output` projection applies no `advertise` transform, so nothing reconciles the two envelopes. - `dist/contracts/toolOutputSchemaPrimitives.js` L13–17 — `evidenceResultOf` is `z.strictObject({ result, evidence_id, evidence })`; L20 — `lifecycleResultOf` is `z.object({ result })`. Zod's **output** projection emits `additionalProperties: false` for both. - `dist/contracts/errorSchemas.js` L31 — `analysisErrorProjectionSchema`, a complete `z.discriminatedUnion("code", …)` documented as *"shared by every CLI and MCP error surface"*, already exists; it is simply never unioned into (or advertised alongside) the tool's `outputSchema`. The success and error surfaces therefore advertise one shape and return two. An alternative reading is that clients are expected to skip validation when `isError` is true. The specification states no such exemption, and its own tool-execution-error example omits `structuredContent` entirely — so under either reading, emitting a present-but-non-conforming `structuredContent` is the one outcome the spec rules out. Which of the two fixes is intended is a maintainer call; reporting the mismatch is the point of this issue. ### Additional context Related issues reviewed before filing — none covers this: - **#100** (*Complete actionable output schemas, structured results, and typed error details*) — concerns error *detail fidelity* (which fields survive redaction). The shipped list confirms "Tool registrations publish outputSchema" and "Successful values are validated before return"; the advertised schema's coverage of the *error* branch is not addressed. - **#1052** (*Three MCP tool input schemas use a root anyOf…*) — the **input**-side root-union problem. This report is the output-side analogue and is not fixed by `flattenRootUnion`, which only runs on inputs. - **#1053** / **#1050** — large-result transport and caller-selected views. #1053 is the `resource_constraint` behavior referenced above; this report does not dispute that fix, only the visibility of its error *envelope* on strict clients. - **#746** — Obsidian 1.12.7 24 MB ASAR (this run used Obsidian 1.8.9 and completed successfully, exit 0). - **#723** (open) — design discussion on result/schema representation; noted because its boundary clause already requires "validate schemas after SDK conversion". A regression test that would have caught this: for every registered tool, assert that a synthetic error result placed in `structuredContent` validates against the `outputSchema` advertised for that tool in `tools/list`.

bsakaguchi

2026-10-15 18:25:12 +00:00

closed this issue
added the
area: mcp

bug
labels

bsakaguchi referenced this issue

2026-10-16 21:13:15 +00:00

[PR #455] [MERGED] chore(main): release rea-agents 3.0.0 #2002

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#455

No description provided.

Rows
Columns