morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4639] [Bug] MCP rejects authentic native Evidence that CLI handles as unsupported #2420

New issue

Closed

opened 2026-10-17 21:04:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:04:23 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4639

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

With fresh authentic managed-boundary and native inspect_signature Evidence, MCP verify_managed_native_boundaries returns evidence_integrity_mismatch and tells the caller to recreate or re-import the records. The CLI accepts the identical input and returns partial verification Evidence: one unresolved P/Invoke, one unsupported observation, retained input Evidence links, and an explicit limitation. The records authenticate and pass the shared input schema.

Reproduction

Use macOS with the Xcode Command Line Tools. From a checkout of the source revision below, use the pinned Node/npm toolchain, run npm ci, then npm run build:cached. No deep-analysis provider is required.
Run the following from the checkout root. It creates a 4096-byte managed PE with one P/Invoke using the repository fixture builder, obtains authentic Evidence through the ordinary stdio MCP server, and submits the same input to both adapters. Child CLI/MCP processes use a fresh HOME and a separate temporary directory; no client configuration override variables are forwarded. The managed assembly is never executed.

Executable reproduction

node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import { mkdtemp, mkdir, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { spawnSync } from 'node:child_process';
import { Client } from '@modelcontextprotocol/client';
import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
import { buildManagedPeFixture } from './scripts/fixtures/managed/pe.mjs';
const runtime = process.env.REA_TEST_ROOT ?? process.cwd();
const { parseEvidence } = await import(pathToFileURL(join(runtime, 'dist/domain/evidence.js')));
const { managedNativeVerificationInputSchema } = await import(pathToFileURL(join(runtime, 'dist/domain/managed/managedNativeVerificationSchemas.js')));
const work = await mkdtemp(join(process.env.REA_TEST_WORK ?? tmpdir(), 'rea-managed-parity-'));
const env = { PATH: process.env.PATH, HOME: join(work, 'home'), TMPDIR: work, REA_LOG_LEVEL: 'silent' };
await mkdir(env.HOME);
const path = join(work, 'fixture.dll');
await writeFile(path, buildManagedPeFixture({
  methods: [{ name: 'NativeMessageBox', flags: 0x2016, rva: 0 }],
  pinvoke: { moduleName: 'user32.dll', importName: 'MessageBoxW', mappingFlags: 0x0345, memberRow: 1 },
}));
const client = new Client({ name: 'managed-parity-check', version: '1' });
const transport = new StdioClientTransport({
  command: process.execPath, args: [join(runtime, 'scripts/rea.mjs'), 'mcp'], env, stderr: 'pipe',
});
transport.stderr?.resume();
const call = (name, args = {}) => client.callTool({ name, arguments: args });
const evidence = response => {
  assert.notEqual(response.isError, true);
  return parseEvidence(response.structuredContent);
};
async function cli(input, label) {
  const file = join(work, `${label}.json`);
  await writeFile(file, JSON.stringify(input));
  const result = spawnSync(process.execPath,
    [join(runtime, 'scripts/rea.mjs'), 'verify-managed-native-boundaries', file, '--json'],
    { env, encoding: 'utf8', timeout: 60000, maxBuffer: 16777216 });
  assert.equal(result.status, 0, result.stderr);
  await writeFile(join(work, `${label}-cli.json`), result.stdout);
  return JSON.parse(result.stdout);
}
try {
  await client.connect(transport);
  const managed = evidence(await call('inspect_managed_native_boundaries', { path }));
  assert.equal(managed.normalized_result.pinvoke_imports.length, 1);
  assert.notEqual((await call('open_binary', { path: '/usr/bin/true' })).isError, true);
  const native = evidence(await call('inspect_signature'));
  const input = { managed_boundaries: managed, native_observations: [native] };
  managedNativeVerificationInputSchema.parse(input);
  for (let attempt = 1; attempt <= 2; attempt++) {
    const response = await call('verify_managed_native_boundaries', input);
    await writeFile(join(work, `attempt-${attempt}-mcp.json`), JSON.stringify(response));
    assert.equal(response.isError, true);
    const error = JSON.parse(response.content.find(item => item.type === 'text').text).error;
    assert.equal(error.code, 'evidence_integrity_mismatch');
    assert.equal(error.details.reason, 'Expected inspect_macho or analyze_function Evidence, received inspect_signature');
    const cliEvidence = await cli(input, `attempt-${attempt}`);
    const result = cliEvidence.normalized_result;
    assert.deepEqual(new Set(result.evidence_links), new Set([managed.evidence_id, native.evidence_id]));
    assert.equal(result.summary.unresolved, 1);
    assert.deepEqual(result.native_observations, { total: 1, accepted: 0, unsupported: 1, symbols: 0 });
    assert.equal(result.pinvoke_imports[0].basis, 'unsupported-native-evidence');
    assert.equal(result.coverage.status, 'partial');
    assert.ok(result.limitations.includes('Some native Evidence operations were unsupported by this workflow.'));
    console.log(JSON.stringify({ attempt, mcp: error.code, reason: error.details.reason, cli: result.native_observations, unresolved: result.summary.unresolved, coverage: result.coverage.status }));
  }
  const control = { managed_boundaries: managed, native_observations: [evidence(await call('inspect_macho'))] };
  const mcpControl = evidence(await call('verify_managed_native_boundaries', control));
  const cliControl = await cli(control, 'control');
  assert.deepEqual(mcpControl.normalized_result, cliControl.normalized_result);
  assert.equal(cliControl.normalized_result.native_observations.accepted, 1);
  console.log('inspect_macho control: both adapters succeed with equal normalized results');
} finally {
  await client.close();
  await transport.close();
}
JS

Both signature-Evidence attempts fail over MCP and succeed over CLI. Replacing only the native observation with authentic inspect_macho Evidence succeeds over both adapters with equal normalized results.

I ran this on both tested builds. For the installed release, I installed the package in a separate temporary prefix and set REA_TEST_ROOT to that installation's node_modules/rea-agents directory before running the same script from the source checkout. Each runtime generated its own fresh Evidence. There were two failing MCP attempts and two successful CLI attempts per build; the supported-operation control passed once per adapter per build.

Expected behavior

Both adapters should preserve the shared workflow's explicit handling of authentic but unsupported observations: count the unsupported operation, retain the input Evidence links, report partial coverage, and leave the declaration unresolved. REA's repository contract requires shared CLI/MCP workflows to preserve Evidence provenance and session semantics through both adapters. This does not require code-signature Evidence to supply export names or establish a native binding.

Impact and scope

An MCP caller loses the partial-verification result and receives misleading record-recreation advice. The observed scope is one P/Invoke fixture with inspect_signature Evidence on macOS. Other unsupported operations and platforms were not exercised. The CLI returned the intended partial result from the same records; the inspect_macho control succeeded through both adapters.

Impact level: P2
Assessment status: supported
Impact basis: A bounded MCP workflow failure discards the shared workflow's useful partial result; no evidence corruption, target mutation, or false binding claim was observed.
Workaround status: available
Workaround basis: The CLI consumed the same authenticated records and retained the unresolved declaration, unsupported count, both input Evidence links, and limitation.

REA version

Current main source commit 71929ed37cdc560b6b8aa62b5f7182e417c8a850 (reports 6.5.0) and published npm rea-agents@6.5.0, confirmed as the latest npm release before testing.

Runtime and platform

Node.js v24.18.0; npm 11.16.0; macOS 27.0.1; Darwin arm64. Native inspection used macOS utilities. No Hopper, Ghidra, or IDA analysis ran.

Target and artifact context

Repository-generated managed PE/CLI fixture with one NativeMessageBox method and a user32.dll!MessageBoxW P/Invoke. Native signature Evidence and the Mach-O control came from static inspection of /usr/bin/true.

Interface

MCP

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Expected source: AGENTS.md:5; src/domain/managed/managedNativeVerificationSchemas.ts:99-128; src/domain/managed/managedNativeVerificationMatch.ts:80-100,379-383,419-434 in the tested source build.
Failure source: Executed CLI/MCP reproduction above on both builds; src/server/registerManagedWorkflowTools/evidence.ts:14-26,46-49; src/server/registerManagedWorkflowTools/verifyManagedNativeBoundaries.ts:43-50; CLI path at src/cli/managedCommands.ts:205-211.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Both builds returned the same meaningful results:

{"mcp":{"isError":true,"code":"evidence_integrity_mismatch","message":"Evidence is invalid or has changed. Recreate or re-import it, then try again.","reason":"Expected inspect_macho or analyze_function Evidence, received inspect_signature"}}
{"cli":{"operation":"verify_managed_native_boundaries","native_observations":{"total":1,"accepted":0,"unsupported":1,"symbols":0},"unresolved":1,"pinvoke_basis":"unsupported-native-evidence","coverage":"partial","limitation":"Some native Evidence operations were unsupported by this workflow."}}

The script asserts CLI exit status 0, both input Evidence IDs retained in the result, and successful inspect_macho controls with equal normalized results and one accepted native observation.

Typed outcome or recovery expected

For this input, both adapters should deliver one unsupported observation, one unresolved P/Invoke with unsupported-native-evidence, partial coverage, retained source links, and the unsupported-operation limitation. Keep the supported-operation control successful and retain rejection of actually invalid Evidence. An unsupported operation alone should not be classified as damaged Evidence.

Analysis or suspected cause

Source-derived reasoning: the MCP handler applies an inspect_macho/analyze_function operation allowlist before calling the shared application service; its resolver wraps another authentic operation in an integrity error. The CLI calls the shared service directly and reaches the domain's explicit unsupported-operation accounting. Schema acceptance alone does not promise symbol extraction; the shared fallback and the observed CLI result establish the inconsistency here.

Additional context

Related #4057 requests a new PE/ELF export inventory. This report concerns inconsistent fallback and error classification for an existing authentic operation, rather than another symbol extractor. Earlier PR #2901 shares duplicate-Evidence-ID diagnostics; its change does not remove this operation allowlist. Open/closed issue and PR searches found no matching report. No last-known-good version was established.

Originally created by @coygeek on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4639 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area MCP or agent integration ### Observed behavior With fresh authentic managed-boundary and native `inspect_signature` Evidence, MCP `verify_managed_native_boundaries` returns `evidence_integrity_mismatch` and tells the caller to recreate or re-import the records. The CLI accepts the identical input and returns partial verification Evidence: one unresolved P/Invoke, one unsupported observation, retained input Evidence links, and an explicit limitation. The records authenticate and pass the shared input schema. ### Reproduction 1. Use macOS with the Xcode Command Line Tools. From a checkout of the source revision below, use the pinned Node/npm toolchain, run `npm ci`, then `npm run build:cached`. No deep-analysis provider is required. 2. Run the following from the checkout root. It creates a 4096-byte managed PE with one P/Invoke using the repository fixture builder, obtains authentic Evidence through the ordinary stdio MCP server, and submits the same input to both adapters. Child CLI/MCP processes use a fresh HOME and a separate temporary directory; no client configuration override variables are forwarded. The managed assembly is never executed. <details> <summary>Executable reproduction</summary> ```sh node --input-type=module <<'JS' import assert from 'node:assert/strict'; import { mkdtemp, mkdir, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { pathToFileURL } from 'node:url'; import { spawnSync } from 'node:child_process'; import { Client } from '@modelcontextprotocol/client'; import { StdioClientTransport } from '@modelcontextprotocol/client/stdio'; import { buildManagedPeFixture } from './scripts/fixtures/managed/pe.mjs'; const runtime = process.env.REA_TEST_ROOT ?? process.cwd(); const { parseEvidence } = await import(pathToFileURL(join(runtime, 'dist/domain/evidence.js'))); const { managedNativeVerificationInputSchema } = await import(pathToFileURL(join(runtime, 'dist/domain/managed/managedNativeVerificationSchemas.js'))); const work = await mkdtemp(join(process.env.REA_TEST_WORK ?? tmpdir(), 'rea-managed-parity-')); const env = { PATH: process.env.PATH, HOME: join(work, 'home'), TMPDIR: work, REA_LOG_LEVEL: 'silent' }; await mkdir(env.HOME); const path = join(work, 'fixture.dll'); await writeFile(path, buildManagedPeFixture({ methods: [{ name: 'NativeMessageBox', flags: 0x2016, rva: 0 }], pinvoke: { moduleName: 'user32.dll', importName: 'MessageBoxW', mappingFlags: 0x0345, memberRow: 1 }, })); const client = new Client({ name: 'managed-parity-check', version: '1' }); const transport = new StdioClientTransport({ command: process.execPath, args: [join(runtime, 'scripts/rea.mjs'), 'mcp'], env, stderr: 'pipe', }); transport.stderr?.resume(); const call = (name, args = {}) => client.callTool({ name, arguments: args }); const evidence = response => { assert.notEqual(response.isError, true); return parseEvidence(response.structuredContent); }; async function cli(input, label) { const file = join(work, `${label}.json`); await writeFile(file, JSON.stringify(input)); const result = spawnSync(process.execPath, [join(runtime, 'scripts/rea.mjs'), 'verify-managed-native-boundaries', file, '--json'], { env, encoding: 'utf8', timeout: 60000, maxBuffer: 16777216 }); assert.equal(result.status, 0, result.stderr); await writeFile(join(work, `${label}-cli.json`), result.stdout); return JSON.parse(result.stdout); } try { await client.connect(transport); const managed = evidence(await call('inspect_managed_native_boundaries', { path })); assert.equal(managed.normalized_result.pinvoke_imports.length, 1); assert.notEqual((await call('open_binary', { path: '/usr/bin/true' })).isError, true); const native = evidence(await call('inspect_signature')); const input = { managed_boundaries: managed, native_observations: [native] }; managedNativeVerificationInputSchema.parse(input); for (let attempt = 1; attempt <= 2; attempt++) { const response = await call('verify_managed_native_boundaries', input); await writeFile(join(work, `attempt-${attempt}-mcp.json`), JSON.stringify(response)); assert.equal(response.isError, true); const error = JSON.parse(response.content.find(item => item.type === 'text').text).error; assert.equal(error.code, 'evidence_integrity_mismatch'); assert.equal(error.details.reason, 'Expected inspect_macho or analyze_function Evidence, received inspect_signature'); const cliEvidence = await cli(input, `attempt-${attempt}`); const result = cliEvidence.normalized_result; assert.deepEqual(new Set(result.evidence_links), new Set([managed.evidence_id, native.evidence_id])); assert.equal(result.summary.unresolved, 1); assert.deepEqual(result.native_observations, { total: 1, accepted: 0, unsupported: 1, symbols: 0 }); assert.equal(result.pinvoke_imports[0].basis, 'unsupported-native-evidence'); assert.equal(result.coverage.status, 'partial'); assert.ok(result.limitations.includes('Some native Evidence operations were unsupported by this workflow.')); console.log(JSON.stringify({ attempt, mcp: error.code, reason: error.details.reason, cli: result.native_observations, unresolved: result.summary.unresolved, coverage: result.coverage.status })); } const control = { managed_boundaries: managed, native_observations: [evidence(await call('inspect_macho'))] }; const mcpControl = evidence(await call('verify_managed_native_boundaries', control)); const cliControl = await cli(control, 'control'); assert.deepEqual(mcpControl.normalized_result, cliControl.normalized_result); assert.equal(cliControl.normalized_result.native_observations.accepted, 1); console.log('inspect_macho control: both adapters succeed with equal normalized results'); } finally { await client.close(); await transport.close(); } JS ``` </details> 3. Both signature-Evidence attempts fail over MCP and succeed over CLI. Replacing only the native observation with authentic `inspect_macho` Evidence succeeds over both adapters with equal normalized results. I ran this on both tested builds. For the installed release, I installed the package in a separate temporary prefix and set `REA_TEST_ROOT` to that installation's `node_modules/rea-agents` directory before running the same script from the source checkout. Each runtime generated its own fresh Evidence. There were two failing MCP attempts and two successful CLI attempts per build; the supported-operation control passed once per adapter per build. ### Expected behavior Both adapters should preserve the shared workflow's explicit handling of authentic but unsupported observations: count the unsupported operation, retain the input Evidence links, report partial coverage, and leave the declaration unresolved. REA's repository contract requires shared CLI/MCP workflows to preserve Evidence provenance and session semantics through both adapters. This does not require code-signature Evidence to supply export names or establish a native binding. ### Impact and scope An MCP caller loses the partial-verification result and receives misleading record-recreation advice. The observed scope is one P/Invoke fixture with `inspect_signature` Evidence on macOS. Other unsupported operations and platforms were not exercised. The CLI returned the intended partial result from the same records; the `inspect_macho` control succeeded through both adapters. - Impact level: P2 - Assessment status: supported - Impact basis: A bounded MCP workflow failure discards the shared workflow's useful partial result; no evidence corruption, target mutation, or false binding claim was observed. - Workaround status: available - Workaround basis: The CLI consumed the same authenticated records and retained the unresolved declaration, unsupported count, both input Evidence links, and limitation. ### REA version Current `main` source commit `71929ed37cdc560b6b8aa62b5f7182e417c8a850` (reports `6.5.0`) and published npm `rea-agents@6.5.0`, confirmed as the latest npm release before testing. ### Runtime and platform Node.js v24.18.0; npm 11.16.0; macOS 27.0.1; Darwin arm64. Native inspection used macOS utilities. No Hopper, Ghidra, or IDA analysis ran. ### Target and artifact context Repository-generated managed PE/CLI fixture with one `NativeMessageBox` method and a `user32.dll!MessageBoxW` P/Invoke. Native signature Evidence and the Mach-O control came from static inspection of `/usr/bin/true`. ### Interface MCP ### Earliest failure stage Request parsing or validation ### Relevant output or evidence - Expected source: `AGENTS.md:5`; `src/domain/managed/managedNativeVerificationSchemas.ts:99-128`; `src/domain/managed/managedNativeVerificationMatch.ts:80-100,379-383,419-434` in the tested source build. - Failure source: Executed CLI/MCP reproduction above on both builds; `src/server/registerManagedWorkflowTools/evidence.ts:14-26,46-49`; `src/server/registerManagedWorkflowTools/verifyManagedNativeBoundaries.ts:43-50`; CLI path at `src/cli/managedCommands.ts:205-211`. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete Both builds returned the same meaningful results: ```json {"mcp":{"isError":true,"code":"evidence_integrity_mismatch","message":"Evidence is invalid or has changed. Recreate or re-import it, then try again.","reason":"Expected inspect_macho or analyze_function Evidence, received inspect_signature"}} {"cli":{"operation":"verify_managed_native_boundaries","native_observations":{"total":1,"accepted":0,"unsupported":1,"symbols":0},"unresolved":1,"pinvoke_basis":"unsupported-native-evidence","coverage":"partial","limitation":"Some native Evidence operations were unsupported by this workflow."}} ``` The script asserts CLI exit status 0, both input Evidence IDs retained in the result, and successful `inspect_macho` controls with equal normalized results and one accepted native observation. ### Typed outcome or recovery expected For this input, both adapters should deliver one unsupported observation, one unresolved P/Invoke with `unsupported-native-evidence`, partial coverage, retained source links, and the unsupported-operation limitation. Keep the supported-operation control successful and retain rejection of actually invalid Evidence. An unsupported operation alone should not be classified as damaged Evidence. ### Analysis or suspected cause Source-derived reasoning: the MCP handler applies an `inspect_macho`/`analyze_function` operation allowlist before calling the shared application service; its resolver wraps another authentic operation in an integrity error. The CLI calls the shared service directly and reaches the domain's explicit unsupported-operation accounting. Schema acceptance alone does not promise symbol extraction; the shared fallback and the observed CLI result establish the inconsistency here. ### Additional context Related [#4057](https://github.com/morluto/rea/issues/4057) requests a new PE/ELF export inventory. This report concerns inconsistent fallback and error classification for an existing authentic operation, rather than another symbol extractor. Earlier [PR #2901](https://github.com/morluto/rea/pull/2901) shares duplicate-Evidence-ID diagnostics; its change does not remove this operation allowlist. Open/closed issue and PR searches found no matching report. No last-known-good version was established.

bsakaguchi closed this issue

2026-10-17 21:04:23 +00:00

bsakaguchi referenced this issue

2026-10-31 00:29:41 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-31 01:04:52 +00:00

[PR #2420] [CLOSED] feat(evm): inspect lossless Solidity terminal CBOR metadata #5214

bsakaguchi referenced this issue

2026-10-31 01:06:17 +00:00

[PR #2790] [MERGED] feat(evm): inspect lossless Solidity terminal CBOR metadata #5394