morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3988] [Bug] Missing IDA callee coverage produces a definite negative call-path result #1584

New issue

Closed

opened 2026-10-16 21:07:26 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:07:26 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3988

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

When a modern headless IDA provider returns an empty callee page without a more field, REA reports complete relationship coverage. A caller that then asks whether main reaches another resolved procedure receives goal_status: "not_reached" with no residual unknowns. The documented contract requires missing relationship coverage to remain unknown. This was reproduced through the production stdio MCP server with a simulated provider and an admitted x86_64 ELF fixture.

Reproduction

Use the baseline's installed dependencies and built runtime. From the repository root, create a disposable directory with repro_dir=$(mktemp -d) and save the following as "$repro_dir/coverage-repro.mjs". The script implements the database-supervisor responses at the upstream stdio boundary and uses the repository's installed MCP client to call the production server. It isolates the server's home and provider workspace, opens the target, requests callees, traces to a different resolved goal, and closes the session. It never launches IDA or executes the target.

import { createInterface } from 'node:readline';
import { mkdtemp, writeFile, rm } from 'node:fs/promises';
import { join, resolve } from 'node:path';
import { tmpdir } from 'node:os';
import { fileURLToPath } from 'node:url';
import { createRequire } from 'node:module';

const script = fileURLToPath(import.meta.url);
if (process.argv[2] === '--provider') {
  const names = ['idb_open', 'idb_list', 'idb_close', 'server_health',
    'list_funcs', 'lookup_funcs', 'find_regex', 'decompile', 'disasm',
    'callees', 'xrefs_to'];
  let sessionId, inputPath;
  for await (const line of createInterface({ input: process.stdin })) {
    const request = JSON.parse(line);
    if (request.id === undefined) continue;
    let result;
    if (request.method === 'initialize') {
      result = { protocolVersion: request.params.protocolVersion,
        capabilities: { tools: {} }, serverInfo: { name: 'coverage-fixture', version: '1' } };
    } else if (request.method === 'tools/list') {
      result = { tools: names.map(name => ({ name, inputSchema: { type: 'object' } })) };
    } else {
      const { name, arguments: args } = request.params;
      let data;
      switch (name) {
        case 'idb_open':
          sessionId = args.preferred_session_id;
          inputPath = args.input_path;
          data = { success: true, session: { session_id: sessionId, input_path: inputPath } };
          break;
        case 'idb_list':
          data = { sessions: sessionId ? [{ session_id: sessionId, input_path: inputPath,
            owned: true, backend: 'worker', is_active: true }] : [] };
          break;
        case 'idb_close':
          data = { success: true, session_id: sessionId, saved: false, owned: true, backend: 'worker' };
          sessionId = undefined;
          break;
        case 'server_health':
          data = { status: 'ok', input_path: inputPath, idb_path: inputPath + '.i64',
            module: 'target.elf', imagebase: '0x1000', auto_analysis_ready: true };
          break;
        case 'lookup_funcs':
          data = args.queries.map(query => ({ query, error: null,
            fn: { addr: query === '0x2000' ? '0x2000' : '0x1000',
              name: query === '0x2000' ? 'goal' : 'main', size: '0x20' } }));
          break;
        case 'callees':
          data = [{ callees: [], ...(process.env.MORE === undefined ? {} :
            { more: process.env.MORE === 'true' }) }];
          break;
        default: throw new Error('Unexpected fixture tool: ' + name);
      }
      result = { content: [{ type: 'text', text: JSON.stringify(data) }] };
    }
    process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n');
  }
} else {
  const repo = resolve(process.argv[2]);
  const require = createRequire(join(repo, 'package.json'));
  const { Client } = await import(require.resolve('@modelcontextprotocol/client'));
  const { StdioClientTransport } = await import(require.resolve('@modelcontextprotocol/client/stdio'));
  const root = await mkdtemp(join(tmpdir(), 'coverage-fixture-'));
  const binary = join(root, 'target.elf');
  const config = join(root, 'provider.json');
  const bytes = Buffer.alloc(64);
  bytes.set([127, 69, 76, 70, 2, 1, 1]);
  bytes.writeUInt16LE(62, 18);
  bytes.writeUInt32LE(1, 20);
  bytes.writeUInt16LE(64, 52);
  await writeFile(binary, bytes);
  await writeFile(config, JSON.stringify({ command: process.execPath,
    args: [script, '--provider'], mode: 'headless', timeoutMs: 2000, workspaceRoot: root }));
  const client = new Client({ name: 'coverage-check', version: '1' });
  const transport = new StdioClientTransport({ command: process.execPath,
    args: [join(repo, 'scripts/rea.mjs'), 'mcp'],
    env: { ...process.env, HOME: root, TMPDIR: root, REA_ANALYSIS_PROVIDER: 'ida',
      REA_IDA_MCP_CONFIG: config, REA_MCP_INPUT_SCHEMA_PROFILE: 'compact',
      REA_MCP_OUTPUT_SCHEMA_PROFILE: 'off', REA_LOG_LEVEL: 'silent' } });
  const call = async (name, args) => {
    const result = await client.callTool({ name, arguments: args });
    if (result.isError) throw new Error(JSON.stringify(result));
    return result.structuredContent;
  };
  try {
    await client.connect(transport);
    await call('open_binary', { path: binary, provider_id: 'ida' });
    const direct = await call('procedure_callees', { procedure: 'main' });
    const trace = await call('trace_call_path', { start: 'main', goal: '0x2000' });
    const value = trace.normalized_result;
    console.log(JSON.stringify({
      upstream_callees: direct.raw_result.observations.find(item => item.tool === 'callees').result,
      relationship_coverage: direct.relationship_coverage,
      goal_status: value.goal_status, truncated: value.truncated,
      residual_unknowns: value.residual_unknowns
    }, null, 2));
    await call('close_binary', {});
    console.log('close_binary succeeded');
  } finally {
    await client.close();
    await rm(root, { recursive: true, force: true });
  }
}

Run these commands from the repository root with the pinned Node.js:

env -u MORE node "$repro_dir/coverage-repro.mjs" "$PWD"
MORE=false node "$repro_dir/coverage-repro.mjs" "$PWD"
MORE=true node "$repro_dir/coverage-repro.mjs" "$PWD"
rm -rf "$repro_dir"

The first run supplies [{"callees":[]}]. The other runs change only the explicit more marker. Each of these three script cases ran once locally. An earlier replay of the supplied reproduction independently confirmed omission twice and each explicit-marker control once.

Expected behavior

A missing truncation marker must leave the provider-observed relationship coverage unknown, or the incomplete provider response must fail. It must not establish an exhaustive negative traversal. If the read remains successful but unresolved, a trace that does not reach its goal must report an unknown goal result and retain the unresolved expansion. Explicit more: false should continue to permit complete coverage of the provider-observed set, while explicit more: true should preserve the capped-read limitation. None of these guarantees claims all possible runtime call edges.

Impact and scope

An accepted but incomplete producer reply yields an unsupported negative call-path answer and misleading coverage metadata. This is a bounded evidence-correctness defect; the executed sample uses an empty page and a simulated provider, and no real-provider frequency is established.

The reproduced direct Evidence exposes the raw upstream callee reply. A caller can inspect it, detect missing more, and withhold the automatic negative conclusion by retaining unknown status. This preserves uncertainty manually; it does not recover omitted edges or correct the automatic traversal.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Synthetic x86_64 ELF header for admission; local simulated headless IDA database-supervisor over stdio.

Interface

MCP

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Expected source: docs/tool-design.md:60-64, caller/callee coverage definition, explicitly requires missing coverage to remain unknown; docs/mcp-contracts.md:83-91, tool results, reserves not_reached for exhausted traversal without unresolved relationship reads. docs/ida-provider.md, "Give REA a target for headless analysis" and "Coverage and evidence", documents this supervisor profile and retained upstream callee truncation limitations.
Failure source: The executed public MCP reproduction above returned complete coverage and a definite negative goal result for the marker-free provider reply. Source-derived explanation is at src/ida/IdaProtocolValues.ts:166-183, which accepts omitted more, and src/ida/IdaOperationRunner.ts:514-535, which chooses complete coverage whenever more is not true.

The successful open, direct read, trace, and close establish reachability through the production MCP entry point for this simulated producer. Source comparison only: src/ida/IdaNativeProtocol.ts:65-81 rejects a successful native callee reply without a marker, so the observed failure is specifically the modern headless route. The emitted Evidence identified the admitted architecture as x86_64, which uses the callee route rather than the non-x86 callgraph route.

The omitted-marker run printed the following fields and then close_binary succeeded. The script exited with status 0.

{
  "upstream_callees": [{"callees":[]}],
  "relationship_coverage": {"status":"complete"},
  "goal_status": "not_reached",
  "truncated": false,
  "residual_unknowns": []
}

Explicit false returned the same coverage and traversal fields. Explicit true returned relationship_coverage.status: "partial", goal_status: "unknown", truncated: true, and residual_unknowns: ["Call relationships for 0x1000 are limited: Upstream callees capped results at 500; additional callees remain unknown."]. Both control scripts exited with status 0 and closed successfully. The direct Evidence retained the exact upstream reply, including the omitted marker. Real-provider frequency and behavior were not tested.

Typed outcome or recovery expected

Repeat the omitted, false, and true cases through the public MCP server. Omission must never return complete relationship coverage or support goal_status: "not_reached" as an exhausted negative traversal. Accept unknown coverage with the corresponding unknown goal and residual limitation, or an explicit provider-response failure. Preserve the explicit-false complete control and the explicit-true partial-coverage, unknown-goal control. Keep target admission, identity checks, and successful close behavior intact.

Analysis or suspected cause

The source references above explain the observed behavior. Runtime claims are limited to the explicitly described simulation boundaries.

Additional context

Runtime claims above describe the supplied local reproduction at the baseline commit. Real-provider frequency remains unmeasured.

The relevant failure code is unchanged on current origin/main fc712b3750.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3988 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior When a modern headless IDA provider returns an empty callee page without a `more` field, REA reports complete relationship coverage. A caller that then asks whether `main` reaches another resolved procedure receives `goal_status: "not_reached"` with no residual unknowns. The documented contract requires missing relationship coverage to remain unknown. This was reproduced through the production stdio MCP server with a simulated provider and an admitted x86_64 ELF fixture. ### Reproduction Use the baseline's installed dependencies and built runtime. From the repository root, create a disposable directory with `repro_dir=$(mktemp -d)` and save the following as `"$repro_dir/coverage-repro.mjs"`. The script implements the database-supervisor responses at the upstream stdio boundary and uses the repository's installed MCP client to call the production server. It isolates the server's home and provider workspace, opens the target, requests callees, traces to a different resolved goal, and closes the session. It never launches IDA or executes the target. ```js import { createInterface } from 'node:readline'; import { mkdtemp, writeFile, rm } from 'node:fs/promises'; import { join, resolve } from 'node:path'; import { tmpdir } from 'node:os'; import { fileURLToPath } from 'node:url'; import { createRequire } from 'node:module'; const script = fileURLToPath(import.meta.url); if (process.argv[2] === '--provider') { const names = ['idb_open', 'idb_list', 'idb_close', 'server_health', 'list_funcs', 'lookup_funcs', 'find_regex', 'decompile', 'disasm', 'callees', 'xrefs_to']; let sessionId, inputPath; for await (const line of createInterface({ input: process.stdin })) { const request = JSON.parse(line); if (request.id === undefined) continue; let result; if (request.method === 'initialize') { result = { protocolVersion: request.params.protocolVersion, capabilities: { tools: {} }, serverInfo: { name: 'coverage-fixture', version: '1' } }; } else if (request.method === 'tools/list') { result = { tools: names.map(name => ({ name, inputSchema: { type: 'object' } })) }; } else { const { name, arguments: args } = request.params; let data; switch (name) { case 'idb_open': sessionId = args.preferred_session_id; inputPath = args.input_path; data = { success: true, session: { session_id: sessionId, input_path: inputPath } }; break; case 'idb_list': data = { sessions: sessionId ? [{ session_id: sessionId, input_path: inputPath, owned: true, backend: 'worker', is_active: true }] : [] }; break; case 'idb_close': data = { success: true, session_id: sessionId, saved: false, owned: true, backend: 'worker' }; sessionId = undefined; break; case 'server_health': data = { status: 'ok', input_path: inputPath, idb_path: inputPath + '.i64', module: 'target.elf', imagebase: '0x1000', auto_analysis_ready: true }; break; case 'lookup_funcs': data = args.queries.map(query => ({ query, error: null, fn: { addr: query === '0x2000' ? '0x2000' : '0x1000', name: query === '0x2000' ? 'goal' : 'main', size: '0x20' } })); break; case 'callees': data = [{ callees: [], ...(process.env.MORE === undefined ? {} : { more: process.env.MORE === 'true' }) }]; break; default: throw new Error('Unexpected fixture tool: ' + name); } result = { content: [{ type: 'text', text: JSON.stringify(data) }] }; } process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n'); } } else { const repo = resolve(process.argv[2]); const require = createRequire(join(repo, 'package.json')); const { Client } = await import(require.resolve('@modelcontextprotocol/client')); const { StdioClientTransport } = await import(require.resolve('@modelcontextprotocol/client/stdio')); const root = await mkdtemp(join(tmpdir(), 'coverage-fixture-')); const binary = join(root, 'target.elf'); const config = join(root, 'provider.json'); const bytes = Buffer.alloc(64); bytes.set([127, 69, 76, 70, 2, 1, 1]); bytes.writeUInt16LE(62, 18); bytes.writeUInt32LE(1, 20); bytes.writeUInt16LE(64, 52); await writeFile(binary, bytes); await writeFile(config, JSON.stringify({ command: process.execPath, args: [script, '--provider'], mode: 'headless', timeoutMs: 2000, workspaceRoot: root })); const client = new Client({ name: 'coverage-check', version: '1' }); const transport = new StdioClientTransport({ command: process.execPath, args: [join(repo, 'scripts/rea.mjs'), 'mcp'], env: { ...process.env, HOME: root, TMPDIR: root, REA_ANALYSIS_PROVIDER: 'ida', REA_IDA_MCP_CONFIG: config, REA_MCP_INPUT_SCHEMA_PROFILE: 'compact', REA_MCP_OUTPUT_SCHEMA_PROFILE: 'off', REA_LOG_LEVEL: 'silent' } }); const call = async (name, args) => { const result = await client.callTool({ name, arguments: args }); if (result.isError) throw new Error(JSON.stringify(result)); return result.structuredContent; }; try { await client.connect(transport); await call('open_binary', { path: binary, provider_id: 'ida' }); const direct = await call('procedure_callees', { procedure: 'main' }); const trace = await call('trace_call_path', { start: 'main', goal: '0x2000' }); const value = trace.normalized_result; console.log(JSON.stringify({ upstream_callees: direct.raw_result.observations.find(item => item.tool === 'callees').result, relationship_coverage: direct.relationship_coverage, goal_status: value.goal_status, truncated: value.truncated, residual_unknowns: value.residual_unknowns }, null, 2)); await call('close_binary', {}); console.log('close_binary succeeded'); } finally { await client.close(); await rm(root, { recursive: true, force: true }); } } ``` Run these commands from the repository root with the pinned Node.js: ```sh env -u MORE node "$repro_dir/coverage-repro.mjs" "$PWD" MORE=false node "$repro_dir/coverage-repro.mjs" "$PWD" MORE=true node "$repro_dir/coverage-repro.mjs" "$PWD" rm -rf "$repro_dir" ``` The first run supplies `[{"callees":[]}]`. The other runs change only the explicit `more` marker. Each of these three script cases ran once locally. An earlier replay of the supplied reproduction independently confirmed omission twice and each explicit-marker control once. ### Expected behavior A missing truncation marker must leave the provider-observed relationship coverage unknown, or the incomplete provider response must fail. It must not establish an exhaustive negative traversal. If the read remains successful but unresolved, a trace that does not reach its goal must report an unknown goal result and retain the unresolved expansion. Explicit `more: false` should continue to permit complete coverage of the provider-observed set, while explicit `more: true` should preserve the capped-read limitation. None of these guarantees claims all possible runtime call edges. ### Impact and scope An accepted but incomplete producer reply yields an unsupported negative call-path answer and misleading coverage metadata. This is a bounded evidence-correctness defect; the executed sample uses an empty page and a simulated provider, and no real-provider frequency is established. The reproduced direct Evidence exposes the raw upstream callee reply. A caller can inspect it, detect missing `more`, and withhold the automatic negative conclusion by retaining unknown status. This preserves uncertainty manually; it does not recover omitted edges or correct the automatic traversal. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Synthetic x86_64 ELF header for admission; local simulated headless IDA database-supervisor over stdio. ### Interface MCP ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence - Expected source: `docs/tool-design.md:60-64`, caller/callee coverage definition, explicitly requires missing coverage to remain unknown; `docs/mcp-contracts.md:83-91`, tool results, reserves `not_reached` for exhausted traversal without unresolved relationship reads. `docs/ida-provider.md`, "Give REA a target for headless analysis" and "Coverage and evidence", documents this supervisor profile and retained upstream callee truncation limitations. - Failure source: The executed public MCP reproduction above returned complete coverage and a definite negative goal result for the marker-free provider reply. Source-derived explanation is at `src/ida/IdaProtocolValues.ts:166-183`, which accepts omitted `more`, and `src/ida/IdaOperationRunner.ts:514-535`, which chooses complete coverage whenever `more` is not true. The successful open, direct read, trace, and close establish reachability through the production MCP entry point for this simulated producer. Source comparison only: `src/ida/IdaNativeProtocol.ts:65-81` rejects a successful native callee reply without a marker, so the observed failure is specifically the modern headless route. The emitted Evidence identified the admitted architecture as `x86_64`, which uses the callee route rather than the non-x86 callgraph route. The omitted-marker run printed the following fields and then `close_binary succeeded`. The script exited with status 0. ```json { "upstream_callees": [{"callees":[]}], "relationship_coverage": {"status":"complete"}, "goal_status": "not_reached", "truncated": false, "residual_unknowns": [] } ``` Explicit false returned the same coverage and traversal fields. Explicit true returned `relationship_coverage.status: "partial"`, `goal_status: "unknown"`, `truncated: true`, and `residual_unknowns: ["Call relationships for 0x1000 are limited: Upstream callees capped results at 500; additional callees remain unknown."]`. Both control scripts exited with status 0 and closed successfully. The direct Evidence retained the exact upstream reply, including the omitted marker. Real-provider frequency and behavior were not tested. ### Typed outcome or recovery expected Repeat the omitted, false, and true cases through the public MCP server. Omission must never return complete relationship coverage or support `goal_status: "not_reached"` as an exhausted negative traversal. Accept unknown coverage with the corresponding unknown goal and residual limitation, or an explicit provider-response failure. Preserve the explicit-false complete control and the explicit-true partial-coverage, unknown-goal control. Keep target admission, identity checks, and successful close behavior intact. ### Analysis or suspected cause The source references above explain the observed behavior. Runtime claims are limited to the explicitly described simulation boundaries. ### Additional context Runtime claims above describe the supplied local reproduction at the baseline commit. Real-provider frequency remains unmeasured. The relevant failure code is unchanged on current `origin/main` fc712b3750cbe16755c4f78c1f90fdf09fbaca31.

bsakaguchi commented

2026-10-16 21:07:28 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Confirmed: an empty callee page without a more field only proves the first page is empty, not that the whole relationship set is covered. Returning not_reached with no residual unknowns turns absent coverage into a definite negative, which violates the documented unknown-preserving contract. Missing callee coverage should leave the goal status unknown until coverage is confirmed complete.

@XinwangTao commented on GitHub (Oct 15, 2026): Confirmed: an empty callee page without a more field only proves the first page is empty, not that the whole relationship set is covered. Returning not_reached with no residual unknowns turns absent coverage into a definite negative, which violates the documented unknown-preserving contract. Missing callee coverage should leave the goal status unknown until coverage is confirmed complete.

bsakaguchi referenced this issue

2026-10-17 21:12:36 +00:00

[PR #1584] [MERGED] test(native): skip the DMG EACCES diagnostic as root #2989

bsakaguchi closed this issue

2026-10-31 00:31:36 +00:00

bsakaguchi commented

2026-10-31 00:31:37 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

Confirmed the fix addresses the core issue: an omitted truncation marker no longer yields complete coverage, so a trace that cannot reach its goal reports an unknown goal with residual unknowns instead of a fabricated not_reached. Preserving the explicit more:false complete control and more:true partial/unknown control keeps the paging semantics intact.

@XinwangTao commented on GitHub (Oct 22, 2026): Confirmed the fix addresses the core issue: an omitted truncation marker no longer yields complete coverage, so a trace that cannot reach its goal reports an unknown goal with residual unknowns instead of a fabricated not_reached. Preserving the explicit more:false complete control and more:true partial/unknown control keeps the paging semantics intact.

bsakaguchi referenced this issue

2026-10-31 00:56:29 +00:00

[PR #1900] [MERGED] test(native): skip the inaccessible-directory diagnostic as root #4891

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1584

No description provided.

Rows
Columns