[GH-ISSUE #7374] [Bug] External functions prevent decompile_all from stopping after repeated output I/O failures #4382

New issue

Open

opened 2026-10-31 00:49:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:49:23 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7374

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The registered MCP decompile_all tool returns success and keeps decompiling after five local function files fail to publish when newly classified external functions alternate with them. In an isolated synthetic-provider run, six local functions all failed to write, five external functions were recorded, and the tool returned isError: false. A six-local-function control stopped after five failures with unreadable_output. The manifest reports the failed files correctly; the missing early stop is the defect.

Reproduction

Use Node.js v24.22.1. For a fresh checkout at the revision below, prepare the locked runtime with npm ci && npm run build:cached. From the checkout root, run this fixture. It uses the production MCP registration and filesystem export, with a synthetic provider and no real engine. Each arm of this reduced probe was executed once; a separate ten-local-function probe also reproduced the difference. Directories created at reserved function-file paths during procedure_info cause real publication I/O failures without filling a disk.

WORK="$(mktemp -d)"
mkdir -p "$WORK"/{home,config,data,cache,state,tmp}
env -i PATH="$PATH" HOME="$WORK/home" XDG_CONFIG_HOME="$WORK/config" XDG_DATA_HOME="$WORK/data" XDG_CACHE_HOME="$WORK/cache" XDG_STATE_HOME="$WORK/state" TMPDIR="$WORK/tmp" WORK="$WORK" node --input-type=module <<'JS'
import { mkdir, readFile } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import pino from "pino";
import { EvidenceMcpServer } from "./dist/server/EvidenceMcpServer.js";
import { registerEnhancedTools } from "./dist/server/registerEnhancedTools.js";
import { createToolResultDelivery } from "./dist/server/toolResult.js";
import { withAdmittedAnalysis } from "./dist/server/analysisAdmission.js";
import { createAnalysisExecution } from "./dist/application/AnalysisProvider.js";
import { createAnalysisProfile } from "./dist/domain/analysisProfile.js";
import { ok } from "./dist/domain/result.js";
const provider = { id: "fixture", name: "Fixture", version: "1" };
const profile = createAnalysisProfile(provider, { decompiler: "fixture" });
const subject = { path: "fixture", sha256: "ab".repeat(32), format: "elf", architecture: "x86_64" };
const observation = value => ok(createAnalysisExecution(value, provider, { subject, analysisProfile: profile, limitations: [] }));
for (const interleave of [false, true]) {
  const out = join(process.env.WORK, interleave ? "mixed" : "control");
  const rows = Array.from({ length: interleave ? 11 : 6 }, (_, i) => ({ address: `0x${(4096 + i * 16).toString(16)}`, value: `f${i}` }));
  let decompiles = 0;
  const analysis = { execute: async (operation, args) => {
    if (operation === "list_procedures") return observation(rows);
    if (operation === "procedure_pseudo_code") { decompiles++; return observation("int f(void) { return 0; }"); }
    if (operation !== "procedure_info") throw new Error(operation);
    const address = args.procedure;
    const external = interleave && rows.findIndex(r => r.address === address) % 2 === 1;
    if (!external) await mkdir(join(out, "functions", `${address}.c`), { recursive: true });
    return observation({ name: address, entrypoint: address, basicblock_count: 1, length: 1, signature: null, locals: [],
      classification: external ? { external: true, thunk: false, thunk_target: null, provenance: "fixture" } : null,
      body: external ? { available: false, reason: "import" } : { available: true, provenance: "fixture", ranges: [{ start: address, end: address }], total_bytes: 1, span_bytes: 1, non_contiguous: false, contains_entry: true } });
  }};
  const server = new EvidenceMcpServer({ name: "fixture", version: "1" }, {}, undefined, createToolResultDelivery(4 * 1024 * 1024));
  registerEnhancedTools(server, { withAdmittedAnalysis: withAdmittedAnalysis({ kind: "fixed", analysis }), logger: pino({ enabled: false }), activeTarget: () => undefined, analysisProfile: () => profile });
  const client = new Client({ name: "fixture", version: "1" });
  const [ct, st] = InMemoryTransport.createLinkedPair();
  try {
    await server.connect(st); await client.connect(ct);
    const reply = await client.callTool({ name: "decompile_all", arguments: { output_directory: out } });
    const payload = JSON.parse(reply.content[0].text);
    const index = JSON.parse(await readFile(join(out, "decompile-index.json"), "utf8"));
    console.log(JSON.stringify({ interleave, isError: reply.isError === true, code: payload.error?.code ?? null, streak: payload.error?.details?.diagnostics?.consecutive_output_io ?? null, counts: payload.normalized_result?.counts ?? null, statuses: index.functions.map(r => r.status), decompiles }));
  } finally { await client.close(); await server.close(); }
}
JS

Expected behavior

Both cases should stop with unreadable_output and details.diagnostics.consecutive_output_io: 5, leaving the index available. docs/native-investigation.md:233-236 promises stopping after five consecutive unreadable/unwritable function files; src/application/decompileDirectory/decompileAll.ts:86-90 explicitly says a recorded external function does not end that streak. External functions produce no file or pseudocode request (docs/native-investigation.md:222-225), so they provide no evidence that output writes recovered.

Impact and scope

P2: a run can spend further provider work on functions whose files cannot be published and omit the documented stop error. The reduced mixed case decompiled six local functions versus five in the control; the larger mixed case decompiled ten. The failed count remains visible and the index survives. The caller can bound attempts with the existing limit input and inspect failed counts. Native-engine behavior, actual disk exhaustion and filesystem permission loss were not tested.

REA version

Exact node scripts/rea.mjs --version output: 6.8.0. Source commit: 79b8219f290ef6304e469d9b8c40dc2af30edba9 (current main checked and rebuilt for verification).

Runtime and platform

Node.js v24.22.1; macOS 27.0.1; Darwin arm64. Locked MCP SDK in-memory transport. No Hopper, Ghidra, IDA or device was opened.

Target and artifact context

Synthetic ELF/x86_64 subject identity and a fixed analysis profile. The provider returns six local functions for the control, or eleven alternating local/external rows for the mixed case. The artifact is a disposable directory export with reserved function-file paths deliberately replaced by directories before publication. No executable target is loaded or run.

Interface

MCP

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

Observed output from the reduced registered-MCP probe:

{"interleave":false,"isError":true,"code":"unreadable_output","streak":5,"counts":null,"statuses":["failed","failed","failed","failed","failed"],"decompiles":5}
{"interleave":true,"isError":false,"code":null,"streak":null,"counts":{"selected":11,"excluded":0,"requested":11,"written":0,"failed":6,"external":5,"skipped":0,"remaining":0},"statuses":["failed","external","failed","external","failed","external","failed","external","failed","external","failed"],"decompiles":6}

The existing focused tests for five consecutive I/O failures and external-function classification both passed on the stated commit (two passed, 53 unselected). Neither exercises their interleaving.

Typed outcome or recovery expected

Stop on the fifth failed local file with five failed and four external index records in this fixture, before requesting pseudocode for the sixth local function. Proposed restoration checks: retain those partial records, preserve external classification, and resume the remaining work after making the reserved paths writable. Writable resume was not exercised in this verification.

Analysis or suspected cause

Source-derived: src/application/decompileDirectory/decompileAll.ts:391-398 clears lastFailureCode, runs each attempt, then resets the streak for every outcome other than output_io. The external branch at :487-494 only appends a record, leaving that code unset. It therefore resets the counter even though no file was written. The stop check at :402-403 cannot reach five in the alternating case. Existing journaled externals are skipped at :381-386 without this reset.

Additional context

Searched open and closed issues and PRs using decompile_all external, output_io, decompile consecutive, and unreadable_output, plus the local full-body snapshot. Open #7351 concerns retained limitations on failed resumes; open #7280 concerns empty CLI targets. Neither has the same failure or acceptance criteria. Closed feature #6090 and merged implementation #6170 are historical workflow context, not fixes for this reproduced failure.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The registered MCP workflow performs decompilation beyond the documented five-file I/O stop and returns success with all local writes failed when external records intervene; failed counts and the index remain available.
Workaround status: available
Workaround basis: The existing positive attempt limit bounds provider work, and the returned manifest exposes the failed count so callers can repair output before continuing. No native-engine or actual full-disk occurrence is claimed.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7374 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior The registered MCP `decompile_all` tool returns success and keeps decompiling after five local function files fail to publish when newly classified external functions alternate with them. In an isolated synthetic-provider run, six local functions all failed to write, five external functions were recorded, and the tool returned `isError: false`. A six-local-function control stopped after five failures with `unreadable_output`. The manifest reports the failed files correctly; the missing early stop is the defect. ## Reproduction Use Node.js v24.22.1. For a fresh checkout at the revision below, prepare the locked runtime with `npm ci && npm run build:cached`. From the checkout root, run this fixture. It uses the production MCP registration and filesystem export, with a synthetic provider and no real engine. Each arm of this reduced probe was executed once; a separate ten-local-function probe also reproduced the difference. Directories created at reserved function-file paths during `procedure_info` cause real publication I/O failures without filling a disk. ```sh WORK="$(mktemp -d)" mkdir -p "$WORK"/{home,config,data,cache,state,tmp} env -i PATH="$PATH" HOME="$WORK/home" XDG_CONFIG_HOME="$WORK/config" XDG_DATA_HOME="$WORK/data" XDG_CACHE_HOME="$WORK/cache" XDG_STATE_HOME="$WORK/state" TMPDIR="$WORK/tmp" WORK="$WORK" node --input-type=module <<'JS' import { mkdir, readFile } from "node:fs/promises"; import { join } from "node:path"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import pino from "pino"; import { EvidenceMcpServer } from "./dist/server/EvidenceMcpServer.js"; import { registerEnhancedTools } from "./dist/server/registerEnhancedTools.js"; import { createToolResultDelivery } from "./dist/server/toolResult.js"; import { withAdmittedAnalysis } from "./dist/server/analysisAdmission.js"; import { createAnalysisExecution } from "./dist/application/AnalysisProvider.js"; import { createAnalysisProfile } from "./dist/domain/analysisProfile.js"; import { ok } from "./dist/domain/result.js"; const provider = { id: "fixture", name: "Fixture", version: "1" }; const profile = createAnalysisProfile(provider, { decompiler: "fixture" }); const subject = { path: "fixture", sha256: "ab".repeat(32), format: "elf", architecture: "x86_64" }; const observation = value => ok(createAnalysisExecution(value, provider, { subject, analysisProfile: profile, limitations: [] })); for (const interleave of [false, true]) { const out = join(process.env.WORK, interleave ? "mixed" : "control"); const rows = Array.from({ length: interleave ? 11 : 6 }, (_, i) => ({ address: `0x${(4096 + i * 16).toString(16)}`, value: `f${i}` })); let decompiles = 0; const analysis = { execute: async (operation, args) => { if (operation === "list_procedures") return observation(rows); if (operation === "procedure_pseudo_code") { decompiles++; return observation("int f(void) { return 0; }"); } if (operation !== "procedure_info") throw new Error(operation); const address = args.procedure; const external = interleave && rows.findIndex(r => r.address === address) % 2 === 1; if (!external) await mkdir(join(out, "functions", `${address}.c`), { recursive: true }); return observation({ name: address, entrypoint: address, basicblock_count: 1, length: 1, signature: null, locals: [], classification: external ? { external: true, thunk: false, thunk_target: null, provenance: "fixture" } : null, body: external ? { available: false, reason: "import" } : { available: true, provenance: "fixture", ranges: [{ start: address, end: address }], total_bytes: 1, span_bytes: 1, non_contiguous: false, contains_entry: true } }); }}; const server = new EvidenceMcpServer({ name: "fixture", version: "1" }, {}, undefined, createToolResultDelivery(4 * 1024 * 1024)); registerEnhancedTools(server, { withAdmittedAnalysis: withAdmittedAnalysis({ kind: "fixed", analysis }), logger: pino({ enabled: false }), activeTarget: () => undefined, analysisProfile: () => profile }); const client = new Client({ name: "fixture", version: "1" }); const [ct, st] = InMemoryTransport.createLinkedPair(); try { await server.connect(st); await client.connect(ct); const reply = await client.callTool({ name: "decompile_all", arguments: { output_directory: out } }); const payload = JSON.parse(reply.content[0].text); const index = JSON.parse(await readFile(join(out, "decompile-index.json"), "utf8")); console.log(JSON.stringify({ interleave, isError: reply.isError === true, code: payload.error?.code ?? null, streak: payload.error?.details?.diagnostics?.consecutive_output_io ?? null, counts: payload.normalized_result?.counts ?? null, statuses: index.functions.map(r => r.status), decompiles })); } finally { await client.close(); await server.close(); } } JS ``` ## Expected behavior Both cases should stop with `unreadable_output` and `details.diagnostics.consecutive_output_io: 5`, leaving the index available. `docs/native-investigation.md:233-236` promises stopping after five consecutive unreadable/unwritable function files; `src/application/decompileDirectory/decompileAll.ts:86-90` explicitly says a recorded external function does not end that streak. External functions produce no file or pseudocode request (`docs/native-investigation.md:222-225`), so they provide no evidence that output writes recovered. ## Impact and scope P2: a run can spend further provider work on functions whose files cannot be published and omit the documented stop error. The reduced mixed case decompiled six local functions versus five in the control; the larger mixed case decompiled ten. The failed count remains visible and the index survives. The caller can bound attempts with the existing `limit` input and inspect failed counts. Native-engine behavior, actual disk exhaustion and filesystem permission loss were not tested. ## REA version Exact `node scripts/rea.mjs --version` output: `6.8.0`. Source commit: `79b8219f290ef6304e469d9b8c40dc2af30edba9` (current main checked and rebuilt for verification). ## Runtime and platform Node.js `v24.22.1`; macOS `27.0.1`; `Darwin arm64`. Locked MCP SDK in-memory transport. No Hopper, Ghidra, IDA or device was opened. ## Target and artifact context Synthetic ELF/x86_64 subject identity and a fixed analysis profile. The provider returns six local functions for the control, or eleven alternating local/external rows for the mixed case. The artifact is a disposable directory export with reserved function-file paths deliberately replaced by directories before publication. No executable target is loaded or run. ## Interface MCP ## Earliest failure stage Artifact or evidence registration ## Relevant output or evidence Observed output from the reduced registered-MCP probe: ```json {"interleave":false,"isError":true,"code":"unreadable_output","streak":5,"counts":null,"statuses":["failed","failed","failed","failed","failed"],"decompiles":5} {"interleave":true,"isError":false,"code":null,"streak":null,"counts":{"selected":11,"excluded":0,"requested":11,"written":0,"failed":6,"external":5,"skipped":0,"remaining":0},"statuses":["failed","external","failed","external","failed","external","failed","external","failed","external","failed"],"decompiles":6} ``` The existing focused tests for five consecutive I/O failures and external-function classification both passed on the stated commit (two passed, 53 unselected). Neither exercises their interleaving. ## Typed outcome or recovery expected Stop on the fifth failed local file with five failed and four external index records in this fixture, before requesting pseudocode for the sixth local function. Proposed restoration checks: retain those partial records, preserve external classification, and resume the remaining work after making the reserved paths writable. Writable resume was not exercised in this verification. ## Analysis or suspected cause Source-derived: `src/application/decompileDirectory/decompileAll.ts:391-398` clears `lastFailureCode`, runs each attempt, then resets the streak for every outcome other than `output_io`. The external branch at `:487-494` only appends a record, leaving that code unset. It therefore resets the counter even though no file was written. The stop check at `:402-403` cannot reach five in the alternating case. Existing journaled externals are skipped at `:381-386` without this reset. ## Additional context Searched open and closed issues and PRs using `decompile_all external`, `output_io`, `decompile consecutive`, and `unreadable_output`, plus the local full-body snapshot. Open [#7351](https://github.com/morluto/rea/issues/7351) concerns retained limitations on failed resumes; open [#7280](https://github.com/morluto/rea/issues/7280) concerns empty CLI targets. Neither has the same failure or acceptance criteria. Closed feature [#6090](https://github.com/morluto/rea/issues/6090) and merged implementation [#6170](https://github.com/morluto/rea/pull/6170) are historical workflow context, not fixes for this reproduced failure. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The registered MCP workflow performs decompilation beyond the documented five-file I/O stop and returns success with all local writes failed when external records intervene; failed counts and the index remain available. - Workaround status: available - Workaround basis: The existing positive attempt limit bounds provider work, and the returned manifest exposes the failed count so callers can repair output before continuing. No native-engine or actual full-disk occurrence is claimed.

bsakaguchi referenced this issue

2026-10-31 01:12:44 +00:00

[PR #4401] [CLOSED] fix(reference): retain static template require dependencies #6214

bsakaguchi referenced this issue

2026-10-31 01:12:47 +00:00

[PR #4409] [CLOSED] fix(reference): retain exact CommonJS template require specifiers #6220

bsakaguchi referenced this issue

2026-10-31 01:13:14 +00:00

[PR #4511] [MERGED] fix(source): retain CommonJS requires spelled as exact no-substitution templates #6279