morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5846] [Bug] Firmware staging-budget refusal returns unreadable_output and doctor guidance #3543

New issue

Closed

opened 2026-10-31 00:41:41 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:41:41 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5846

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

An in-memory MCP call to extract_firmware with max_output_files: 1 returned isError: true, code unreadable_output, category execution_failure, and a message and remediation instructing the caller to run rea doctor. The response preserved Staging entry budget exceeded (1) in details.reason, but omitted details.reported_limits and the max_output_files parameter name.

The repository's synthetic Unblob producer creates rootfs/config: one directory and one regular file. Repeating the same request in the same session with only max_output_files raised to 2 succeeded. The failed request left the output destination absent, and the repository's process/workspace cleanup assertions passed after both calls.

This reproduction exercised production MCP handling, request validation, firmware inventory and error projection with the existing producer fake. Linux/tool admission was mocked to run on macOS; real Unblob, Linux prlimit, and the public CLI were not exercised. The equivalent byte-budget error is source-confirmed only.

Reproduction

Place this essential reproduction body in src/firmware/repro.test.ts and run npm run test:local -- src/firmware/repro.test.ts. The verifier ran it under a temporary filename beside the same module, then deleted it. One invocation reproduced the refusal and passed the sufficient-budget control.

The fixture injects its Node producer instead of running Unblob or the limiter. The additional mock bypasses only host/tool admission; inventory and projection are not mocked.

import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { access } from "node:fs/promises";
import { expect, it, onTestFinished, vi } from "vitest";

vi.mock("./FirmwareCommand.js", async (importOriginal) => {
  const original = await importOriginal<
    typeof import("./FirmwareCommand.js")
  >();
  return {
    ...original,
    resolveFirmwareCommand: async () => ({
      command: process.execPath,
      limiter: "/fixture/prlimit",
      sha256: "0".repeat(64),
    }),
  };
});

import { createServer } from "../server/createServer.js";
import { createTestBinarySession } from "../../tests/fixtures/binarySession.js";
import {
  firmwareFixture, assertFirmwareCleanup,
} from "../../tests/fixtures/firmware/provider.js";

it("projects a staging entry refusal and accepts a sufficient budget", async () => {
  const fixture = await firmwareFixture();
  const session = createTestBinarySession(() => {
    throw new Error("No native target required");
  });
  const server = createServer(
    { kind: "session", session }, { firmwareAnalysis: fixture.provider },
  );
  const client = new Client({ name: "firmware-budget-repro", version: "1" });
  onTestFinished(async () => {
    await client.close();
    await server.close();
    await session.close();
  });
  const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair();
  await server.connect(serverTransport);
  await client.connect(clientTransport);
  const invoke = (max_output_files: number) => client.callTool({
    name: "extract_firmware",
    arguments: {
      path: fixture.path, output_directory: fixture.output, max_output_files,
    },
  });
  const response = await invoke(1);
  expect(response.isError).toBe(true);
  const content = response.content as { type: string; text?: string }[];
  const error = JSON.parse(
    content.find((item) => item.type === "text")!.text!,
  ).error;
  console.log("REFUSAL", JSON.stringify(error));
  expect(error).toMatchObject({
    code: "unreadable_output", retryable: false,
    details: {
      operation: "extract_firmware", reason: "Staging entry budget exceeded (1)",
    },
  });
  expect(error.message).toContain("Run `rea doctor`");
  expect(error.remediation.action).toContain("Run `rea doctor`");
  expect(error.details).not.toHaveProperty("reported_limits");
  await expect(access(fixture.output)).rejects.toMatchObject({ code: "ENOENT" });
  await assertFirmwareCleanup(fixture.launches);
  const control = await invoke(2);
  expect(control.isError).not.toBe(true);
  expect(control.structuredContent).toBeDefined();
  console.log("CONTROL", JSON.stringify({
    max_output_files: 2, isError: control.isError ?? false,
    hasStructuredContent: control.structuredContent !== undefined,
  }));
  await assertFirmwareCleanup(fixture.launches);
});

For a real supported Linux setup, the corresponding request is extract_firmware with an absolute firmware path, an absent absolute output_directory, and max_output_files: 1, using a producer that writes at least two staging entries. This real-provider check is proposed, not executed.

Expected behavior

Keep refusing an extraction that exceeds its selected staging budget, but identify the actual resource constraint and give recovery about that budget. When the needed count is within the supported maximum, the caller can raise max_output_files and rerun with an absent output destination. Doctor guidance does not describe this recovery.

The firmware policy explicitly separates producer-report and staging allowances, documents caller-selected staging budgets, and requires inventory enforcement. The input contract defines these controls and counts directories and links as entries. The shared resource-error contract distinguishes resource failures from malformed input and models bounded admission refusals, including deterministic caller-selected bounds. Applying resource_constraint here follows those semantics; the firmware guide does not prescribe exact error wording or a particular resource-kind value.

Impact and scope

P2: callers receive the wrong machine-readable failure classification and irrelevant recovery for a supported, bounded request. The budget refusal remains effective, its numeric threshold survives in the reason, and unchanged retryability is correctly false. No corruption or security boundary crossing was observed.

A practical workaround was demonstrated for the synthetic entry case: raise the entry allowance from 1 to 2. Real-world prevalence and downstream automated-agent behavior were not measured. The byte-budget sibling and CLI applicability are derived from shared source, not additional executed reproductions. This concerns staging inventory refusals, not every possible producer or filesystem limit failure.

REA version

rea-agents 6.7.0; source baseline a5f982de8d.

Runtime and platform

Node.js v26.12.0; Darwin arm64; source test with synthetic producer and mocked Linux/tool admission; repository Node pin is 24.18.0.

Target and artifact context

Repository firmware fixture: nonempty synthetic input, valid synthetic Unblob report, rootfs directory and 14-byte config file; no real firmware engine or device.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

Trimmed stdout from the executed source test:

REFUSAL {"code":"unreadable_output","category":"execution_failure","message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","retryable":false,"remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."},"details":{"operation":"extract_firmware","reason":"Staging entry budget exceeded (1)"}}
CONTROL {"max_output_files":2,"isError":false,"hasStructuredContent":true}
Test Files  1 passed (1)
     Tests  1 passed (1)

Source corroboration: both staging overflow branches throw AnalysisOutputError. The provider forwards the caller's values. The projection maps this class to unreadable_output, and the presentation fallback supplies the observed doctor message.

There is already a budget integration test. It checks failure and cleanup, without asserting the projected code or recovery. No claim of absent budget coverage is needed.

Typed outcome or recovery expected

code: "resource_constraint", category resource_constraint, selected staging limits in details.reported_limits, and recovery naming the exceeded staging control. Keep retryable: false for an unchanged deterministic request. Do not imply that waiting, doctor, or adjusting an evidence-reply limit changes the staging allowance.

Analysis or suspected cause

Source-derived: the inventory uses the same output-error class for caller-selected staging overflow that projection uses for unreadable producer output. The producing boundary has the selected budgets, but its error does not carry resource-constraint metadata or budget-specific recovery.

Additional context

Searches of open and closed issues and PRs found no exact duplicate. #5808 proposes a broader recovery audit for firmware analysis timeouts, a different phase from this staging-allowance refusal. The closest related firmware work is issue 3888 and open PR 4430, which address the process file-size limit also capping the producer report. That PR preserves the staging overflow error class and does not correct this classification/recovery defect.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5846 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior An in-memory MCP call to `extract_firmware` with `max_output_files: 1` returned `isError: true`, code `unreadable_output`, category `execution_failure`, and a message and remediation instructing the caller to run `rea doctor`. The response preserved `Staging entry budget exceeded (1)` in `details.reason`, but omitted `details.reported_limits` and the `max_output_files` parameter name. The repository's synthetic Unblob producer creates `rootfs/config`: one directory and one regular file. Repeating the same request in the same session with only `max_output_files` raised to 2 succeeded. The failed request left the output destination absent, and the repository's process/workspace cleanup assertions passed after both calls. This reproduction exercised production MCP handling, request validation, firmware inventory and error projection with the existing producer fake. Linux/tool admission was mocked to run on macOS; real Unblob, Linux `prlimit`, and the public CLI were not exercised. The equivalent byte-budget error is source-confirmed only. ### Reproduction Place this essential reproduction body in `src/firmware/repro.test.ts` and run `npm run test:local -- src/firmware/repro.test.ts`. The verifier ran it under a temporary filename beside the same module, then deleted it. One invocation reproduced the refusal and passed the sufficient-budget control. The fixture injects its Node producer instead of running Unblob or the limiter. The additional mock bypasses only host/tool admission; inventory and projection are not mocked. ```ts import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { access } from "node:fs/promises"; import { expect, it, onTestFinished, vi } from "vitest"; vi.mock("./FirmwareCommand.js", async (importOriginal) => { const original = await importOriginal< typeof import("./FirmwareCommand.js") >(); return { ...original, resolveFirmwareCommand: async () => ({ command: process.execPath, limiter: "/fixture/prlimit", sha256: "0".repeat(64), }), }; }); import { createServer } from "../server/createServer.js"; import { createTestBinarySession } from "../../tests/fixtures/binarySession.js"; import { firmwareFixture, assertFirmwareCleanup, } from "../../tests/fixtures/firmware/provider.js"; it("projects a staging entry refusal and accepts a sufficient budget", async () => { const fixture = await firmwareFixture(); const session = createTestBinarySession(() => { throw new Error("No native target required"); }); const server = createServer( { kind: "session", session }, { firmwareAnalysis: fixture.provider }, ); const client = new Client({ name: "firmware-budget-repro", version: "1" }); onTestFinished(async () => { await client.close(); await server.close(); await session.close(); }); const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); await server.connect(serverTransport); await client.connect(clientTransport); const invoke = (max_output_files: number) => client.callTool({ name: "extract_firmware", arguments: { path: fixture.path, output_directory: fixture.output, max_output_files, }, }); const response = await invoke(1); expect(response.isError).toBe(true); const content = response.content as { type: string; text?: string }[]; const error = JSON.parse( content.find((item) => item.type === "text")!.text!, ).error; console.log("REFUSAL", JSON.stringify(error)); expect(error).toMatchObject({ code: "unreadable_output", retryable: false, details: { operation: "extract_firmware", reason: "Staging entry budget exceeded (1)", }, }); expect(error.message).toContain("Run `rea doctor`"); expect(error.remediation.action).toContain("Run `rea doctor`"); expect(error.details).not.toHaveProperty("reported_limits"); await expect(access(fixture.output)).rejects.toMatchObject({ code: "ENOENT" }); await assertFirmwareCleanup(fixture.launches); const control = await invoke(2); expect(control.isError).not.toBe(true); expect(control.structuredContent).toBeDefined(); console.log("CONTROL", JSON.stringify({ max_output_files: 2, isError: control.isError ?? false, hasStructuredContent: control.structuredContent !== undefined, })); await assertFirmwareCleanup(fixture.launches); }); ``` For a real supported Linux setup, the corresponding request is `extract_firmware` with an absolute firmware path, an absent absolute `output_directory`, and `max_output_files: 1`, using a producer that writes at least two staging entries. This real-provider check is proposed, not executed. ### Expected behavior Keep refusing an extraction that exceeds its selected staging budget, but identify the actual resource constraint and give recovery about that budget. When the needed count is within the supported maximum, the caller can raise `max_output_files` and rerun with an absent output destination. Doctor guidance does not describe this recovery. The [firmware policy](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/firmware-analysis.md#L173-L185) explicitly separates producer-report and staging allowances, documents caller-selected staging budgets, and requires inventory enforcement. The [input contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/firmware/firmwareAnalysis.ts#L32-L51) defines these controls and counts directories and links as entries. The [shared resource-error contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorCore.ts#L23-L55) distinguishes resource failures from malformed input and models bounded admission refusals, including deterministic caller-selected bounds. Applying `resource_constraint` here follows those semantics; the firmware guide does not prescribe exact error wording or a particular resource-kind value. ### Impact and scope P2: callers receive the wrong machine-readable failure classification and irrelevant recovery for a supported, bounded request. The budget refusal remains effective, its numeric threshold survives in the reason, and unchanged retryability is correctly false. No corruption or security boundary crossing was observed. A practical workaround was demonstrated for the synthetic entry case: raise the entry allowance from 1 to 2. Real-world prevalence and downstream automated-agent behavior were not measured. The byte-budget sibling and CLI applicability are derived from shared source, not additional executed reproductions. This concerns staging inventory refusals, not every possible producer or filesystem limit failure. ### REA version rea-agents 6.7.0; source baseline a5f982de8dbe4b092cc20ab89d492b7e7028b758. ### Runtime and platform Node.js v26.12.0; Darwin arm64; source test with synthetic producer and mocked Linux/tool admission; repository Node pin is 24.18.0. ### Target and artifact context Repository firmware fixture: nonempty synthetic input, valid synthetic Unblob report, rootfs directory and 14-byte config file; no real firmware engine or device. ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence Trimmed stdout from the executed source test: ```text REFUSAL {"code":"unreadable_output","category":"execution_failure","message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","retryable":false,"remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."},"details":{"operation":"extract_firmware","reason":"Staging entry budget exceeded (1)"}} CONTROL {"max_output_files":2,"isError":false,"hasStructuredContent":true} Test Files 1 passed (1) Tests 1 passed (1) ``` Source corroboration: both [staging overflow branches](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/firmware/FirmwareFiles.ts#L289-L320) throw `AnalysisOutputError`. The [provider forwards the caller's values](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/firmware/FirmwareProvider.ts#L454-L485). The [projection maps this class to unreadable_output](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorProjection.ts#L242-L254), and the [presentation fallback supplies the observed doctor message](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorPresentation.ts#L374-L399). There is already a [budget integration test](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/tests/boundary/firmware/firmwareIntegration.test.ts#L667-L684). It checks failure and cleanup, without asserting the projected code or recovery. No claim of absent budget coverage is needed. ### Typed outcome or recovery expected `code: "resource_constraint"`, category `resource_constraint`, selected staging limits in `details.reported_limits`, and recovery naming the exceeded staging control. Keep `retryable: false` for an unchanged deterministic request. Do not imply that waiting, doctor, or adjusting an evidence-reply limit changes the staging allowance. ### Analysis or suspected cause Source-derived: the inventory uses the same output-error class for caller-selected staging overflow that projection uses for unreadable producer output. The producing boundary has the selected budgets, but its error does not carry resource-constraint metadata or budget-specific recovery. ### Additional context Searches of open and closed issues and PRs found no exact duplicate. [#5808](https://github.com/morluto/rea/issues/5808) proposes a broader recovery audit for firmware analysis timeouts, a different phase from this staging-allowance refusal. The closest related firmware work is [issue 3888](https://github.com/morluto/rea/issues/3888) and [open PR 4430](https://github.com/morluto/rea/pull/4430), which address the process file-size limit also capping the producer report. That PR preserves the staging overflow error class and does not correct this classification/recovery defect.

bsakaguchi closed this issue

2026-10-31 00:41:41 +00:00

bsakaguchi commented

2026-10-31 00:41:41 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Same class of problem as the EVM reply-budget report: a deterministic budget refusal ("Staging entry budget exceeded (1)") is being projected into unreadable_output with doctor guidance, which misleads callers into thinking the output was corrupted rather than that the request simply asked for too many files. The refusal already carries the budget in details.reason — it should map to a resource-constraint style error that echoes reported_limits and the max_output_files parameter, so the fix is also to retry with a higher allowance. Keeping this distinct from genuinely unreadable outputs would help automation react correctly.

@XinwangTao commented on GitHub (Oct 21, 2026): Same class of problem as the EVM reply-budget report: a deterministic budget refusal ("Staging entry budget exceeded (1)") is being projected into unreadable_output with doctor guidance, which misleads callers into thinking the output was corrupted rather than that the request simply asked for too many files. The refusal already carries the budget in details.reason — it should map to a resource-constraint style error that echoes reported_limits and the max_output_files parameter, so the fix is also to retry with a higher allowance. Keeping this distinct from genuinely unreadable outputs would help automation react correctly.

bsakaguchi commented

2026-10-31 00:41:42 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

Firmware inventory now raises a deterministic capacity resource constraint with the selected max_output_files or max_output_bytes value and parameter-specific recovery; it no longer constructs AnalysisOutputError for these budgets. The regression exercises both real inventory limits and asserts error type, reportedLimits, nonretryable unchanged request and parameter remediation.

Implementation: FirmwareFiles.ts, firmwareReportFiles.test.ts.

The relevant regression checks in firmwareReportFiles.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. Firmware inventory now raises a deterministic capacity resource constraint with the selected max_output_files or max_output_bytes value and parameter-specific recovery; it no longer constructs AnalysisOutputError for these budgets. The regression exercises both real inventory limits and asserts error type, reportedLimits, nonretryable unchanged request and parameter remediation. Implementation: [FirmwareFiles.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/firmware/FirmwareFiles.ts#L445), [firmwareReportFiles.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/firmware/firmwareReportFiles.test.ts#L190). The relevant regression checks in [firmwareReportFiles.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/firmware/firmwareReportFiles.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:09:39 +00:00

[PR #3549] [MERGED] fix(ci): require portable inventory reports on selected runs #5817

bsakaguchi referenced this issue

2026-10-31 01:11:25 +00:00

[PR #4080] [MERGED] perf(tests): reuse CLI compilation for exported binding checks #6043