morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5827] [Bug] Headless IDA classifies a valid not-ready health reply as non-retryable unreadable output #3524

New issue

Closed

opened 2026-10-31 00:41:29 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:41:29 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5827

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

An executed adapter-boundary reproduction of list_procedures with the repository's recording IDA transport returns unreadable_output, retryable: false when a headless worker replies to server_health with valid identity fields, status: "ok", and auto_analysis_ready: false. REA reads health once, never dispatches list_funcs, and tells the caller that repeating the unchanged request is not expected to succeed.

The identical request succeeds on the same client after the fixture reports readiness as true. A separate case with auto_analysis_ready: null produces the same refusal and recovery. Null establishes unknown readiness, not ongoing analysis.

This was executed against the production session client and error projection with a fake transport. No real IDA supervisor, public CLI command, or complete MCP request was run; the frequency of either readiness value in a real deployment is unknown.

Reproduction

From a baseline checkout with locked dependencies installed, save the following temporary test as src/ida/repro.test.ts. This placement belongs to the repository's adapter test project. It uses the existing ELF target and recording MCP fixtures, including their worker ownership and private-input lifecycle.

import { rm } from "node:fs/promises";
import { expect, it } from "vitest";
import { createIdaTarget, RecordingIdaMcp } from "../../tests/fixtures/idaMcp.js";
import { IdaSessionClient } from "./IdaSessionClient.js";
import { modernHealthSchema } from "./IdaProtocolValues.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";

it.each([false, null])("projects accepted readiness %s as permanent unreadable output", async (readiness) => {
  const { root, target } = await createIdaTarget();
  const producer = new RecordingIdaMcp(target, "headless");
  let reads = 0;
  const client = new IdaSessionClient({
    command: "fixture", args: [], env: {}, mode: "headless",
    timeoutMs: 1000, workspaceRoot: root,
  }, target, producer);
  producer.overrideCall = (name) => {
    if (name !== "server_health") return undefined;
    reads++;
    const payload = {
      status: "ok", input_path: producer.inputPath,
      idb_path: `${producer.inputPath}.i64`, imagebase: producer.imageBase,
      module: "target.elf", auto_analysis_ready: readiness,
    };
    expect(modernHealthSchema.safeParse(payload).success).toBe(true);
    return payload;
  };
  try {
    const result = await client.execute("list_procedures", {});
    if (result.ok) throw new Error("expected readiness refusal");
    const projected = projectAnalysisError(result.error);
    expect(result.error._tag).toBe("AnalysisProtocolError");
    expect(projected).toMatchObject({code: "unreadable_output", retryable: false});
    expect(reads).toBe(1);
    expect(producer.calls.filter(({name}) => name === "list_funcs")).toHaveLength(0);
    console.log(JSON.stringify({readiness, healthReads: reads, tag: result.error._tag,
      code: projected.code, retryable: projected.retryable,
      message: projected.message, remediation: projected.remediation}));
    producer.overrideCall = undefined;
    const recovered = await client.execute("list_procedures", {});
    expect(recovered.ok).toBe(true);
    console.log(JSON.stringify({readiness, sameRequestAfterReady: recovered.ok}));
  } finally {
    try { expect((await client.close()).ok).toBe(true); }
    finally { await rm(root, {recursive: true, force: true}); }
  }
});

Run npm run test:local -- src/ida/repro.test.ts with test home/configuration and temporary directories isolated. Delete the temporary test afterward. Both parameter cases passed in each of two focused runs: they assert the incorrect refusal, one health read, no inventory dispatch, successful later retry, and successful close. Passing means the test reproduced the reported defect, not that REA met the intended readiness contract.

Expected behavior

REA should preserve a valid readiness refusal as a readiness condition, with recovery instructions that agree with the possibility of later success. For a worker that explicitly reports analysis incomplete, wait within an appropriate bounded readiness deadline or return an actionable retryable readiness outcome. Preserve unknown readiness distinctly when the value is null; do not infer that analysis is in progress from null alone.

The provider guide promises waiting for headless auto-analysis and bounded retryable handling of busy workers. REA's health schema accepts false and null, and the readiness guard explicitly recommends retrying after analysis completes. The lifecycle contract separates provider readiness from admission and preserves unknown readiness.

Impact and scope

P2: the reproduced condition prevents a supported headless inventory request and gives callers incorrect recovery guidance. In the fixture, retrying after readiness becomes true is a safe workaround and returns success without changing the request. This does not demonstrate how long a real caller would need to wait or whether reopening a real worker would reproduce the condition.

The defect is conditional on a schema-valid not-ready reply after the private input identity checks pass. Attached profiles use a different observation branch. Other admitted headless operations use the same observation guard, but only list_procedures was executed here. No incorrect analysis result, data corruption, or security boundary crossing was observed.

REA version

rea-agents 6.7.0, source revision a5f982de8d.

Runtime and platform

Node.js v24.22.0; macOS arm64, Darwin 27.0.0; Vitest 5.0.3; recording transport, no IDA installation used; exact repository Node pin 24.18.0 was not used.

Target and artifact context

Repository-generated ELF x86_64 fixture with a digest-checked private copy and recording headless worker lifecycle; no private target or real engine.

Interface

MCP

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Trimmed output from the final focused run:

> rea-agents@6.7.0 test:local
> node scripts/run-development-tests.mjs local src/ida/repro.test.ts

{"readiness":false,"healthReads":1,"tag":"AnalysisProtocolError","code":"unreadable_output","retryable":false,"message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}}
{"readiness":false,"sameRequestAfterReady":true}
{"readiness":null,"healthReads":1,"tag":"AnalysisProtocolError","code":"unreadable_output","retryable":false,"message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}}
{"readiness":null,"sameRequestAfterReady":true}

✓ |adapters| src/ida/repro.test.ts (2 tests) 12ms
Test Files  1 passed (1)
Tests       2 passed (2)

Source-derived public reachability: list_procedures is a registered contract; the tool handler forwards its operation, the session dispatches to its active client, and composite routing selects the provider client. The IDA client observes health before dispatch. The recording fixture passed the real startup catalog, owned-worker, path and digest guards in this reproduction.

Typed outcome or recovery expected

A readiness outcome whose code, retryability and remediation describe the state actually observed. A bounded wait expiring while analysis remains incomplete could use a retryable timeout; null readiness should retain its uncertainty. The caller should not be sent to doctor solely because an otherwise readable health result reports incomplete analysis.

Analysis or suspected cause

Source-derived: the polling loop recognizes only the busy payload. It passes an ok-shaped reply immediately to the separate readiness guard, which throws AnalysisProtocolError. That tag maps to unreadable_output; retryability follows code/type rules, and the generic message contradicts the demonstrated successful retry.

Additional context

Read-only searches of open and closed issues and PRs found no matching report. The closest report, issue 1766, concerns a different status: "busy" payload without identity fields; the current busy polling code handles that case. This report isolates the valid ok-shaped reply with false or unknown readiness. It does not assert that binary_session executes this guard or independently proves analysis readiness.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5827 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior An executed adapter-boundary reproduction of `list_procedures` with the repository's recording IDA transport returns `unreadable_output`, `retryable: false` when a headless worker replies to `server_health` with valid identity fields, `status: "ok"`, and `auto_analysis_ready: false`. REA reads health once, never dispatches `list_funcs`, and tells the caller that repeating the unchanged request is not expected to succeed. The identical request succeeds on the same client after the fixture reports readiness as true. A separate case with `auto_analysis_ready: null` produces the same refusal and recovery. Null establishes unknown readiness, not ongoing analysis. This was executed against the production session client and error projection with a fake transport. No real IDA supervisor, public CLI command, or complete MCP request was run; the frequency of either readiness value in a real deployment is unknown. ### Reproduction From a baseline checkout with locked dependencies installed, save the following temporary test as `src/ida/repro.test.ts`. This placement belongs to the repository's adapter test project. It uses the existing ELF target and recording MCP fixtures, including their worker ownership and private-input lifecycle. ```ts import { rm } from "node:fs/promises"; import { expect, it } from "vitest"; import { createIdaTarget, RecordingIdaMcp } from "../../tests/fixtures/idaMcp.js"; import { IdaSessionClient } from "./IdaSessionClient.js"; import { modernHealthSchema } from "./IdaProtocolValues.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; it.each([false, null])("projects accepted readiness %s as permanent unreadable output", async (readiness) => { const { root, target } = await createIdaTarget(); const producer = new RecordingIdaMcp(target, "headless"); let reads = 0; const client = new IdaSessionClient({ command: "fixture", args: [], env: {}, mode: "headless", timeoutMs: 1000, workspaceRoot: root, }, target, producer); producer.overrideCall = (name) => { if (name !== "server_health") return undefined; reads++; const payload = { status: "ok", input_path: producer.inputPath, idb_path: `${producer.inputPath}.i64`, imagebase: producer.imageBase, module: "target.elf", auto_analysis_ready: readiness, }; expect(modernHealthSchema.safeParse(payload).success).toBe(true); return payload; }; try { const result = await client.execute("list_procedures", {}); if (result.ok) throw new Error("expected readiness refusal"); const projected = projectAnalysisError(result.error); expect(result.error._tag).toBe("AnalysisProtocolError"); expect(projected).toMatchObject({code: "unreadable_output", retryable: false}); expect(reads).toBe(1); expect(producer.calls.filter(({name}) => name === "list_funcs")).toHaveLength(0); console.log(JSON.stringify({readiness, healthReads: reads, tag: result.error._tag, code: projected.code, retryable: projected.retryable, message: projected.message, remediation: projected.remediation})); producer.overrideCall = undefined; const recovered = await client.execute("list_procedures", {}); expect(recovered.ok).toBe(true); console.log(JSON.stringify({readiness, sameRequestAfterReady: recovered.ok})); } finally { try { expect((await client.close()).ok).toBe(true); } finally { await rm(root, {recursive: true, force: true}); } } }); ``` Run `npm run test:local -- src/ida/repro.test.ts` with test home/configuration and temporary directories isolated. Delete the temporary test afterward. Both parameter cases passed in each of two focused runs: they assert the incorrect refusal, one health read, no inventory dispatch, successful later retry, and successful close. Passing means the test reproduced the reported defect, not that REA met the intended readiness contract. ### Expected behavior REA should preserve a valid readiness refusal as a readiness condition, with recovery instructions that agree with the possibility of later success. For a worker that explicitly reports analysis incomplete, wait within an appropriate bounded readiness deadline or return an actionable retryable readiness outcome. Preserve unknown readiness distinctly when the value is null; do not infer that analysis is in progress from null alone. The [provider guide](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/ida-provider.md#L145) promises waiting for headless auto-analysis and bounded retryable handling of busy workers. REA's [health schema](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/ida/IdaProtocolValues.ts#L47-L56) accepts false and null, and the [readiness guard](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/ida/IdaSessionClient.ts#L698-L712) explicitly recommends retrying after analysis completes. The [lifecycle contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/tool-design.md#L78-L95) separates provider readiness from admission and preserves unknown readiness. ### Impact and scope P2: the reproduced condition prevents a supported headless inventory request and gives callers incorrect recovery guidance. In the fixture, retrying after readiness becomes true is a safe workaround and returns success without changing the request. This does not demonstrate how long a real caller would need to wait or whether reopening a real worker would reproduce the condition. The defect is conditional on a schema-valid not-ready reply after the private input identity checks pass. Attached profiles use a different observation branch. Other admitted headless operations use the same observation guard, but only `list_procedures` was executed here. No incorrect analysis result, data corruption, or security boundary crossing was observed. ### REA version rea-agents 6.7.0, source revision a5f982de8dbe4b092cc20ab89d492b7e7028b758. ### Runtime and platform Node.js v24.22.0; macOS arm64, Darwin 27.0.0; Vitest 5.0.3; recording transport, no IDA installation used; exact repository Node pin 24.18.0 was not used. ### Target and artifact context Repository-generated ELF x86_64 fixture with a digest-checked private copy and recording headless worker lifecycle; no private target or real engine. ### Interface MCP ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Trimmed output from the final focused run: ```text > rea-agents@6.7.0 test:local > node scripts/run-development-tests.mjs local src/ida/repro.test.ts {"readiness":false,"healthReads":1,"tag":"AnalysisProtocolError","code":"unreadable_output","retryable":false,"message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}} {"readiness":false,"sameRequestAfterReady":true} {"readiness":null,"healthReads":1,"tag":"AnalysisProtocolError","code":"unreadable_output","retryable":false,"message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}} {"readiness":null,"sameRequestAfterReady":true} ✓ |adapters| src/ida/repro.test.ts (2 tests) 12ms Test Files 1 passed (1) Tests 2 passed (2) ``` Source-derived public reachability: [`list_procedures` is a registered contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/contracts/officialToolContracts.ts#L120-L124); the [tool handler forwards its operation](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/server/registerEvidenceTools.ts#L88-L105), the [session dispatches to its active client](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/application/binary/BinarySession.ts#L1129-L1151), and [composite routing selects the provider client](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/application/binary/CompositeProvider.ts#L218-L238). The [IDA client observes health before dispatch](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/ida/IdaSessionClient.ts#L295-L313). The recording fixture passed the real startup catalog, owned-worker, path and digest guards in this reproduction. ### Typed outcome or recovery expected A readiness outcome whose code, retryability and remediation describe the state actually observed. A bounded wait expiring while analysis remains incomplete could use a retryable timeout; null readiness should retain its uncertainty. The caller should not be sent to doctor solely because an otherwise readable health result reports incomplete analysis. ### Analysis or suspected cause Source-derived: [the polling loop recognizes only the busy payload](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/ida/IdaBusyHealth.ts#L94-L109). It passes an ok-shaped reply immediately to the separate readiness guard, which throws `AnalysisProtocolError`. That tag [maps to unreadable_output](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorProjection.ts#L242-L253); [retryability follows code/type rules](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorProjection.ts#L111-L117), and the [generic message](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisErrorPresentation.ts#L374-L376) contradicts the demonstrated successful retry. ### Additional context Read-only searches of open and closed issues and PRs found no matching report. The closest report, [issue 1766](https://github.com/morluto/rea/issues/1766), concerns a different `status: "busy"` payload without identity fields; the current busy polling code handles that case. This report isolates the valid ok-shaped reply with false or unknown readiness. It does not assert that `binary_session` executes this guard or independently proves analysis readiness.

bsakaguchi closed this issue

2026-10-31 00:41:29 +00:00

bsakaguchi commented

2026-10-31 00:41:29 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Confirmed — a valid health reply with status "ok" but auto_analysis_ready false/null is a normal not-ready signal, yet it gets projected as unreadable_output with retryable: false, telling callers that retrying the unchanged request won't help. Since readiness is expected to flip to true shortly in headless mode, mapping not-ready (and unknown/null) to a retryable "provider not ready" class — rather than a permanent output failure — matches both the health contract and practical recovery behavior.

@XinwangTao commented on GitHub (Oct 21, 2026): Confirmed — a valid health reply with status "ok" but auto_analysis_ready false/null is a normal not-ready signal, yet it gets projected as unreadable_output with retryable: false, telling callers that retrying the unchanged request won't help. Since readiness is expected to flip to true shortly in headless mode, mapping not-ready (and unknown/null) to a retryable "provider not ready" class — rather than a permanent output failure — matches both the health contract and practical recovery behavior.

bsakaguchi commented

2026-10-31 00:41:29 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

Valid headless health replies with auto_analysis_ready: false now produce a capability-unavailable result with retryable: true and advice to wait for analysis. A null readiness value stays explicitly unknown and non-retryable until readiness is inspected. The regression covers false/null and a successful health check after readiness is restored on the same client.

Implementation: IdaSessionClient.ts.

The relevant regression checks in IdaSessionClient.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. Valid headless health replies with `auto_analysis_ready: false` now produce a capability-unavailable result with `retryable: true` and advice to wait for analysis. A null readiness value stays explicitly unknown and non-retryable until readiness is inspected. The regression covers false/null and a successful health check after readiness is restored on the same client. Implementation: [IdaSessionClient.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/ida/IdaSessionClient.ts#L804). The relevant regression checks in [IdaSessionClient.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/ida/IdaSessionClient.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:09:33 +00:00

[PR #3524] [MERGED] fix(reconstruction): merge repeated observations into one generated obligation #5801

bsakaguchi referenced this issue

2026-10-31 01:09:33 +00:00

[PR #3526] [MERGED] fix(reconstruction): withhold readiness for manifest references to absent Evidence #5803