morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #8226] [Bug] Batch decompilation loses per-function provider failure associations #6500

New issue

Closed

opened 2026-11-01 21:11:42 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8226

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

A mixed batch_decompile reports the selected function whose Kuna project export failed as unreadable_output, with advice to run rea doctor and reason provider returned empty pseudocode. The actual worker-stall limitation is retained only at the batch Evidence level, without an item association. A batch-only Evidence export contained the composed batch record and no per-function provider records or Evidence links in this test.

Reproduction

On a self-built arm64 Mach-O with Kuna 1.986, use an isolated cache and a test wrapper that injects Kuna's test-only stall:0x100000410 and a one-second watchdog for decompile-project with two jobs. REA creates a project export with that entry failed and other entries successful. This instrumentation forces the condition; it is not a claim of a natural stall.
Start a fresh MCP session on the same completed partial cache and call batch_decompile with addresses: ["0x100000430", "0x100000410", "0x100000400"]. The first entry succeeds, the stalled entry reports unreadable_output, and the last entry is an invalid procedure selector. Counts and ordering are correct.
Export the session's Evidence bundle after the batch only. It contains the batch Evidence with the watchdog text as a top-level limitation, no per-function records and an empty evidence_links array. In a separate run, calling procedure_pseudo_code for the failed address produced a separate Evidence record with null result and the watchdog limitation attached to its procedure parameter.

The batch-only run reproduced the mixed result in a second MCP session. NOT_ENOUGH_INFO: a standalone public fixture and test wrapper are not attached, and natural failure frequency is unknown.

Expected behavior

A caller should be able to determine which selected item carries a provider's per-function failure or caveat, and should not receive doctor guidance as the sole item-level explanation for a known watchdog failure. Preserve provider facts and uncertainty; distinguish empty pseudocode with an unknown reason from a provider-reported failure. Consider associating the observation and its limitations with the item or retaining inspectable linked Evidence, while keeping the aggregate limitations and ordered partial results.

Impact and scope

The provider reason survives in the batch as a whole, but its item association is not retained in the tested bundle. The single-item call recovers the association at the cost of another provider query. I did not test multiple different provider failures, Ghidra, CLI batch entry points, or an autonomous agent; their behavior and practical incidence remain unknown. The source binary was not executed and its SHA-256 was unchanged.

REA version

Source revision 53a4ab78451a0654e9c3beb3d43a5e371a288fcc.

Runtime and platform

Node 24.22.1; macOS 26.3 arm64; Kuna 1.986 via an explicitly instrumented test wrapper.

Target and artifact context

Self-built arm64 Mach-O; target SHA-256 ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

For the failed entry, status: "error", code: "unreadable_output", details.reason: "provider returned empty pseudocode"; the batch's limitations separately state Kuna could not decompile this function: worker stalled past the per-function watchdog (1s); the worker was killed. evidence_links: [] in the batch-only exported record.

The directory-export verification already notes that batch output does not retain per-function caveat associations.

Typed outcome or recovery expected

Preserve the per-item provider observation and its recovery-relevant limitation or an explicit link to it, without claiming the provider failure is retryable unless known. A precise representation and bounded large-batch behavior need design and verification.

Analysis or suspected cause

Source-derived: #batchPseudocode receives the simplified result and maps null to AnalysisOutputError; executeComposedWorkflow aggregates successful source limitations by provider and operation. A fix should address the owning observation boundary instead of parsing text in the adapter.

Additional context

Related #8167 concerns explicit recovery from a cached partial Kuna project, and #8181 concerns an omitted batch selection; neither describes per-item association in this batch result. Last known good version: NOT_ENOUGH_INFO.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8226 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior A mixed `batch_decompile` reports the selected function whose Kuna project export failed as `unreadable_output`, with advice to run `rea doctor` and reason `provider returned empty pseudocode`. The actual worker-stall limitation is retained only at the batch Evidence level, without an item association. A batch-only Evidence export contained the composed batch record and no per-function provider records or Evidence links in this test. ### Reproduction 1. On a self-built arm64 Mach-O with Kuna 1.986, use an isolated cache and a test wrapper that injects Kuna's test-only `stall:0x100000410` and a one-second watchdog for `decompile-project` with two jobs. REA creates a project export with that entry failed and other entries successful. This instrumentation forces the condition; it is not a claim of a natural stall. 2. Start a fresh MCP session on the same completed partial cache and call `batch_decompile` with `addresses: ["0x100000430", "0x100000410", "0x100000400"]`. The first entry succeeds, the stalled entry reports `unreadable_output`, and the last entry is an invalid procedure selector. Counts and ordering are correct. 3. Export the session's Evidence bundle after the batch only. It contains the batch Evidence with the watchdog text as a top-level limitation, no per-function records and an empty `evidence_links` array. In a separate run, calling `procedure_pseudo_code` for the failed address produced a separate Evidence record with null result and the watchdog limitation attached to its procedure parameter. The batch-only run reproduced the mixed result in a second MCP session. NOT_ENOUGH_INFO: a standalone public fixture and test wrapper are not attached, and natural failure frequency is unknown. ### Expected behavior A caller should be able to determine which selected item carries a provider's per-function failure or caveat, and should not receive doctor guidance as the sole item-level explanation for a known watchdog failure. Preserve provider facts and uncertainty; distinguish empty pseudocode with an unknown reason from a provider-reported failure. Consider associating the observation and its limitations with the item or retaining inspectable linked Evidence, while keeping the aggregate limitations and ordered partial results. ### Impact and scope The provider reason survives in the batch as a whole, but its item association is not retained in the tested bundle. The single-item call recovers the association at the cost of another provider query. I did not test multiple different provider failures, Ghidra, CLI batch entry points, or an autonomous agent; their behavior and practical incidence remain unknown. The source binary was not executed and its SHA-256 was unchanged. ### REA version Source revision `53a4ab78451a0654e9c3beb3d43a5e371a288fcc`. ### Runtime and platform Node 24.22.1; macOS 26.3 arm64; Kuna 1.986 via an explicitly instrumented test wrapper. ### Target and artifact context Self-built arm64 Mach-O; target SHA-256 `ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d`. ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence For the failed entry, `status: "error"`, `code: "unreadable_output"`, `details.reason: "provider returned empty pseudocode"`; the batch's limitations separately state `Kuna could not decompile this function: worker stalled past the per-function watchdog (1s); the worker was killed`. `evidence_links: []` in the batch-only exported record. The [directory-export verification](https://github.com/morluto/rea/blob/53a4ab78451a0654e9c3beb3d43a5e371a288fcc/docs/verification/native-decompile-directory-2026-10-23.md) already notes that batch output does not retain per-function caveat associations. ### Typed outcome or recovery expected Preserve the per-item provider observation and its recovery-relevant limitation or an explicit link to it, without claiming the provider failure is retryable unless known. A precise representation and bounded large-batch behavior need design and verification. ### Analysis or suspected cause Source-derived: [`#batchPseudocode`](https://github.com/morluto/rea/blob/53a4ab78451a0654e9c3beb3d43a5e371a288fcc/src/application/EnhancedTools.ts#L488) receives the simplified result and maps null to `AnalysisOutputError`; [`executeComposedWorkflow`](https://github.com/morluto/rea/blob/53a4ab78451a0654e9c3beb3d43a5e371a288fcc/src/application/WorkflowEvidence.ts#L114) aggregates successful source limitations by provider and operation. A fix should address the owning observation boundary instead of parsing text in the adapter. ### Additional context Related #8167 concerns explicit recovery from a cached partial Kuna project, and #8181 concerns an omitted batch selection; neither describes per-item association in this batch result. Last known good version: NOT_ENOUGH_INFO.

bsakaguchi

2026-11-01 21:11:42 +00:00

closed this issue
added the
bug

triage
labels

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

A real Ghidra 12.1.4 control on macOS arm64 gives a different outcome from the Kuna null-result case. With an owned 2,000-branch Mach-O and a one-second ghidra.decompile_timeout_seconds, an MCP batch selected a fast function, the slow function, and the fast function again. It returned ordered ok, provider_timeout, ok items with counts 2/1. The timeout item itself retained retryable: true, the effective setting and specific update_analysis_settings guidance. This timeout was a real configured deadline, not the Kuna injected worker stall.

After setting the runtime deadline to 60 seconds, a second batch successfully decompiled the slow and fast functions in the same session. A prior fast-function rename and comment still appeared in its pseudocode. The target digest was unchanged. This is a useful existing recovery path and does not reproduce the missing failure association for typed Ghidra timeouts; it does not establish the behavior of null or other Ghidra outcomes, or of other providers.

The first batch's exported bundle retained the timeout within its item; its composed record had no Evidence links. The successful retry's slow pseudocode was 265,016 characters, and the full SDK response serialized to 576,454 bytes including both text and structured representations. This is one unusually large fixture and a response-byte observation, not a model-token or host-display measurement. The existing tool description already advises selected small batches and explains that complete bodies are retained. No throughput or general timeout-frequency claim is made.

@morluto commented on GitHub (Oct 30, 2026): A real Ghidra 12.1.4 control on macOS arm64 gives a different outcome from the Kuna null-result case. With an owned 2,000-branch Mach-O and a one-second `ghidra.decompile_timeout_seconds`, an MCP batch selected a fast function, the slow function, and the fast function again. It returned ordered `ok`, `provider_timeout`, `ok` items with counts 2/1. The timeout item itself retained `retryable: true`, the effective setting and specific `update_analysis_settings` guidance. This timeout was a real configured deadline, not the Kuna injected worker stall. After setting the runtime deadline to 60 seconds, a second batch successfully decompiled the slow and fast functions in the same session. A prior fast-function rename and comment still appeared in its pseudocode. The target digest was unchanged. This is a useful existing recovery path and does **not** reproduce the missing failure association for typed Ghidra timeouts; it does not establish the behavior of null or other Ghidra outcomes, or of other providers. The first batch's exported bundle retained the timeout within its item; its composed record had no Evidence links. The successful retry's slow pseudocode was 265,016 characters, and the full SDK response serialized to 576,454 bytes including both text and structured representations. This is one unusually large fixture and a response-byte observation, not a model-token or host-display measurement. The existing tool description already advises selected small batches and explains that complete bodies are retained. No throughput or general timeout-frequency claim is made.

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

An additional real Kuna 1.986 control on main e6d061897 exercises a null project result without an injected watchdog failure. The source-owned x86-64 ELF deliberately declares false_func as a function symbol whose bytes are in non-executable data; this is an adversarial fixture, not evidence of natural frequency or runtime behavior. A completed serial Kuna project export lists the entry but provides no body.

A mixed MCP batch_decompile selected false_func, a separately discovered code entry, and an unknown address. The respective outcomes were unreadable_output with doctor advice, success, and invalid selector. The batch-wide limitations included Kuna's explanation that this inventory entry has no exported body, but the first item did not associate it with the entry. A separate procedure_pseudo_code(false_func) returned null with that same limitation on its own Evidence. This is a different provider outcome from the earlier injected worker stall; neither should be relabeled as the other or as proof of runtime behavior.

On the same fixture, real Ghidra's batch returned pseudocode for both selected entries and an error for the invalid address; independent memory and instruction observations in #8168 show why a pseudocode answer alone does not establish executable code. The target digest was unchanged. This control supports distinguishing an observed no-body outcome from unreadable provider output and preserving its item association, while leaving the underlying inventory classification to #8168.

@morluto commented on GitHub (Oct 31, 2026): An additional real Kuna 1.986 control on main `e6d061897` exercises a null project result **without an injected watchdog failure**. The source-owned x86-64 ELF deliberately declares `false_func` as a function symbol whose bytes are in non-executable data; this is an adversarial fixture, not evidence of natural frequency or runtime behavior. A completed serial Kuna project export lists the entry but provides no body. A mixed MCP `batch_decompile` selected `false_func`, a separately discovered code entry, and an unknown address. The respective outcomes were `unreadable_output` with doctor advice, success, and invalid selector. The batch-wide limitations included Kuna's explanation that this inventory entry has no exported body, but the first item did not associate it with the entry. A separate `procedure_pseudo_code(false_func)` returned null with that same limitation on its own Evidence. This is a different provider outcome from the earlier injected worker stall; neither should be relabeled as the other or as proof of runtime behavior. On the same fixture, real Ghidra's batch returned pseudocode for both selected entries and an error for the invalid address; independent memory and instruction observations in [#8168](https://github.com/morluto/rea/issues/8168#issuecomment-6540463299) show why a pseudocode answer alone does not establish executable code. The target digest was unchanged. This control supports distinguishing an observed no-body outcome from unreadable provider output and preserving its item association, while leaving the underlying inventory classification to #8168.

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

An uninstrumented cold-versus-warm batch on a second source-owned Mach-O reproduces the no-body limitation association and generic item-level error; the cold batch also accepts the data symbol as pseudocode before export completes. Details and control: https://github.com/morluto/rea/issues/8168#issuecomment-6542191887.

@morluto commented on GitHub (Oct 31, 2026): An uninstrumented cold-versus-warm batch on a second source-owned Mach-O reproduces the no-body limitation association and generic item-level error; the cold batch also accepts the data symbol as pseudocode before export completes. Details and control: https://github.com/morluto/rea/issues/8168#issuecomment-6542191887.

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

TypeScript SDK mixed-item contract control

I checked a different mixed batch against the live full and compact MCP output-schema profiles on REA b43c329df779708e43ae297f498a844e45d9c923. A TypeScript MCP SDK 2.3.1 client explicitly supplied each tool definition returned by tools/list to callTool, activating its output-schema validator. With real Ghidra 12.1.4 and Kuna 1.986 on a self-built arm64 Mach-O containing overloaded functions, all four provider/profile combinations returned normally: five ordered inputs yielded ok, error, ok, ok, error, with three successes and two failures. The successful result was accepted by each advertised schema; the target hash stayed unchanged and it was not executed.

As a separate tool-error control, the same client received normal isError results for find_xrefs_to_name (ambiguous input on Ghidra and unavailable general name inventory on Kuna). The installed SDK skips output-schema validation for isError responses; their receipt is not proof those errors validate against a success schema. This control exercises ambiguity and a missing selector, not the intentionally stalled Kuna project failure in this issue, and does not establish that schema validation catches the semantic loss of per-item provider limitations described above.

@morluto commented on GitHub (Oct 31, 2026): ### TypeScript SDK mixed-item contract control I checked a different mixed batch against the live full and compact MCP output-schema profiles on REA `b43c329df779708e43ae297f498a844e45d9c923`. A TypeScript MCP SDK 2.3.1 client explicitly supplied each tool definition returned by `tools/list` to `callTool`, activating its output-schema validator. With real Ghidra 12.1.4 and Kuna 1.986 on a self-built arm64 Mach-O containing overloaded functions, all four provider/profile combinations returned normally: five ordered inputs yielded `ok, error, ok, ok, error`, with three successes and two failures. The successful result was accepted by each advertised schema; the target hash stayed unchanged and it was not executed. As a separate tool-error control, the same client received normal `isError` results for `find_xrefs_to_name` (ambiguous input on Ghidra and unavailable general name inventory on Kuna). The installed SDK skips output-schema validation for `isError` responses; their receipt is not proof those errors validate against a success schema. This control exercises ambiguity and a missing selector, **not** the intentionally stalled Kuna project failure in this issue, and does not establish that schema validation catches the semantic loss of per-item provider limitations described above.

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real-provider follow-up for merged #8323 on REA 2d5dec26c31662764c9adfe3ff996174901b8d49, Kuna 1.986, Ghidra 12.1.4 and the previously reported source-owned x86-64 ELF fixture. Real Kuna MCP batch [false_func, 0x1401, 0xdeadbeef] returned error/ok/error in order. The null-pseudocode item now reported capability_unavailable, provider returned null pseudocode, and its own provider caveat that this inventory entry has no exported body; it no longer directed the caller to rea doctor. A separate single-function request confirmed null and the same caveat. The complete batch Evidence survived export and target-free import exactly. SDK validation against the compact advertised success schema passed.

As a separate provider control, real Ghidra returned pseudocode for both selected entries and an error for the invalid address. This is a provider-observation difference, not proof that either provider's function classification is correct. The fixture hash was unchanged and it was not executed. This run used the natural null fixture; the earlier watchdog failure was injected and was not rerun here.

@morluto commented on GitHub (Oct 31, 2026): Real-provider follow-up for merged #8323 on REA `2d5dec26c31662764c9adfe3ff996174901b8d49`, Kuna 1.986, Ghidra 12.1.4 and the previously reported source-owned x86-64 ELF fixture. Real Kuna MCP batch `[false_func, 0x1401, 0xdeadbeef]` returned error/ok/error in order. The null-pseudocode item now reported `capability_unavailable`, `provider returned null pseudocode`, and its own provider caveat that this inventory entry has no exported body; it no longer directed the caller to `rea doctor`. A separate single-function request confirmed null and the same caveat. The complete batch Evidence survived export and target-free import exactly. SDK validation against the compact advertised success schema passed. As a separate provider control, real Ghidra returned pseudocode for both selected entries and an error for the invalid address. This is a provider-observation difference, not proof that either provider's function classification is correct. The fixture hash was unchanged and it was not executed. This run used the natural null fixture; the earlier watchdog failure was injected and was not rerun here.

bsakaguchi commented

2026-11-01 21:11:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Additional real-Kuna control after #8323, on REA 2d5dec26c31662764c9adfe3ff996174901b8d49 and Kuna 1.986: the previously captured deliberately injected per-function worker-stall export now produces a batch with one successful neighbor, the stalled item, and an invalid selector. The stalled item's own limitations include the exact one-second watchdog failure; its error is capability_unavailable / provider returned null pseudocode, and it no longer recommends rea doctor. A separate single-function query from that cache returns null with the same caveat. Both the batch and its item caveat survived export and import into a fresh server unchanged, and the SDK validated the compact success schema.

In a separate server using the explicitly selected previously prepared clean cache, the same target, provider profile and selectors yielded a body for the formerly stalled function; the valid neighbor still succeeded and the invalid selector still failed. The target hash was unchanged and it was not executed. The watchdog fault was injected by a test-only wrapper, so this is not a natural timeout-rate measurement or a demonstration of automatic recovery. #8167 remains the recovery discussion.

@morluto commented on GitHub (Oct 31, 2026): Additional real-Kuna control after #8323, on REA `2d5dec26c31662764c9adfe3ff996174901b8d49` and Kuna 1.986: the previously captured **deliberately injected** per-function worker-stall export now produces a batch with one successful neighbor, the stalled item, and an invalid selector. The stalled item's own `limitations` include the exact one-second watchdog failure; its error is `capability_unavailable` / `provider returned null pseudocode`, and it no longer recommends `rea doctor`. A separate single-function query from that cache returns null with the same caveat. Both the batch and its item caveat survived export and import into a fresh server unchanged, and the SDK validated the compact success schema. In a separate server using the explicitly selected previously prepared clean cache, the same target, provider profile and selectors yielded a body for the formerly stalled function; the valid neighbor still succeeded and the invalid selector still failed. The target hash was unchanged and it was not executed. The watchdog fault was injected by a test-only wrapper, so this is not a natural timeout-rate measurement or a demonstration of automatic recovery. #8167 remains the recovery discussion.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6500

No description provided.

Rows
Columns