[GH-ISSUE #8565] [Design] Guide recovery from unsupported provider operations #6652

New issue

Open

opened 2026-11-01 21:12:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:12:58 +00:00

Owner

Originally created by @morluto on GitHub (Nov 1, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8565

Decision needed

How should REA guide an agent from an unsupported operation on the currently selected provider to discovery of another suitable provider, without automatically switching providers or claiming an unverified candidate will work?

Context and evidence

Observed with REA 1662c037b900d0cc12346cffdb32bbbc69e9dc4b, Kuna 1.988, Ghidra 12.1.4, Node 24.22.1 and macOS arm64. On an owned arm64 Mach-O, opening with Kuna and calling read_bytes or search_bytes returned capability_unavailable/unsupported_provider, with details identifying provider_id: kuna, the operation, and operation is not declared by this provider. Both errors' message and remediation said only: This analysis is unavailable for the current target. Choose another analysis or target.

In the same session, binary_session with detail: "complete" advertised Ghidra as an available candidate with read_bytes and search_bytes capabilities. Explicit open_binary of the same path with provider_id: "ghidra" succeeded, and read_bytes returned the expected bytes. The target was unchanged and not executed. Default binary_session omitted candidate capability details, as its contract states. We did not test an autonomous agent, a host without Ghidra, or behavior with unsaved session annotations.

This is a recovery/discoverability question, not a request for automatic fallback. Provider selection and resulting Evidence semantics remain caller-owned, and a candidate's advertised capability does not guarantee this target or operation will succeed.

Unavailable candidates and discovery size

On PR #8567 head 8d1118a49, a real Kuna 1.988 session on macOS arm64 used deliberately nonexistent Ghidra and Hopper paths. In both compact and full stdio MCP output profiles, binary_session(detail: "complete") reported those providers unavailable while separately showing that each declares read_bytes and supports the target format; IDA was unavailable and Kuna did not declare the operation. Thus this local configuration had no candidate satisfying all three conditions. The revised error makes a provider switch conditional, and a failed explicit Ghidra open preserved the active Kuna session and the same inventory Evidence ID. The target hash was unchanged; it was not executed.

The serialized structured binary_session result measured 70,006 UTF-8 bytes in either output profile; its default summary measured 24,089 bytes but omitted per-candidate capability details. Complete candidate metadata occupied 30,599 bytes and the tool-availability section 27,681 bytes in this specific run. These are JSON byte measurements, not model-token counts, latency, or an agent study. This suggests evaluating a bounded operation-specific candidate view that retains readiness, target support and capability as separate facts, without dropping complete diagnostics or automatically choosing a provider. A CLI control found that rea providers and rea capabilities both invoke the same target-free status workflow; each returned target support unknown because no target was open. PR #8567 now points to one command and states this limitation. The existing summary may be sufficient when availability alone rules out every alternative, but that does not establish the general operation-specific case.

Expected interaction with targets

Read existing files and launch the explicitly selected provider when the caller opens it; no additional target effects are proposed by the guidance.

Options and tradeoffs

Consider provider-specific recovery pointing to binary_session with detail: "complete" to inspect candidate capabilities, followed by explicit open_binary on the same selected path and chosen provider when appropriate. Preserve provider identity and warn through existing session contracts where switching affects ephemeral state. Avoid hard-coding Ghidra as available, silently retrying, or recommending a different target when the actionable alternative is a provider.

Verification available so far

I have observed results. One compact MCP workflow and one local configuration; candidate discovery, switching with annotations, other refusal causes, provider startup failure and non-native tools require separate checks.

Success criteria and open questions

Test that an unsupported-provider error tells the caller how to discover applicable alternatives, that no candidate is represented as guaranteed, and that explicit selection and existing Evidence/session semantics survive. Check unavailable, unsupported-target and no-alternative cases without suggesting an unsafe or nonexistent switch.

References

Provider discovery and explicit selection
Related Kuna byte-read gap: #8170; cross-block search opportunity: #8563.

cc @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Current recovery PR on a signature-search task

On #8567 head b282fece7, I exercised real Kuna 1.989 and Ghidra 12.1.4 through stdio MCP on macOS arm64, using a source-owned table-lookup fixture. Both compact and full profiles followed the recovery: Kuna returned an exact-address xref; its unsupported search_bytes response directed candidate discovery; complete binary_session showed Ghidra available, target-supported and declaring search_bytes; explicitly opening Ghidra found the caller-supplied 256-byte table. An exported Evidence bundle retained the prior Kuna xref record with the same Evidence ID and Kuna attribution. The complete structured status was 69,914 JSON bytes in this configuration; this is not a token or latency measurement. The target SHA-256 db652d426f596d311d91d5e20f1e3d714784f0512faa70dca8a2a788d99b15a7 stayed unchanged and it was not executed. The fixture contains a known table and lookup functions, not an AES implementation. This is an explicitly driven positive handoff, not a test of autonomous agent selection or all provider availability states.

Operation-specific discovery sketch, not an implemented API

From a new real Kuna 1.989 session on #8567 head b282fece7, I inspected complete status for search_bytes. Its candidate facts distinguished four cases on this host: Ghidra was available, target-supported and declared the operation available; Hopper was available and target-supported but declared masked search unavailable with a reason; IDA was not configured; and selected Kuna did not declare the operation. With no target open, all candidates' target support was unknown. This is why a proposed filtered view must preserve readiness, target support, declaration and selection independently, including omissions and owner-provided reasons; it must not claim a candidate is guaranteed to work or infer absent target support.

A local post-processing sketch extracting those candidate facts from the actual complete response was 1,324 JSON bytes versus 69,917 bytes for the entire open-session response in that run. The sketch omitted other status facts and diagnostics, coerced some absent optional fields to null, and was not a real MCP tool call; the two byte counts are therefore not equivalent workflows, token counts, or evidence that a production projection is safe or faster. No production contract change is proposed from that measurement alone.

search_bytes maps to a single provider operation in this example. Source inspection shows composed tools such as trace_native_values require multiple operations and may have partial behavior, so filtering by a tool name cannot generally be implemented as a same-named provider capability lookup. Any design should derive requirements from the existing workflow owner, retain unknown and partial modes, and keep full diagnostics available. Validate actual agent tasks and total schema/call cost before adding a new surface.

Stored-pointer task on the 1.990 validation branch

For the owned Mach-O pointer fixture in #8510, both compact and full public MCP sessions using Kuna 1.990 returned capability_unavailable for read_bytes and address_to_file_offset. Each error identified Kuna and the undeclared operation in details, but its user-facing action only said to choose another analysis or target. In separate Ghidra 12.1.4 sessions on the same unchanged target, read_bytes at the slot succeeded and explicitly identified its bytes as initialized Ghidra memory including loader fixups; independent file bytes differed. This supports a provider-discovery handoff for an actual task while preserving the important difference between loaded memory and source bytes. It does not show a successful Ghidra address_to_file_offset call in this control or prove an autonomous agent would select it.

On the uncommitted conflict-resolution experiment described in #8428 (combining #8567 b282fece7 and #8537 b71ee066b), real Ghidra 12.1.4 compact/full stdio MCP returned conditional annotate_native_function and general binary_session guidance for unsupported set_comment at a known function entry. Following the entry-specific route wrote and read back the comment in Ghidra's ephemeral session. After explicitly switching to real Kuna 1.990, the exact annotation Evidence remained in the investigation bundle; this does not restore the live Ghidra edit. The original unexecuted target SHA-256 stayed 496e188170232c15f1ceeebf33165f060c6a31b46be9571b6f49778ef0e8d910.

This verifies that one experimental conflict resolution retains the entry-specific guidance while the navigation alternatives coexist; it is not a merge, arbitrary-address comment support, or an autonomous-agent trial. Final precedence and remaining integration tests remain open in #8428.

Originally created by @morluto on GitHub (Nov 1, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8565 ### Decision needed How should REA guide an agent from an unsupported operation on the currently selected provider to discovery of another suitable provider, without automatically switching providers or claiming an unverified candidate will work? ### Context and evidence Observed with REA `1662c037b900d0cc12346cffdb32bbbc69e9dc4b`, Kuna 1.988, Ghidra 12.1.4, Node 24.22.1 and macOS arm64. On an owned arm64 Mach-O, opening with Kuna and calling `read_bytes` or `search_bytes` returned `capability_unavailable`/`unsupported_provider`, with details identifying `provider_id: kuna`, the operation, and `operation is not declared by this provider`. Both errors' message and remediation said only: `This analysis is unavailable for the current target. Choose another analysis or target.` In the same session, `binary_session` with `detail: "complete"` advertised Ghidra as an available candidate with `read_bytes` and `search_bytes` capabilities. Explicit `open_binary` of the same path with `provider_id: "ghidra"` succeeded, and `read_bytes` returned the expected bytes. The target was unchanged and not executed. Default `binary_session` omitted candidate capability details, as its contract states. We did not test an autonomous agent, a host without Ghidra, or behavior with unsaved session annotations. This is a recovery/discoverability question, not a request for automatic fallback. Provider selection and resulting Evidence semantics remain caller-owned, and a candidate's advertised capability does not guarantee this target or operation will succeed. ### Unavailable candidates and discovery size On PR #8567 head `8d1118a49`, a real Kuna 1.988 session on macOS arm64 used deliberately nonexistent Ghidra and Hopper paths. In both compact and full stdio MCP output profiles, `binary_session(detail: "complete")` reported those providers unavailable while separately showing that each declares `read_bytes` and supports the target format; IDA was unavailable and Kuna did not declare the operation. Thus this local configuration had no candidate satisfying all three conditions. The revised error makes a provider switch conditional, and a failed explicit Ghidra open preserved the active Kuna session and the same inventory Evidence ID. The target hash was unchanged; it was not executed. The serialized structured `binary_session` result measured 70,006 UTF-8 bytes in either output profile; its default summary measured 24,089 bytes but omitted per-candidate capability details. Complete candidate metadata occupied 30,599 bytes and the tool-availability section 27,681 bytes in this specific run. These are JSON byte measurements, not model-token counts, latency, or an agent study. This suggests evaluating a bounded operation-specific candidate view that retains readiness, target support and capability as separate facts, without dropping complete diagnostics or automatically choosing a provider. A CLI control found that `rea providers` and `rea capabilities` both invoke the same target-free status workflow; each returned target support unknown because no target was open. PR #8567 now points to one command and states this limitation. The existing summary may be sufficient when availability alone rules out every alternative, but that does not establish the general operation-specific case. ### Expected interaction with targets Read existing files and launch the explicitly selected provider when the caller opens it; no additional target effects are proposed by the guidance. ### Options and tradeoffs Consider provider-specific recovery pointing to `binary_session` with `detail: "complete"` to inspect candidate capabilities, followed by explicit `open_binary` on the same selected path and chosen provider when appropriate. Preserve provider identity and warn through existing session contracts where switching affects ephemeral state. Avoid hard-coding Ghidra as available, silently retrying, or recommending a different target when the actionable alternative is a provider. ### Verification available so far I have observed results. One compact MCP workflow and one local configuration; candidate discovery, switching with annotations, other refusal causes, provider startup failure and non-native tools require separate checks. ### Success criteria and open questions Test that an unsupported-provider error tells the caller how to discover applicable alternatives, that no candidate is represented as guaranteed, and that explicit selection and existing Evidence/session semantics survive. Check unavailable, unsupported-target and no-alternative cases without suggesting an unsafe or nonexistent switch. ### References - [Provider discovery and explicit selection](https://github.com/morluto/rea/blob/1662c037b900d0cc12346cffdb32bbbc69e9dc4b/docs/mcp-contracts.md#L805-L818) - Related Kuna byte-read gap: #8170; cross-block search opportunity: #8563. cc @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 ### Current recovery PR on a signature-search task On #8567 head `b282fece7`, I exercised real Kuna 1.989 and Ghidra 12.1.4 through stdio MCP on macOS arm64, using a source-owned table-lookup fixture. Both compact and full profiles followed the recovery: Kuna returned an exact-address xref; its unsupported `search_bytes` response directed candidate discovery; complete `binary_session` showed Ghidra available, target-supported and declaring `search_bytes`; explicitly opening Ghidra found the caller-supplied 256-byte table. An exported Evidence bundle retained the prior Kuna xref record with the same Evidence ID and Kuna attribution. The complete structured status was 69,914 JSON bytes in this configuration; this is not a token or latency measurement. The target SHA-256 `db652d426f596d311d91d5e20f1e3d714784f0512faa70dca8a2a788d99b15a7` stayed unchanged and it was not executed. The fixture contains a known table and lookup functions, not an AES implementation. This is an explicitly driven positive handoff, not a test of autonomous agent selection or all provider availability states. ### Operation-specific discovery sketch, not an implemented API From a new real Kuna 1.989 session on #8567 head `b282fece7`, I inspected complete status for `search_bytes`. Its candidate facts distinguished four cases on this host: Ghidra was available, target-supported and declared the operation available; Hopper was available and target-supported but declared masked search unavailable with a reason; IDA was not configured; and selected Kuna did not declare the operation. With no target open, all candidates' target support was `unknown`. This is why a proposed filtered view must preserve readiness, target support, declaration and selection independently, including omissions and owner-provided reasons; it must not claim a candidate is guaranteed to work or infer absent target support. A local post-processing sketch extracting those candidate facts from the actual complete response was 1,324 JSON bytes versus 69,917 bytes for the entire open-session response in that run. The sketch omitted other status facts and diagnostics, coerced some absent optional fields to null, and was not a real MCP tool call; the two byte counts are therefore not equivalent workflows, token counts, or evidence that a production projection is safe or faster. No production contract change is proposed from that measurement alone. `search_bytes` maps to a single provider operation in this example. Source inspection shows composed tools such as `trace_native_values` require multiple operations and may have partial behavior, so filtering by a tool name cannot generally be implemented as a same-named provider capability lookup. Any design should derive requirements from the existing workflow owner, retain unknown and partial modes, and keep full diagnostics available. Validate actual agent tasks and total schema/call cost before adding a new surface. ### Stored-pointer task on the 1.990 validation branch For the owned Mach-O pointer fixture in #8510, both compact and full public MCP sessions using Kuna 1.990 returned `capability_unavailable` for `read_bytes` and `address_to_file_offset`. Each error identified Kuna and the undeclared operation in `details`, but its user-facing action only said to choose another analysis or target. In separate Ghidra 12.1.4 sessions on the same unchanged target, `read_bytes` at the slot succeeded and explicitly identified its bytes as initialized Ghidra memory including loader fixups; independent file bytes differed. This supports a provider-discovery handoff for an actual task while preserving the important difference between loaded memory and source bytes. It does not show a successful Ghidra `address_to_file_offset` call in this control or prove an autonomous agent would select it. ### Combined navigation and entry-comment recovery probe On the uncommitted conflict-resolution experiment described in #8428 (combining #8567 `b282fece7` and #8537 `b71ee066b`), real Ghidra 12.1.4 compact/full stdio MCP returned conditional `annotate_native_function` and general `binary_session` guidance for unsupported `set_comment` at a known function entry. Following the entry-specific route wrote and read back the comment in Ghidra's ephemeral session. After explicitly switching to real Kuna 1.990, the exact annotation Evidence remained in the investigation bundle; this does not restore the live Ghidra edit. The original unexecuted target SHA-256 stayed `496e188170232c15f1ceeebf33165f060c6a31b46be9571b6f49778ef0e8d910`. This verifies that one experimental conflict resolution retains the entry-specific guidance while the navigation alternatives coexist; it is not a merge, arbitrary-address comment support, or an autonomous-agent trial. Final precedence and remaining integration tests remain open in #8428.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:12:58 +00:00

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 00:30:07 +00:00

bsakaguchi added