morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5446] [Bug] Ambiguous provider selection offers providers already rejected as unconfigured #3304

New issue

Closed

opened 2026-10-31 00:38:58 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:38:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5446

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Provider or capability discovery

Observed behavior

node scripts/rea.mjs analyze "$VERIFY_ROOT/thin-macho" --json fails with an ambiguous automatic provider selection and offers ghidra, hopper, ida, and kuna in details.candidate_ids. The same error rejects Ghidra and Kuna as not_configured. Following the documented recovery by retrying either rejected ID with --provider fails again with provider_unavailable before analysis.

An independent verifier executed the isolated CLI commands below once each. Registered stdio MCP open_binary also reproduced the ambiguity and rejected choices; explicit Hopper and IDA opens succeeded, and binary_session confirmed each requested binding before successful close. These observations establish provider selection and session binding only, not live engine readiness or analysis success.

Reproduction

Prerequisites: the built source baseline below on macOS, Hopper's launcher discoverable, and the Apple C compiler available. Ghidra and Kuna are unconfigured. From the checkout root, the independent verifier ran:

VERIFY_ROOT="$(mktemp -d)"
mkdir -p "$VERIFY_ROOT/home" "$VERIFY_ROOT/config" "$VERIFY_ROOT/data" \
  "$VERIFY_ROOT/cache" "$VERIFY_ROOT/state" "$VERIFY_ROOT/out"
printf '%s\n' '{"command":"true","mode":"attached"}' > "$VERIFY_ROOT/config/ida-registration.json"
/usr/bin/clang tests/fixtures/native-macos/native-macho-header/fixture.c -o "$VERIFY_ROOT/thin-macho"
run_rea() {
  env -i PATH="$PATH" TMPDIR="$VERIFY_ROOT/cache" \
    HOME="$VERIFY_ROOT/home" XDG_CONFIG_HOME="$VERIFY_ROOT/config" \
    XDG_DATA_HOME="$VERIFY_ROOT/data" XDG_CACHE_HOME="$VERIFY_ROOT/cache" \
    XDG_STATE_HOME="$VERIFY_ROOT/state" REA_ANALYSIS_PROVIDER=auto \
    REA_LOG_LEVEL=silent REA_IDA_MCP_CONFIG="$VERIFY_ROOT/config/ida-registration.json" \
    node scripts/rea.mjs "$@"
}
run_rea doctor --provider ida --json > "$VERIFY_ROOT/out/doctor.json"
run_rea analyze "$VERIFY_ROOT/thin-macho" --json > "$VERIFY_ROOT/out/ambiguous.json"
run_rea analyze "$VERIFY_ROOT/thin-macho" --provider ghidra --json > "$VERIFY_ROOT/out/ghidra.json"
run_rea analyze "$VERIFY_ROOT/thin-macho" --provider kuna --json > "$VERIFY_ROOT/out/kuna.json"
cat "$VERIFY_ROOT/out/ambiguous.json" "$VERIFY_ROOT/out/ghidra.json" "$VERIFY_ROOT/out/kuna.json"

Doctor exits 0 with valid IDA registration; the three analyze calls each exit 1. The attached registration is intentionally admitted without starting its command or contacting an upstream IDA engine. The generated directory retains the outputs for inspection.

Expected behavior

docs/cli.md:170-190 says automatic selection uses the single available supporting provider and tells ambiguous-error callers to choose from details.candidate_ids. docs/installation.md:342-349 repeats that recovery.

For this case, the recovery list should contain only Hopper and IDA, which passed target selection/profile admission. Ghidra and Kuna should remain visible in rejection diagnostics without being offered as immediate choices. Genuine ambiguity should remain an error. Passing selection does not promise that later engine startup or analysis will succeed.

Impact and scope

The documented recovery offers unusable choices and can immediately cause another selection failure. This was observed for one thin arm64 Mach-O with two admitted providers and two unconfigured providers; broader platform and provider combinations were not tested. No target corruption or security boundary crossing was observed.

The bounded P2 impact has a practical workaround: exclude IDs in details.rejections and explicitly select an admitted provider. The independent MCP checks demonstrated selection/binding for both Hopper and IDA. With the IDA registration removed and a fresh MCP server, automatic selection opened the same target with Hopper as auto-single-candidate.

REA version

6.7.0; source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8 (built runtime).

Runtime and platform

Node.js v26.12.0; macOS 27.0.1 arm64. Hopper's installed launcher was discoverable. NOT_ENOUGH_INFO: the verifier did not record the Hopper version. No live IDA service was used; Ghidra and Kuna were unconfigured.

Target and artifact context

Thin arm64 Mach-O executable compiled from the repository's public tests/fixtures/native-macos/native-macho-header/fixture.c. Home, all XDG roots, registration, target, temporary runtime storage, and captured output were isolated. No deep analysis operation or provider database acquisition was needed.

Interface

CLI

Earliest failure stage

Setup or provider admission

Relevant output or evidence

Automatic CLI selection returned these relevant fields (diagnostics shortened):

{"code":"capability_unavailable","details":{"selection_reason":"ambiguous","requested_provider_id":"auto","candidate_ids":["ghidra","hopper","ida","kuna"],"rejections":[{"provider_id":"ghidra","code":"not_configured"},{"provider_id":"kuna","code":"not_configured"}]}}

Both explicit CLI retries returned code: provider_unavailable, details.selection_reason: provider_unavailable, and the rejected ID in details.requested_provider_id, with exit 1 before analysis.

Submission-time source check: current upstream main 052956af3416fa01d93837e54fdec3ad2739f6d1 has no changes since the tested baseline to src/application/binary/AnalysisProviderRegistry.ts, docs/cli.md, or docs/installation.md. No additional runtime reproduction is claimed at that newer revision.

Typed outcome or recovery expected

Keep capability_unavailable with selection_reason: ambiguous, offer only the admitted providers in candidate_ids, and preserve the unavailable-provider rejections. Each offered ID should pass provider selection when retried explicitly. Retain the observed single-provider automatic-selection behavior.

Analysis or suspected cause

Source-derived: src/application/binary/AnalysisProviderRegistry.ts:640-655 filters evaluations to usable profiles to detect ambiguity, but passes all evaluated statuses into selectionError. At :755-769, that helper derives candidate IDs from the entire status list, including rejected providers.

Additional context

Prepared with AI assistance and independently reproduced through CLI and registered stdio MCP. Successful native analysis and a live upstream IDA connection were not tested. NOT_ENOUGH_INFO: no last-known-good version or regression boundary was established.

Related context: #2417 and PR #3090 discuss changing automatic-selection policy; this report preserves the existing ambiguity behavior and concerns its documented recovery choices. #4555 concerns unrelated deep-provider admission blocking static native helper commands, rather than which IDs the recovery list offers.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5446 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Provider or capability discovery ### Observed behavior `node scripts/rea.mjs analyze "$VERIFY_ROOT/thin-macho" --json` fails with an ambiguous automatic provider selection and offers `ghidra`, `hopper`, `ida`, and `kuna` in `details.candidate_ids`. The same error rejects Ghidra and Kuna as `not_configured`. Following the documented recovery by retrying either rejected ID with `--provider` fails again with `provider_unavailable` before analysis. An independent verifier executed the isolated CLI commands below once each. Registered stdio MCP `open_binary` also reproduced the ambiguity and rejected choices; explicit Hopper and IDA opens succeeded, and `binary_session` confirmed each requested binding before successful close. These observations establish provider selection and session binding only, not live engine readiness or analysis success. ### Reproduction Prerequisites: the built source baseline below on macOS, Hopper's launcher discoverable, and the Apple C compiler available. Ghidra and Kuna are unconfigured. From the checkout root, the independent verifier ran: ```sh VERIFY_ROOT="$(mktemp -d)" mkdir -p "$VERIFY_ROOT/home" "$VERIFY_ROOT/config" "$VERIFY_ROOT/data" \ "$VERIFY_ROOT/cache" "$VERIFY_ROOT/state" "$VERIFY_ROOT/out" printf '%s\n' '{"command":"true","mode":"attached"}' > "$VERIFY_ROOT/config/ida-registration.json" /usr/bin/clang tests/fixtures/native-macos/native-macho-header/fixture.c -o "$VERIFY_ROOT/thin-macho" run_rea() { env -i PATH="$PATH" TMPDIR="$VERIFY_ROOT/cache" \ HOME="$VERIFY_ROOT/home" XDG_CONFIG_HOME="$VERIFY_ROOT/config" \ XDG_DATA_HOME="$VERIFY_ROOT/data" XDG_CACHE_HOME="$VERIFY_ROOT/cache" \ XDG_STATE_HOME="$VERIFY_ROOT/state" REA_ANALYSIS_PROVIDER=auto \ REA_LOG_LEVEL=silent REA_IDA_MCP_CONFIG="$VERIFY_ROOT/config/ida-registration.json" \ node scripts/rea.mjs "$@" } run_rea doctor --provider ida --json > "$VERIFY_ROOT/out/doctor.json" run_rea analyze "$VERIFY_ROOT/thin-macho" --json > "$VERIFY_ROOT/out/ambiguous.json" run_rea analyze "$VERIFY_ROOT/thin-macho" --provider ghidra --json > "$VERIFY_ROOT/out/ghidra.json" run_rea analyze "$VERIFY_ROOT/thin-macho" --provider kuna --json > "$VERIFY_ROOT/out/kuna.json" cat "$VERIFY_ROOT/out/ambiguous.json" "$VERIFY_ROOT/out/ghidra.json" "$VERIFY_ROOT/out/kuna.json" ``` Doctor exits 0 with valid IDA registration; the three analyze calls each exit 1. The attached registration is intentionally admitted without starting its command or contacting an upstream IDA engine. The generated directory retains the outputs for inspection. ### Expected behavior `docs/cli.md:170-190` says automatic selection uses the single available supporting provider and tells ambiguous-error callers to choose from `details.candidate_ids`. `docs/installation.md:342-349` repeats that recovery. For this case, the recovery list should contain only Hopper and IDA, which passed target selection/profile admission. Ghidra and Kuna should remain visible in rejection diagnostics without being offered as immediate choices. Genuine ambiguity should remain an error. Passing selection does not promise that later engine startup or analysis will succeed. ### Impact and scope The documented recovery offers unusable choices and can immediately cause another selection failure. This was observed for one thin arm64 Mach-O with two admitted providers and two unconfigured providers; broader platform and provider combinations were not tested. No target corruption or security boundary crossing was observed. The bounded P2 impact has a practical workaround: exclude IDs in `details.rejections` and explicitly select an admitted provider. The independent MCP checks demonstrated selection/binding for both Hopper and IDA. With the IDA registration removed and a fresh MCP server, automatic selection opened the same target with Hopper as `auto-single-candidate`. ### REA version 6.7.0; source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8` (built runtime). ### Runtime and platform Node.js v26.12.0; macOS 27.0.1 arm64. Hopper's installed launcher was discoverable. NOT_ENOUGH_INFO: the verifier did not record the Hopper version. No live IDA service was used; Ghidra and Kuna were unconfigured. ### Target and artifact context Thin arm64 Mach-O executable compiled from the repository's public `tests/fixtures/native-macos/native-macho-header/fixture.c`. Home, all XDG roots, registration, target, temporary runtime storage, and captured output were isolated. No deep analysis operation or provider database acquisition was needed. ### Interface CLI ### Earliest failure stage Setup or provider admission ### Relevant output or evidence Automatic CLI selection returned these relevant fields (diagnostics shortened): ```json {"code":"capability_unavailable","details":{"selection_reason":"ambiguous","requested_provider_id":"auto","candidate_ids":["ghidra","hopper","ida","kuna"],"rejections":[{"provider_id":"ghidra","code":"not_configured"},{"provider_id":"kuna","code":"not_configured"}]}} ``` Both explicit CLI retries returned `code: provider_unavailable`, `details.selection_reason: provider_unavailable`, and the rejected ID in `details.requested_provider_id`, with exit 1 before analysis. Submission-time source check: current upstream main `052956af3416fa01d93837e54fdec3ad2739f6d1` has no changes since the tested baseline to `src/application/binary/AnalysisProviderRegistry.ts`, `docs/cli.md`, or `docs/installation.md`. No additional runtime reproduction is claimed at that newer revision. ### Typed outcome or recovery expected Keep `capability_unavailable` with `selection_reason: ambiguous`, offer only the admitted providers in `candidate_ids`, and preserve the unavailable-provider rejections. Each offered ID should pass provider selection when retried explicitly. Retain the observed single-provider automatic-selection behavior. ### Analysis or suspected cause Source-derived: `src/application/binary/AnalysisProviderRegistry.ts:640-655` filters evaluations to `usable` profiles to detect ambiguity, but passes all evaluated statuses into `selectionError`. At `:755-769`, that helper derives candidate IDs from the entire status list, including rejected providers. ### Additional context Prepared with AI assistance and independently reproduced through CLI and registered stdio MCP. Successful native analysis and a live upstream IDA connection were not tested. NOT_ENOUGH_INFO: no last-known-good version or regression boundary was established. Related context: #2417 and PR #3090 discuss changing automatic-selection policy; this report preserves the existing ambiguity behavior and concerns its documented recovery choices. #4555 concerns unrelated deep-provider admission blocking static native helper commands, rather than which IDs the recovery list offers.

bsakaguchi closed this issue

2026-10-31 00:38:59 +00:00

bsakaguchi commented

2026-10-31 00:38:59 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

Automatic ambiguity now passes only usable/admitted statuses as candidate choices and passes all statuses separately for rejection diagnostics. selectionError no longer derives choices from all evaluated statuses. The regression includes two admitted providers plus unconfigured Ghidra/Kuna, expects only admitted IDs as candidate_ids and retains not_configured rejections.

Implementation: AnalysisProviderRegistry.ts, AnalysisProviderRegistry.ts.

The relevant regression checks in AnalysisProviderRegistry.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. Automatic ambiguity now passes only usable/admitted statuses as candidate choices and passes all statuses separately for rejection diagnostics. selectionError no longer derives choices from all evaluated statuses. The regression includes two admitted providers plus unconfigured Ghidra/Kuna, expects only admitted IDs as candidate_ids and retains not_configured rejections. Implementation: [AnalysisProviderRegistry.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/binary/AnalysisProviderRegistry.ts#L705), [AnalysisProviderRegistry.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/binary/AnalysisProviderRegistry.ts#L860). The relevant regression checks in [AnalysisProviderRegistry.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/binary/AnalysisProviderRegistry.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:08:51 +00:00

[PR #3304] [CLOSED] test(setup): use verified commands in positive uninstall fixtures #5714

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3304

No description provided.

Rows
Columns