morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8289] [Design] Surface ephemeral annotation loss when switching analysis providers #6527

New issue

Open

opened 2026-11-01 21:11:56 +00:00 by bsakaguchi · 8 comments

bsakaguchi commented

2026-11-01 21:11:56 +00:00

Owner

Originally created by @morluto on GitHub (Oct 31, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8289

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

How should an agent discover that explicitly switching providers will retire an edited ephemeral Ghidra database and discard its live annotations, including when the next provider later fails on first use?

Context and evidence

Real MCP on macOS arm64, Ghidra 12.1.4, Kuna 1.986 and REA b43c329:

Open a self-built, unexecuted Mach-O in Ghidra; rename _evaluate to audit_switch_test using annotate_native_function. The renamed selector works.
Reopen the same target with Ghidra. The renamed selector still works.
Open the same target explicitly with Kuna, then reopen with Ghidra. Both opens succeed. The renamed selector is now unknown and _evaluate works again. The switch response contained the target identity, with no transition-specific annotation-loss notice.
Before closing, get_evidence_bundle(detail:"summary") still listed the completed annotation record. It is historical Evidence, not a restored edited database.

The annotation tool description and its Evidence already say edits are ephemeral and discarded on close; open_binary says same-target/same-profile reopen retains edits. The question is whether the fact that a provider switch closes that database is sufficiently visible at the decision point. In this run, binary_session(detail:"complete") after the rename exposed no field identifying unsaved annotation state.

Two controlled failure boundaries: if a test-only Kuna wrapper fails its initial --version probe, the switch returns an error and the Ghidra rename remains available. If the wrapper allows the first version probe but fails the next one, open_binary succeeds; the first Kuna operation fails, and reopening Ghidra restores the original name rather than the rename. This is fault injection, not a claim about natural failure frequency. It is not a failed open_binary restoration test.

Source inspection shows a different-target/profile transition retires the previous client; the relevant source was unchanged at main 23de10b. The target SHA-256 remained ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d.

Expected interaction with targets

Read an existing binary, use existing ephemeral Ghidra metadata edits and provider switching. No target execution or mutation is proposed.

Boundary details (optional)

Keep same-profile reopen semantics, caller-selected provider and evidence history. Do not claim an Evidence bundle or immutable snapshot restores a live edited project. Avoid imposing an approval workflow without a product decision. Existing project sessions have different read-only semantics.

Options and tradeoffs

Consider accurately surfacing edit lifetime and impending retirement in session state and the open contract, a transition result describing the retired session, or a separate non-destructive way to retain edited work. Compare these with #8165's checkpoint and #6405's annotation export/replay proposals. A post-switch notice alone cannot prevent a surprise; an implicit provider switch or silent replay would need its own semantics and authority review.

Verification available so far

I have a prototype or observed results.

Success criteria and open questions

Exercise same-profile reopen, successful provider and target switches after edits, failure before and after retirement, cancellation, and cleanup failure using real CLI/MCP and controlled boundaries as applicable. Check that the user can identify what remains recoverable and that any prior Evidence remains attributable. No autonomous agent-selection study has been run.

References

Kuna 1.990 provider-switch control

On the 1.990 validation branch and macOS arm64, separate compact/full MCP sessions opened an owned arm64 Mach-O in real Ghidra 12.1.4 and renamed its function at 0x100000328 to audit_switch_1990 with set_address_name. Reopening the same target with Ghidra preserved the rename. Explicitly switching to real Kuna 1.990 and then back to Ghidra returned successful open responses; the renamed selector no longer resolved, while the original _audit_value selector resolved to the same address. The provider-switch result contained target identity and no transition-specific edit-loss notice. The original rename Evidence remained exactly present in the retained bundle. The target SHA-256 0e0ecf82c96e4918ee2ca1287dc25513d8fd13d5ab7fa5b98db00ad342f03930 remained unchanged. This repeats the known ephemeral-session behavior on one fixture and the newer provider version; it does not establish agent behavior, cancellation or a durable edited-database recovery.

Invalid target switch control

On the same Ghidra 12.1.4 / Kuna 1.990 validation setup, separate compact and full MCP sessions renamed 0x100000328 to audit_switch_1990, then attempted to open a nonexistent target with Kuna selected. open_binary returned target_unavailable with a path-specific recovery message, and the renamed Ghidra selector still resolved afterward in each session. The source target SHA-256 stayed unchanged. This tests a target-validation refusal before retirement, not cancellation, cleanup failure, or failures after a provider has been replaced.

Originally created by @morluto on GitHub (Oct 31, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8289 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed How should an agent discover that explicitly switching providers will retire an edited ephemeral Ghidra database and discard its live annotations, including when the next provider later fails on first use? ### Context and evidence Real MCP on macOS arm64, Ghidra 12.1.4, Kuna 1.986 and REA `b43c329`: 1. Open a self-built, unexecuted Mach-O in Ghidra; rename `_evaluate` to `audit_switch_test` using `annotate_native_function`. The renamed selector works. 2. Reopen the same target with Ghidra. The renamed selector still works. 3. Open the same target explicitly with Kuna, then reopen with Ghidra. Both opens succeed. The renamed selector is now unknown and `_evaluate` works again. The switch response contained the target identity, with no transition-specific annotation-loss notice. 4. Before closing, `get_evidence_bundle(detail:"summary")` still listed the completed annotation record. It is historical Evidence, not a restored edited database. The annotation tool description and its Evidence already say edits are ephemeral and discarded on close; `open_binary` says same-target/same-profile reopen retains edits. The question is whether the fact that a provider switch *closes that database* is sufficiently visible at the decision point. In this run, `binary_session(detail:"complete")` after the rename exposed no field identifying unsaved annotation state. Two controlled failure boundaries: if a test-only Kuna wrapper fails its initial `--version` probe, the switch returns an error and the Ghidra rename remains available. If the wrapper allows the first version probe but fails the next one, `open_binary` succeeds; the first Kuna operation fails, and reopening Ghidra restores the original name rather than the rename. This is fault injection, not a claim about natural failure frequency. It is not a failed `open_binary` restoration test. Source inspection shows a different-target/profile transition retires the previous client; the relevant source was unchanged at main `23de10b`. The target SHA-256 remained `ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d`. ### Expected interaction with targets Read an existing binary, use existing ephemeral Ghidra metadata edits and provider switching. No target execution or mutation is proposed. ### Boundary details (optional) Keep same-profile reopen semantics, caller-selected provider and evidence history. Do not claim an Evidence bundle or immutable snapshot restores a live edited project. Avoid imposing an approval workflow without a product decision. Existing project sessions have different read-only semantics. ### Options and tradeoffs Consider accurately surfacing edit lifetime and impending retirement in session state and the open contract, a transition result describing the retired session, or a separate non-destructive way to retain edited work. Compare these with #8165's checkpoint and #6405's annotation export/replay proposals. A post-switch notice alone cannot prevent a surprise; an implicit provider switch or silent replay would need its own semantics and authority review. ### Verification available so far I have a prototype or observed results. ### Success criteria and open questions Exercise same-profile reopen, successful provider and target switches after edits, failure before and after retirement, cancellation, and cleanup failure using real CLI/MCP and controlled boundaries as applicable. Check that the user can identify what remains recoverable and that any prior Evidence remains attributable. No autonomous agent-selection study has been run. ### References Related: https://github.com/morluto/rea/issues/8165 and https://github.com/morluto/rea/issues/6405 ### Kuna 1.990 provider-switch control On the 1.990 validation branch and macOS arm64, separate compact/full MCP sessions opened an owned arm64 Mach-O in real Ghidra 12.1.4 and renamed its function at `0x100000328` to `audit_switch_1990` with `set_address_name`. Reopening the same target with Ghidra preserved the rename. Explicitly switching to real Kuna 1.990 and then back to Ghidra returned successful open responses; the renamed selector no longer resolved, while the original `_audit_value` selector resolved to the same address. The provider-switch result contained target identity and no transition-specific edit-loss notice. The original rename Evidence remained exactly present in the retained bundle. The target SHA-256 `0e0ecf82c96e4918ee2ca1287dc25513d8fd13d5ab7fa5b98db00ad342f03930` remained unchanged. This repeats the known ephemeral-session behavior on one fixture and the newer provider version; it does not establish agent behavior, cancellation or a durable edited-database recovery. ### Invalid target switch control On the same Ghidra 12.1.4 / Kuna 1.990 validation setup, separate compact and full MCP sessions renamed `0x100000328` to `audit_switch_1990`, then attempted to open a nonexistent target with Kuna selected. `open_binary` returned `target_unavailable` with a path-specific recovery message, and the renamed Ghidra selector still resolved afterward in each session. The source target SHA-256 stayed unchanged. This tests a target-validation refusal before retirement, not cancellation, cleanup failure, or failures after a provider has been replaced.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:56 +00:00

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

For maintainer visibility: @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

@morluto commented on GitHub (Oct 31, 2026): For maintainer visibility: @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Controlled failed-open restoration follow-up, separate from the earlier first-use failure: in a disposable copy of the built MCP runtime, a one-line test injection returned a tagged cancellation at the replacement client's health check, after REA retired the previous client. The public MCP session still used real Ghidra 12.1.4: the rename was observed before the switch and after same-profile reopen. The Kuna switch returned cancelled with cleanup:complete; binary_session then reported the original Ghidra binding open with no cleanup pending. But the renamed selector was no longer found and the original name was present. The completed annotation record remained in retained Evidence, and the target hash stayed unchanged.

This tests the restoration path under an injected transition failure, not a naturally occurring cancellation or a transport-cancellation race. The injected runtime is not production code. An earlier setup attempt failed to load Ghidra scripts due to a symlinked test-runtime path and is excluded; copying the bridge scripts into that disposable runtime allowed the real annotation to complete.

Source inspection: BinarySession.#restore creates a fresh prior-provider client, while the existing synthetic composition test closes a failed candidate and reopens the previous target checks target/client lifecycle, not preservation of metadata within an edited real Ghidra session. This makes the distinction between reopening the target and restoring the edited state concrete. A regression should preserve that distinction, report the actual transition outcome, and retain the existing cancellation and cleanup guarantees; it should not infer a safe editable checkpoint from an Evidence bundle.

@morluto commented on GitHub (Oct 31, 2026): Controlled *failed-open restoration* follow-up, separate from the earlier first-use failure: in a disposable copy of the built MCP runtime, a one-line test injection returned a tagged cancellation at the replacement client's health check, **after** REA retired the previous client. The public MCP session still used real Ghidra 12.1.4: the rename was observed before the switch and after same-profile reopen. The Kuna switch returned `cancelled` with `cleanup:complete`; `binary_session` then reported the original Ghidra binding open with no cleanup pending. But the renamed selector was no longer found and the original name was present. The completed annotation record remained in retained Evidence, and the target hash stayed unchanged. This tests the restoration path under an injected transition failure, not a naturally occurring cancellation or a transport-cancellation race. The injected runtime is not production code. An earlier setup attempt failed to load Ghidra scripts due to a symlinked test-runtime path and is excluded; copying the bridge scripts into that disposable runtime allowed the real annotation to complete. Source inspection: `BinarySession.#restore` creates a fresh prior-provider client, while the existing synthetic composition test `closes a failed candidate and reopens the previous target` checks target/client lifecycle, not preservation of metadata within an edited real Ghidra session. This makes the distinction between reopening the target and restoring the edited state concrete. A regression should preserve that distinction, report the actual transition outcome, and retain the existing cancellation and cleanup guarantees; it should not infer a safe editable checkpoint from an Evidence bundle.

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Evidence-retrieval control: on both the unmodified successful provider switch and the separately injected failed-open restoration, analyze_function({procedure:"0x100000430",detail:"complete"}) returned a renamed dossier before retirement and the original _evaluate dossier afterward. The post-restoration dossier reused the same Evidence ID as the pre-edit observation; the edited dossier had a different ID. inspect_analysis_view using the edited dossier's retained ID still returned the renamed pseudocode. get_evidence_bundle(detail:"summary") listed both names and IDs for the same subject/provider/profile/address, without an explicit current-vs-historical flag in these rows.

This is useful preservation, not a stale live dossier on this fixture; the requested current read reflected the fresh Ghidra session. It does leave the caller to retain the sequence or re-query when deciding which observation reflects the current live database. The two runs closed successfully and the target hash stayed unchanged. The failed-open control used the test-only injected cancellation described above; the successful-switch control used the unmodified runtime.

@morluto commented on GitHub (Oct 31, 2026): Evidence-retrieval control: on both the unmodified successful provider switch and the separately injected failed-open restoration, `analyze_function({procedure:"0x100000430",detail:"complete"})` returned a renamed dossier before retirement and the original `_evaluate` dossier afterward. The post-restoration dossier reused the same Evidence ID as the pre-edit observation; the edited dossier had a different ID. `inspect_analysis_view` using the edited dossier's retained ID still returned the renamed pseudocode. `get_evidence_bundle(detail:"summary")` listed both names and IDs for the same subject/provider/profile/address, without an explicit current-vs-historical flag in these rows. This is useful preservation, not a stale live dossier on this fixture; the requested current read reflected the fresh Ghidra session. It does leave the caller to retain the sequence or re-query when deciding which observation reflects the current live database. The two runs closed successfully and the target hash stayed unchanged. The failed-open control used the test-only injected cancellation described above; the successful-switch control used the unmodified runtime.

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Agent guidance control: the packaged Function annotations in Ghidra section says changes are session-scoped and later MCP calls observe them until close; it also distinguishes annotation Evidence from comparison inputs and tells callers to export needed Evidence. In public CLI/MCP guidance search, the query Ghidra annotations finds that section first. The section does not explicitly say that an open_binary provider switch retires the active database. A query phrased switch provider annotations instead matches a broad parent whose terms occur under different subsections; see #8290 for the separate search-design question. This is guidance/source inspection and scripted retrieval, not an autonomous agent trial.

@morluto commented on GitHub (Oct 31, 2026): Agent guidance control: the packaged `Function annotations in Ghidra` section says changes are session-scoped and later MCP calls observe them until close; it also distinguishes annotation Evidence from comparison inputs and tells callers to export needed Evidence. In public CLI/MCP guidance search, the query `Ghidra annotations` finds that section first. The section does not explicitly say that an `open_binary` provider switch retires the active database. A query phrased `switch provider annotations` instead matches a broad parent whose terms occur under different subsections; see #8290 for the separate search-design question. This is guidance/source inspection and scripted retrieval, not an autonomous agent trial.

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Manual recovery control, including an omitted name, regular-comment clear and inline comment, is recorded in https://github.com/morluto/rea/issues/6405#issuecomment-6541990498. It required selecting a specific retained record and explicitly reapplying its complete readback; this is not automatic replay or a general checkpoint.

@morluto commented on GitHub (Oct 31, 2026): Manual recovery control, including an omitted name, regular-comment clear and inline comment, is recorded in https://github.com/morluto/rea/issues/6405#issuecomment-6541990498. It required selecting a specific retained record and explicitly reapplying its complete readback; this is not automatic replay or a general checkpoint.

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real MCP transport-cancellation control at the retirement boundary: a disposable copy of the built MCP runtime added only a test barrier immediately after the prior provider client successfully closed and before the existing cancellation check. A real Ghidra 12.1.4 session renamed _evaluate; the TypeScript MCP SDK then requested an explicit switch to real Kuna 1.986. Once the barrier confirmed retirement, the SDK aborted the request. The SDK rejected with the caller's cancellation and the barrier independently observed the server request signal as aborted. REA's existing cancellation/restoration code then ran; binary_session reported Ghidra open and cleanup_pending:false, the renamed selector was absent, _evaluate resolved, one annotation Evidence record was still retained, and close_binary succeeded.

A separate barrier control released the same boundary without cancellation: the switch succeeded and switching back also returned the original name, as expected for an ephemeral database. Both used the same source-owned Mach-O; its SHA-256 stayed unchanged. The barrier changes scheduling to make this exact point reproducible, so this is not a frequency measurement, a claim about cancellation during other cleanup stages, or an unmodified-runtime race reproduction. Unlike the previous injected error, this test sent a real SDK/MCP cancellation and observed the server-side signal. The SDK rejection itself does not expose the server's final cancelled response.

@morluto commented on GitHub (Oct 31, 2026): Real MCP transport-cancellation control at the retirement boundary: a disposable copy of the built MCP runtime added only a test barrier immediately after the prior provider client successfully closed and before the existing cancellation check. A real Ghidra 12.1.4 session renamed `_evaluate`; the TypeScript MCP SDK then requested an explicit switch to real Kuna 1.986. Once the barrier confirmed retirement, the SDK aborted the request. The SDK rejected with the caller's cancellation and the barrier independently observed the server request signal as aborted. REA's existing cancellation/restoration code then ran; `binary_session` reported Ghidra open and `cleanup_pending:false`, the renamed selector was absent, `_evaluate` resolved, one annotation Evidence record was still retained, and `close_binary` succeeded. A separate barrier control released the same boundary without cancellation: the switch succeeded and switching back also returned the original name, as expected for an ephemeral database. Both used the same source-owned Mach-O; its SHA-256 stayed unchanged. The barrier changes scheduling to make this exact point reproducible, so this is not a frequency measurement, a claim about cancellation during other cleanup stages, or an unmodified-runtime race reproduction. Unlike the previous injected error, this test sent a real SDK/MCP cancellation and observed the server-side signal. The SDK rejection itself does not expose the server's final cancelled response.

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Positive boundary control: on a source-owned RV64 ELF in real REA b43c329 / Ghidra 12.1.4 MCP, I renamed a function in a fresh ephemeral session and then requested the same target with Kuna, which is not admitted for RISC-V. open_binary returned capability_unavailable; afterward the Ghidra session remained open with no cleanup pending, the edited selector still resolved, and a typed global READ remained accessible. The target hash stayed unchanged; the session closed normally.

This is a target-admission rejection before a successful provider transition, not evidence that edits survive the successful Ghidra-to-Kuna switch reported here. It is a useful preservation control for any transition/recovery regression. More context: #8195.

@morluto commented on GitHub (Oct 31, 2026): Positive boundary control: on a source-owned RV64 ELF in real REA `b43c329` / Ghidra 12.1.4 MCP, I renamed a function in a fresh ephemeral session and then requested the same target with Kuna, which is not admitted for RISC-V. `open_binary` returned `capability_unavailable`; afterward the Ghidra session remained open with no cleanup pending, the edited selector still resolved, and a typed global READ remained accessible. The target hash stayed unchanged; the session closed normally. This is a target-admission rejection before a successful provider transition, not evidence that edits survive the successful Ghidra-to-Kuna switch reported here. It is a useful preservation control for any transition/recovery regression. More context: #8195.

bsakaguchi commented

2026-11-01 21:11:57 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Different-target switch with the same provider

I exercised the outstanding target-switch case through real stdio MCP on REA b43c329df779708e43ae297f498a844e45d9c923 and Ghidra 12.1.4. After renaming a function and adding an entry comment in owned target A's ephemeral database, I opened a different owned target B with Ghidra, verified a function on B, and reopened A. All opens succeeded; the edited selector was then absent and A's original selector succeeded. The open response contained the selected target's identity without a transition-specific annotation-loss notice. Both targets' hashes were unchanged, neither executed, and close succeeded.

While B was active, Evidence summary still found A's annotation by operation and identified A's digest; the complete retained bundle returned its historical name and comment. Thus switching targets preserves that recorded observation in this run, not the live Ghidra edit. I opened #8307 for the separate question of retrieving one selected historical annotation without transferring the whole bundle. This is one successful Ghidra-to-Ghidra target transition, not a cancellation, cleanup-failure or autonomous-agent test.

@morluto commented on GitHub (Oct 31, 2026): ### Different-target switch with the same provider I exercised the outstanding target-switch case through real stdio MCP on REA `b43c329df779708e43ae297f498a844e45d9c923` and Ghidra 12.1.4. After renaming a function and adding an entry comment in owned target A's ephemeral database, I opened a different owned target B with Ghidra, verified a function on B, and reopened A. All opens succeeded; the edited selector was then absent and A's original selector succeeded. The open response contained the selected target's identity without a transition-specific annotation-loss notice. Both targets' hashes were unchanged, neither executed, and close succeeded. While B was active, Evidence summary still found A's annotation by operation and identified A's digest; the complete retained bundle returned its historical name and comment. Thus switching targets preserves that recorded observation in this run, not the live Ghidra edit. I opened #8307 for the separate question of retrieving one selected historical annotation without transferring the whole bundle. This is one successful Ghidra-to-Ghidra target transition, not a cancellation, cleanup-failure or autonomous-agent test.

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 17:34:58 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6527

No description provided.

Rows
Columns