morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8380] [Bug] Ghidra snapshot verifier expects retained Evidence to disappear #6578

New issue

Open

opened 2026-11-01 21:12:23 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-11-01 21:12:23 +00:00

Owner

Originally created by @morluto on GitHub (Oct 31, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8380

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The host-native real-Ghidra verifier asserts that a newly saved analysis snapshot contains no earlier annotate_native_function Evidence after closing and reopening the engine. After #8361, engine cleanup retains investigation records, so a real-provider lane on PR #8379 stopped at real-ghidra-snapshot-lifecycle.mjs:127. That lane had the separate #8172 retained-reference workaround applied; it did not reach later checks.

Reproduction

On REA main 6e9ee97a69cb16c7150d8b550e705cba055ee4f0, I opened a self-built arm64 Mach-O with real Ghidra 12.1.4 through stdio MCP, analyzed _evaluate and closed to an initial snapshot.
I reopened the target, renamed the function and added a comment, then closed the engine without resetting the investigation. A summary query still found the exact annotation Evidence ID.
I reopened the initial snapshot, checked the function's procedure and comments against the original, and confirmed lookup of the edited name failed. I closed to a fresh snapshot. It contained the historical annotation Evidence ID; the initial snapshot did not. The target hash stayed unchanged and the target was not executed.

Separately, the full lane based on this main revision plus #8379 failed at the assertion above after #8172's temporary workaround. #8379 does not change snapshot or investigation code; the verifier file is identical on main. I have not run the complete host lane on unmodified main, so its full-lane failure is source-supported rather than separately reproduced here.

Expected behavior

Keep the test's check that the live Ghidra program restores the original procedure/comments and does not reapply the edit. Verify historical annotation Evidence remains retained with its identity and historical semantics after engine close and snapshot save; do not interpret its presence as a live database mutation.

Impact and scope

The reported failure blocks a real-provider verifier and can hide checks later in that lane. One macOS arm64 host and one constructed fixture were exercised; no claim is made about other providers or hosts. The production behavior above is consistent with #8361, not evidence that annotation changes were replayed.

REA version

Runtime control: 6e9ee97a69cb16c7150d8b550e705cba055ee4f0; full lane: PR #8379 commit 34670812ca1bee8f06dc3fa7b81e192af6661025 plus the isolated #8172 verifier workaround.

Runtime and platform

Node 24.22.1; macOS 26.3 arm64; Ghidra 12.1.4; JDK 21.0.12.1.

Target and artifact context

Self-built arm64 Mach-O (ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d), caller-owned temporary snapshots, no executable run or modification.

Interface

MCP and the real-Ghidra verification lane

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

The unmodified-main public MCP control returned: initialSnapshotHasAnnotation: false, retainedAfterClose: true, freshSnapshotHasHistoricalAnnotation: true, restoredProcedureMatchesOriginal: true, restoredCommentsMatchOriginal: true, and editedNameLookupError: invalid_request. Full-lane failure: AssertionError in verifyGhidraSnapshotLifecycle at line 127, the negated annotation-record assertion.

Analysis or suspected cause

Source-derived: #8361 explicitly changed close_binary to retain investigation Evidence; the existing verifier still expects the old record-clearing behavior. The record's presence is compatible with unchanged program state, so dropping this assertion without replacing it with the two separate invariants would lose useful regression coverage.

Additional context

Related #8274 (real-provider testing) and #8165 (durable Ghidra project design). cc @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Combined real-provider verification on 2026-11-02

On macOS arm64 with Ghidra 12.1.4 and JDK 21, npm run verify:ghidra passed in a temporary checkout of #8379 0212145b0 with verifier-only changes from #8541 2bf92eb9, #8381 64e42396, #8536 78b6c662 and #8578 dd281594 applied. The report recorded 439 successful and 52 rejected boundary calls, regex recovery, legacy snapshot rejection, mutation snapshot lifecycle, distinct project identity refusals, complete cleanup, verified process lineage and no descendants. This is a combined run, not a clean pass of this issue’s PR or main.

Originally created by @morluto on GitHub (Oct 31, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8380 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior The host-native real-Ghidra verifier asserts that a newly saved analysis snapshot contains no earlier `annotate_native_function` Evidence after closing and reopening the engine. After #8361, engine cleanup retains investigation records, so a real-provider lane on PR #8379 stopped at [`real-ghidra-snapshot-lifecycle.mjs:127`](https://github.com/morluto/rea/blob/6e9ee97a69cb16c7150d8b550e705cba055ee4f0/scripts/lib/real-ghidra-snapshot-lifecycle.mjs#L122-L134). That lane had the separate #8172 retained-reference workaround applied; it did not reach later checks. ### Reproduction 1. On REA main `6e9ee97a69cb16c7150d8b550e705cba055ee4f0`, I opened a self-built arm64 Mach-O with real Ghidra 12.1.4 through stdio MCP, analyzed `_evaluate` and closed to an initial snapshot. 2. I reopened the target, renamed the function and added a comment, then closed the engine without resetting the investigation. A summary query still found the exact annotation Evidence ID. 3. I reopened the initial snapshot, checked the function's procedure and comments against the original, and confirmed lookup of the edited name failed. I closed to a fresh snapshot. It contained the historical annotation Evidence ID; the initial snapshot did not. The target hash stayed unchanged and the target was not executed. Separately, the full lane based on this main revision plus #8379 failed at the assertion above after #8172's temporary workaround. #8379 does not change snapshot or investigation code; the verifier file is identical on main. I have **not** run the complete host lane on unmodified main, so its full-lane failure is source-supported rather than separately reproduced here. ### Expected behavior Keep the test's check that the live Ghidra program restores the original procedure/comments and does not reapply the edit. Verify historical annotation Evidence remains retained with its identity and historical semantics after engine close and snapshot save; do not interpret its presence as a live database mutation. ### Impact and scope The reported failure blocks a real-provider verifier and can hide checks later in that lane. One macOS arm64 host and one constructed fixture were exercised; no claim is made about other providers or hosts. The production behavior above is consistent with #8361, not evidence that annotation changes were replayed. ### REA version Runtime control: `6e9ee97a69cb16c7150d8b550e705cba055ee4f0`; full lane: PR #8379 commit `34670812ca1bee8f06dc3fa7b81e192af6661025` plus the isolated #8172 verifier workaround. ### Runtime and platform Node 24.22.1; macOS 26.3 arm64; Ghidra 12.1.4; JDK 21.0.12.1. ### Target and artifact context Self-built arm64 Mach-O (`ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d`), caller-owned temporary snapshots, no executable run or modification. ### Interface MCP and the real-Ghidra verification lane ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence The unmodified-main public MCP control returned: `initialSnapshotHasAnnotation: false`, `retainedAfterClose: true`, `freshSnapshotHasHistoricalAnnotation: true`, `restoredProcedureMatchesOriginal: true`, `restoredCommentsMatchOriginal: true`, and `editedNameLookupError: invalid_request`. Full-lane failure: `AssertionError` in `verifyGhidraSnapshotLifecycle` at line 127, the negated annotation-record assertion. ### Analysis or suspected cause Source-derived: #8361 explicitly changed `close_binary` to retain investigation Evidence; the existing verifier still expects the old record-clearing behavior. The record's presence is compatible with unchanged program state, so dropping this assertion without replacing it with the two separate invariants would lose useful regression coverage. ### Additional context Related #8274 (real-provider testing) and #8165 (durable Ghidra project design). cc @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 ### Combined real-provider verification on 2026-11-02 On macOS arm64 with Ghidra 12.1.4 and JDK 21, `npm run verify:ghidra` passed in a temporary checkout of #8379 `0212145b0` with verifier-only changes from #8541 `2bf92eb9`, #8381 `64e42396`, #8536 `78b6c662` and #8578 `dd281594` applied. The report recorded 439 successful and 52 rejected boundary calls, regex recovery, legacy snapshot rejection, mutation snapshot lifecycle, distinct project identity refusals, complete cleanup, verified process lineage and no descendants. This is a combined run, not a clean pass of this issue’s PR or main.

bsakaguchi added the

bug

triage

labels

2026-11-01 21:12:23 +00:00

bsakaguchi commented

2026-11-01 21:12:24 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Follow-up: I invoked the existing verifyGhidraSnapshotLifecycle itself through real Ghidra on unmodified main 6e9ee97a. It fails at the same line 127 assertion, independently of #8379 and without the full lane or the #8172 workaround.

In an isolated checkout, I replaced the negative record-presence assertion with exact-ID and full-record equality for the earlier annotation and edited inspect_native_api observation. The existing original procedure/comment checks remained intact. The same real-provider verifier then passed 22 successful calls and four expected snapshot-binding rejections; the target hash was unchanged. This is a candidate verifier correction on one macOS fixture, not a claim that all snapshot or provider cases are covered.

@morluto commented on GitHub (Oct 31, 2026): Follow-up: I invoked the existing `verifyGhidraSnapshotLifecycle` itself through real Ghidra on unmodified main `6e9ee97a`. It fails at the same line 127 assertion, independently of #8379 and without the full lane or the #8172 workaround. In an isolated checkout, I replaced the negative record-presence assertion with exact-ID and full-record equality for the earlier annotation and edited `inspect_native_api` observation. The existing original procedure/comment checks remained intact. The same real-provider verifier then passed 22 successful calls and four expected snapshot-binding rejections; the target hash was unchanged. This is a candidate verifier correction on one macOS fixture, not a claim that all snapshot or provider cases are covered.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6578

No description provided.

Rows
Columns