[GH-ISSUE #3751] Ghidra provider: open an existing analysed project instead of re-importing every session (large native binaries)

bsakaguchi commented

2026-10-16 21:05:36 +00:00

Owner

Originally created by @whalemenace on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3751

Summary

Allow the Ghidra provider to open an existing, already-analysed Ghidra program (a project + program name, or a .gzf) instead of always importing the raw binary into an ephemeral project. Today every session pays the full auto-analysis again, which on large native binaries is the whole cost.

Measurements (rea-agents 6.5.0, Ghidra 12.1.4, JDK 21, Windows 11 x64, 63 GB RAM)

Target: a shipped Unreal Engine 5.8 game executable, x86-64 PE, 191 MiB, ~379k functions.

Path	Wall clock
`rea analyze` with the default ephemeral import (`GHIDRA_HEADLESS_MAXMEM=12G`)	hit a 60 min `REA_GHIDRA_STARTUP_TIMEOUT_MS`, analysis unfinished
Ghidra `analyzeHeadless -import` directly into a persistent project (32G heap)	74 min once; analysis complete
Re-opening that project with `-process -noanalysis` and running 3 post-scripts incl. 5 decompiles	15 s
`rea function` with `REA_GHIDRA_SEED_FILE` (all 377,642 functions exported from the project) and 9 heavy analyzers in `REA_GHIDRA_DISABLED_ANALYZERS`	15 min 40 s; a second CLI call on the same target with `--snapshot`: 15 min 08 s again

Where the 74 minutes go (Ghidra's own table): Create Address Tables 21 min, Stack 14 min, x86 Constant Reference 12 min, Disassemble Entry Points 7 min, Decompiler Switch 7 min. Seeds plus disabled analyzers remove about 60 of the 74 minutes, but the remaining ~15 minutes are paid on every CLI invocation and on every MCP session start, because import_mode is ephemeral-source-immutable.

Proposal

A provider option (env or open_binary parameter) that points at an existing Ghidra project directory + program name, opened read-only with -process <program> -noanalysis (or an imported .gzf), with the same binary-identity check REA already does (sha256 of the program's original executable vs the selected target).
Keep the ephemeral import as the default. Evidence records could carry import_mode: persistent-project-read-only and mark the program's existing names as caller assertions, exactly as the seed outcome text already does.
Precedent in REA: the Hopper provider already accepts an "analysis database path" (.hop).

This would make the Ghidra provider usable for daily work on large game binaries (names applied once via PDB, BinDiff or scripts, then queried for seconds) without an IDA licence, and it keeps the evidence ledger intact for the whole investigation.

Happy to draft the bridge change if the direction is acceptable.

Originally created by @whalemenace on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3751 ## Summary Allow the Ghidra provider to open an existing, already-analysed Ghidra program (a project + program name, or a `.gzf`) instead of always importing the raw binary into an ephemeral project. Today every session pays the full auto-analysis again, which on large native binaries is the whole cost. ## Measurements (rea-agents 6.5.0, Ghidra 12.1.4, JDK 21, Windows 11 x64, 63 GB RAM) Target: a shipped Unreal Engine 5.8 game executable, x86-64 PE, 191 MiB, ~379k functions. | Path | Wall clock | |---|---| | `rea analyze` with the default ephemeral import (`GHIDRA_HEADLESS_MAXMEM=12G`) | hit a 60 min `REA_GHIDRA_STARTUP_TIMEOUT_MS`, analysis unfinished | | Ghidra `analyzeHeadless -import` directly into a persistent project (32G heap) | 74 min once; analysis complete | | Re-opening that project with `-process -noanalysis` and running 3 post-scripts incl. 5 decompiles | 15 s | | `rea function` with `REA_GHIDRA_SEED_FILE` (all 377,642 functions exported from the project) and 9 heavy analyzers in `REA_GHIDRA_DISABLED_ANALYZERS` | 15 min 40 s; a second CLI call on the same target with `--snapshot`: 15 min 08 s again | Where the 74 minutes go (Ghidra's own table): Create Address Tables 21 min, Stack 14 min, x86 Constant Reference 12 min, Disassemble Entry Points 7 min, Decompiler Switch 7 min. Seeds plus disabled analyzers remove about 60 of the 74 minutes, but the remaining ~15 minutes are paid on every CLI invocation and on every MCP session start, because `import_mode` is `ephemeral-source-immutable`. ## Proposal - A provider option (env or `open_binary` parameter) that points at an existing Ghidra project directory + program name, opened read-only with `-process <program> -noanalysis` (or an imported `.gzf`), with the same binary-identity check REA already does (sha256 of the program's original executable vs the selected target). - Keep the ephemeral import as the default. Evidence records could carry `import_mode: persistent-project-read-only` and mark the program's existing names as caller assertions, exactly as the seed outcome text already does. - Precedent in REA: the Hopper provider already accepts an "analysis database path" (`.hop`). This would make the Ghidra provider usable for daily work on large game binaries (names applied once via PDB, BinDiff or scripts, then queried for seconds) without an IDA licence, and it keeps the evidence ledger intact for the whole investigation. Happy to draft the bridge change if the direction is acceptable.

bsakaguchi commented

2026-10-16 21:05:38 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

The numbers make the case convincingly: 74 min for one-time analysis vs 15 s to reopen with -noanalysis, and the remaining ~15 min per invocation is pure re-import overhead with ephemeral mode. Opening a caller-selected persistent project read-only, with the existing sha256 identity check carried over, is the right design — it keeps evidence semantics intact (import_mode recorded as persistent-project-read-only) while making daily work on large binaries feasible. Also good that the Hopper provider already sets precedent with .hop analysis paths, so this slots into an existing pattern rather than inventing a new one.

@XinwangTao commented on GitHub (Oct 15, 2026): The numbers make the case convincingly: 74 min for one-time analysis vs 15 s to reopen with -noanalysis, and the remaining ~15 min per invocation is pure re-import overhead with ephemeral mode. Opening a caller-selected persistent project read-only, with the existing sha256 identity check carried over, is the right design — it keeps evidence semantics intact (import_mode recorded as persistent-project-read-only) while making daily work on large binaries feasible. Also good that the Hopper provider already sets precedent with .hop analysis paths, so this slots into an existing pattern rather than inventing a new one.

bsakaguchi referenced this issue

2026-10-17 21:11:33 +00:00

[PR #1456] [MERGED] fix(process)!: preserve literal replacement and comparison semantics #2898

bsakaguchi commented

2026-10-31 00:30:49 +00:00

Author

Owner

@RoKenobi commented on GitHub (Oct 18, 2026):

A data point for this issue, from REA 6.5.0 on Linux x64 (20 cores, 31 GB RAM) with Ghidra 12.1.4:

Target: iTunes.exe 12.10.11.2, a 38 MB x64 PE.
The first query timed out at the default 330 s startup budget, and REA deleted the partial project.
The timeout error said "Try a smaller request, then run rea doctor". The fix was raising REA_GHIDRA_STARTUP_TIMEOUT_MS, which the README mentions, but the error message, the doctor output and the skill don't. Naming the setting in the error would help. This is the same in 6.7.0. PR #4864 documents the same startup-budget case.
With a 2-hour budget, auto-analysis finished, but it took several times longer than 330 s.
Later the MCP server restarted, the finished analysis was gone, and the whole import had to run again.

Reusing a finished analysis across server restarts (as this issue asks) would have saved most of that time.

@RoKenobi commented on GitHub (Oct 18, 2026): A data point for this issue, from REA 6.5.0 on Linux x64 (20 cores, 31 GB RAM) with Ghidra 12.1.4: - Target: `iTunes.exe` 12.10.11.2, a 38 MB x64 PE. - The first query timed out at the default 330 s startup budget, and REA deleted the partial project. - The timeout error said "Try a smaller request, then run `rea doctor`". The fix was raising `REA_GHIDRA_STARTUP_TIMEOUT_MS`, which the README mentions, but the error message, the doctor output and the skill don't. Naming the setting in the error would help. This is the same in 6.7.0. PR #4864 documents the same startup-budget case. - With a 2-hour budget, auto-analysis finished, but it took several times longer than 330 s. - Later the MCP server restarted, the finished analysis was gone, and the whole import had to run again. Reusing a finished analysis across server restarts (as this issue asks) would have saved most of that time.

bsakaguchi commented

2026-10-31 00:30:49 +00:00

Author

Owner

@morluto commented on GitHub (Oct 26, 2026):

Thanks for documenting the large-project workflow and its target identity needs. #7236 landed reuse of a selected .gpr project through a verified private copy, with read-only Ghidra processing and binary identity checks. I am leaving this report open because your reproduced host was Windows, which that implementation explicitly does not support, and .gzf import is also outside its described scope. The merged PR reports real Linux verification; this triage did not rerun the large target.

@morluto commented on GitHub (Oct 26, 2026): Thanks for documenting the large-project workflow and its target identity needs. #7236 landed reuse of a selected `.gpr` project through a verified private copy, with read-only Ghidra processing and binary identity checks. I am leaving this report open because your reproduced host was Windows, which that implementation explicitly does not support, and `.gzf` import is also outside its described scope. The merged PR reports real Linux verification; this triage did not rerun the large target.

bsakaguchi commented

2026-10-31 00:30:49 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 27, 2026):

#7236 落地后把 Windows 缺口补上是有明确收益的：RoKenobi 的数据点说明默认 330s 启动预算对大二进制根本不够，而错误消息引导用户去跑 rea doctor 是误导。如果 persistent-project 复用能覆盖 Windows（.gpr 只读打开 + 身份校验），这类用例会直接从"调大 timeout 反复等待重导入"变成"秒级重开"，超时错误也就自然消失。建议 Windows 实现时留意三点：Ghidra 项目目录在 Windows 上的文件锁语义（并发打开同一 .gpr 时的 .lock 行为）、路径长度上限（UE 大项目树很容易超 MAX_PATH）、以及把超时错误消息改成同时提示 REA_GHIDRA_STARTUP_TIMEOUT_MS 与 persistent-project 复用两个选项。

@XinwangTao commented on GitHub (Oct 27, 2026): #7236 落地后把 Windows 缺口补上是有明确收益的：RoKenobi 的数据点说明默认 330s 启动预算对大二进制根本不够，而错误消息引导用户去跑 rea doctor 是误导。如果 persistent-project 复用能覆盖 Windows（.gpr 只读打开 + 身份校验），这类用例会直接从"调大 timeout 反复等待重导入"变成"秒级重开"，超时错误也就自然消失。建议 Windows 实现时留意三点：Ghidra 项目目录在 Windows 上的文件锁语义（并发打开同一 .gpr 时的 .lock 行为）、路径长度上限（UE 大项目树很容易超 MAX_PATH）、以及把超时错误消息改成同时提示 REA_GHIDRA_STARTUP_TIMEOUT_MS 与 persistent-project 复用两个选项。

bsakaguchi commented

2026-10-31 00:30:50 +00:00

Author

Owner

@macjayz commented on GitHub (Oct 27, 2026):

I would like to contribute the remaining Windows support for persistent Ghidra-project reuse, building on #7236. Before I start, can a maintainer confirm that this follow-up is available and indicate whether Windows .gpr reuse alone is the preferred first scope, with .gzf left separate? I can keep the change focused on the current verified-copy and identity-check design, and add Windows-specific regression coverage where the repository can support it.

@macjayz commented on GitHub (Oct 27, 2026): I would like to contribute the remaining Windows support for persistent Ghidra-project reuse, building on #7236. Before I start, can a maintainer confirm that this follow-up is available and indicate whether Windows `.gpr` reuse alone is the preferred first scope, with `.gzf` left separate? I can keep the change focused on the current verified-copy and identity-check design, and add Windows-specific regression coverage where the repository can support it.

bsakaguchi commented

2026-10-31 00:30:50 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 28, 2026):

+1 for the Windows follow-up scoped as .gpr reuse first, .gzf left separate — that matches how #7236 landed and keeps the diff reviewable. Echoing my earlier notes as acceptance criteria for the Windows port: (1) verify the verified-copy step works under Windows file-lock semantics (concurrent open of the same .gpr, .lock behavior), and (2) watch MAX_PATH in the Ghidra project tree copy — deep UE trees can trip the 260-char limit before any Ghidra code runs. Also worth a tiny independent change: make the startup-timeout error name REA_GHIDRA_STARTUP_TIMEOUT_MS and mention persistent-project reuse as the alternative, since RoKenobi showed the current "run rea doctor" hint misleads.

@XinwangTao commented on GitHub (Oct 28, 2026): +1 for the Windows follow-up scoped as .gpr reuse first, .gzf left separate — that matches how #7236 landed and keeps the diff reviewable. Echoing my earlier notes as acceptance criteria for the Windows port: (1) verify the verified-copy step works under Windows file-lock semantics (concurrent open of the same .gpr, .lock behavior), and (2) watch MAX_PATH in the Ghidra project tree copy — deep UE trees can trip the 260-char limit before any Ghidra code runs. Also worth a tiny independent change: make the startup-timeout error name REA_GHIDRA_STARTUP_TIMEOUT_MS and mention persistent-project reuse as the alternative, since RoKenobi showed the current "run rea doctor" hint misleads.

bsakaguchi commented

2026-10-31 00:30:50 +00:00

Author

Owner

@macjayz commented on GitHub (Oct 28, 2026):

Thanks @XinwangTao, I'll treat those as acceptance criteria for the Windows .gpr reuse:

the verified-copy step under Windows file locking, including a .gpr that is already open elsewhere and a leftover .lock file
path length in the copied project tree, failing with a clear limitation (not a raw Java or filesystem error) when it exceeds what the host supports

On the startup-timeout message: I agree it's a separate change, and it's useful on every platform, not just Windows. I can open it as its own small PR. It would name REA_GHIDRA_STARTUP_TIMEOUT_MS and mention persistent-project reuse instead of pointing to rea doctor.

@morluto, could you confirm whether (a) the Windows .gpr follow-up is open for me to take, and (b) you'd accept the timeout-message fix as a separate PR?

@macjayz commented on GitHub (Oct 28, 2026): Thanks @XinwangTao, I'll treat those as acceptance criteria for the Windows `.gpr` reuse: 1. the verified-copy step under Windows file locking, including a `.gpr` that is already open elsewhere and a leftover `.lock` file 2. path length in the copied project tree, failing with a clear limitation (not a raw Java or filesystem error) when it exceeds what the host supports On the startup-timeout message: I agree it's a separate change, and it's useful on every platform, not just Windows. I can open it as its own small PR. It would name `REA_GHIDRA_STARTUP_TIMEOUT_MS` and mention persistent-project reuse instead of pointing to `rea doctor`. @morluto, could you confirm whether (a) the Windows `.gpr` follow-up is open for me to take, and (b) you'd accept the timeout-message fix as a separate PR?

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

Current-state note from a real macOS arm64 / Ghidra 12.1.4 check: REA now supports the expanded-project portion of this request on POSIX via REA_GHIDRA_PROJECT_PATH and optional REA_GHIDRA_PROGRAM_NAME. It copies the caller's project into an owned runtime, checks the selected program's recorded executable SHA-256 against the chosen binary, opens the copy read-only with -noanalysis, and reports that prior completeness/settings are unknown. These settings are server-start environment configuration, not a per-open_binary project selector.

In a new owned test project I used direct headless Ghidra to export and reimport a .gzf containing a renamed function, comment, structure and typed signature. Selecting the restored expanded project in REA returned the refined pseudocode and type; REA refused further annotation, and all 10 original project regular-file content digests remained unchanged. The packed export/import were done outside REA; this does not demonstrate direct .gzf selection, editable project continuation, Windows support, or large-binary speed. The full observation and open design questions are in #8165. A deliberately unanalysed selected project also motivated the recovery question in #8208.

@morluto commented on GitHub (Oct 30, 2026): Current-state note from a real macOS arm64 / Ghidra 12.1.4 check: REA now supports the expanded-project portion of this request on POSIX via `REA_GHIDRA_PROJECT_PATH` and optional `REA_GHIDRA_PROGRAM_NAME`. It copies the caller's project into an owned runtime, checks the selected program's recorded executable SHA-256 against the chosen binary, opens the copy read-only with `-noanalysis`, and reports that prior completeness/settings are unknown. These settings are server-start environment configuration, not a per-`open_binary` project selector. In a new owned test project I used direct headless Ghidra to export and reimport a `.gzf` containing a renamed function, comment, structure and typed signature. Selecting the restored expanded project in REA returned the refined pseudocode and type; REA refused further annotation, and all 10 original project regular-file content digests remained unchanged. The packed export/import were done outside REA; this does not demonstrate direct `.gzf` selection, editable project continuation, Windows support, or large-binary speed. The full observation and open design questions are in #8165. A deliberately unanalysed selected project also motivated the recovery question in #8208.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

Additional real macOS/Ghidra 12.1.4 control for existing-project selection: a single self-owned project contained two root programs imported from byte-identical Mach-O files, but each had a different user-defined function name and comment. Separate REA servers explicitly selecting program_alpha and program_beta returned the corresponding distinct pseudocode. Their profiles retained the same project content digest but different exact program names and profile digests. A snapshot created while selecting alpha was refused with profile_mismatch when beta was selected; opening beta without that snapshot then worked. All 12 source-project regular-file digests and the original target digest remained unchanged.

This supports preserving exact program choice in future per-open selection designs: a matching binary hash alone cannot choose between separately annotated programs. It is one POSIX/headless fixture, not Windows, nested-folder or .gzf support. Omitting the program name when the target basename was not a project program produced the separate recovery issue #8216.

@morluto commented on GitHub (Oct 30, 2026): Additional real macOS/Ghidra 12.1.4 control for existing-project selection: a single self-owned project contained two root programs imported from byte-identical Mach-O files, but each had a different user-defined function name and comment. Separate REA servers explicitly selecting `program_alpha` and `program_beta` returned the corresponding distinct pseudocode. Their profiles retained the same project content digest but different exact program names and profile digests. A snapshot created while selecting alpha was refused with `profile_mismatch` when beta was selected; opening beta without that snapshot then worked. All 12 source-project regular-file digests and the original target digest remained unchanged. This supports preserving exact program choice in future per-open selection designs: a matching binary hash alone cannot choose between separately annotated programs. It is one POSIX/headless fixture, not Windows, nested-folder or `.gzf` support. Omitting the program name when the target basename was not a project program produced the separate recovery issue #8216.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

Another macOS 12.1.4 headless control exercised source-project freshness. REA committed a selected project/profile at open_binary. A separate Ghidra headless script then intentionally renamed a function in that disposable source project before the first query; REA refused the stale copy. Reopening the same selected target committed a new project fingerprint and returned the updated annotation. A further source-project edit while REA's private Ghidra copy was already running did not change the active query result; explicitly reopening loaded the newer project state. The executable digest was unchanged and close succeeded.

The safety boundary worked in this fixture, but the stale-project refusal reached the agent as generic provider startup failure and doctor advice; the typed project-change cause survived only inside diagnostics. #8217 records that separate recovery issue. This is not evidence about concurrent writes during the copy, GUI use, or Windows.

@morluto commented on GitHub (Oct 30, 2026): Another macOS 12.1.4 headless control exercised source-project freshness. REA committed a selected project/profile at `open_binary`. A separate Ghidra headless script then intentionally renamed a function in that disposable source project before the first query; REA refused the stale copy. Reopening the same selected target committed a new project fingerprint and returned the updated annotation. A further source-project edit while REA's private Ghidra copy was already running did not change the active query result; explicitly reopening loaded the newer project state. The executable digest was unchanged and close succeeded. The safety boundary worked in this fixture, but the stale-project refusal reached the agent as generic provider startup failure and doctor advice; the typed project-change cause survived only inside diagnostics. #8217 records that separate recovery issue. This is not evidence about concurrent writes during the copy, GUI use, or Windows.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

A project-identity control: I intentionally changed an instruction in a disposable Ghidra project, leaving its imported Mach-O unchanged. REA selected the project read-only and returned the saved changed bytes/pseudocode; a fresh-import session on the same original returned the original bytes/pseudocode. Both mapped the same address to the same original file offset. The selected-project profile and limitations disclosed its identity and possible earlier edits, consistent with the installation guide's warning that matching the recorded import SHA-256 does not establish an unedited program. REA did not write the project or target.

The existing load-image source-files view showed different original/modified digests for this edited project, but an independent fresh-import fixture with an applied relocation also showed different digests. #8232 discusses whether a bounded source-versus-image comparison would help agents without classifying every difference as a patch. This is one controlled project on macOS, not a broad analysis of project contents or Windows.

@morluto commented on GitHub (Oct 30, 2026): A project-identity control: I intentionally changed an instruction in a disposable Ghidra project, leaving its imported Mach-O unchanged. REA selected the project read-only and returned the saved changed bytes/pseudocode; a fresh-import session on the same original returned the original bytes/pseudocode. Both mapped the same address to the same original file offset. The selected-project profile and limitations disclosed its identity and possible earlier edits, consistent with the installation guide's warning that matching the recorded import SHA-256 does not establish an unedited program. REA did not write the project or target. The existing load-image source-files view showed different original/modified digests for this edited project, but an independent fresh-import fixture with an applied relocation also showed different digests. #8232 discusses whether a bounded source-versus-image comparison would help agents without classifying every difference as a patch. This is one controlled project on macOS, not a broad analysis of project contents or Windows.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 31, 2026):

Thanks for the detailed POSIX verification — reopening the persistent project read-only with the sha256 identity check and refusing further annotation keeps the evidence ledger intact, which is exactly what this request needed. Echoing macjayz: scoping the Windows port to .gpr reuse first (with the file-lock and MAX_PATH checks discussed above) and leaving .gzf separate keeps the follow-up diff reviewable. Glad the startup-timeout message fix landed as its own PR (#8114) since it benefits every platform.

@XinwangTao commented on GitHub (Oct 31, 2026): Thanks for the detailed POSIX verification — reopening the persistent project read-only with the sha256 identity check and refusing further annotation keeps the evidence ledger intact, which is exactly what this request needed. Echoing macjayz: scoping the Windows port to .gpr reuse first (with the file-lock and MAX_PATH checks discussed above) and leaving .gzf separate keeps the follow-up diff reviewable. Glad the startup-timeout message fix landed as its own PR (#8114) since it benefits every platform.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Snapshot control after changing the same selected project

On REA b43c329df779708e43ae297f498a844e45d9c923 and Ghidra 12.1.4, I copied a source-owned project into a disposable directory, opened its selected program through real stdio MCP, analyzed one function, and closed with snapshot_path. The resulting snapshot reported one primitive entry and one Evidence record. Reopening the unchanged project with that snapshot and querying the function succeeded and explicitly reported snapshot replay.

After those sessions closed, a separate headless Ghidra script deliberately renamed the function in the disposable source project at the same project path and program name. Opening with the old snapshot then returned evidence_integrity_mismatch / profile_mismatch and advised analyzing without the snapshot and saving a fresh one. Opening without it succeeded and returned the newly named function. Thus the observed refusal is not just a different selected program or project pathname.

REA did not alter the target, selected project, or saved snapshot during the replay/refusal/recovery calls; their before/after content hashes matched within each phase. The intentional project edit was outside REA. The target was not executed. The error names target, provider or profile as possible causes rather than identifying the changed project field; this control establishes safe refusal and usable recovery, not a field-level diagnostic or broad race coverage.

@morluto commented on GitHub (Oct 31, 2026): ### Snapshot control after changing the same selected project On REA `b43c329df779708e43ae297f498a844e45d9c923` and Ghidra 12.1.4, I copied a source-owned project into a disposable directory, opened its selected program through real stdio MCP, analyzed one function, and closed with `snapshot_path`. The resulting snapshot reported one primitive entry and one Evidence record. Reopening the **unchanged** project with that snapshot and querying the function succeeded and explicitly reported snapshot replay. After those sessions closed, a separate headless Ghidra script deliberately renamed the function in the disposable source project at the **same project path and program name**. Opening with the old snapshot then returned `evidence_integrity_mismatch` / `profile_mismatch` and advised analyzing without the snapshot and saving a fresh one. Opening without it succeeded and returned the newly named function. Thus the observed refusal is not just a different selected program or project pathname. REA did not alter the target, selected project, or saved snapshot during the replay/refusal/recovery calls; their before/after content hashes matched within each phase. The intentional project edit was outside REA. The target was not executed. The error names target, provider or profile as possible causes rather than identifying the changed project field; this control establishes safe refusal and usable recovery, not a field-level diagnostic or broad race coverage.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real CLI refusal and recovery control

Using the same changed disposable project, rea function <target> 0x1000002d8 --provider ghidra --snapshot <old-snapshot> --format json --full-output exited 1 with evidence_integrity_mismatch / profile_mismatch and the same analyze-without-this-snapshot recovery as MCP. The old snapshot's SHA-256 remained unchanged.

A subsequent rea function call with a new snapshot path succeeded and returned audit_project_changed; repeating with that new path returned identical Evidence. In a separate process-tree sampling control at roughly 100 ms intervals, the uncached CLI call showed Ghidra Java/decompiler descendants, while the saved-snapshot call showed none. The two results and Evidence IDs matched. Sampling cannot rule out a very short-lived process; source snapshotEvidenceForQuery and its CLI fast path explain how a matching record can be returned without starting the provider. The CLI returned the original Evidence without the explicit snapshot-replay limitation observed in the earlier MCP control, so the captured output alone does not establish which path was used.

Target, selected-project file hashes, and the old snapshot hash remained unchanged through the CLI checks; the new snapshot was caller-selected test output. The target was not executed. This is one macOS/Ghidra selected-project case, not a cross-platform or performance measurement.

@morluto commented on GitHub (Oct 31, 2026): ### Real CLI refusal and recovery control Using the same changed disposable project, `rea function <target> 0x1000002d8 --provider ghidra --snapshot <old-snapshot> --format json --full-output` exited 1 with `evidence_integrity_mismatch` / `profile_mismatch` and the same analyze-without-this-snapshot recovery as MCP. The old snapshot's SHA-256 remained unchanged. A subsequent `rea function` call with a new snapshot path succeeded and returned `audit_project_changed`; repeating with that new path returned identical Evidence. In a separate process-tree sampling control at roughly 100 ms intervals, the uncached CLI call showed Ghidra Java/decompiler descendants, while the saved-snapshot call showed none. The two results and Evidence IDs matched. Sampling cannot rule out a very short-lived process; source [`snapshotEvidenceForQuery`](https://github.com/morluto/rea/blob/b43c329df779708e43ae297f498a844e45d9c923/src/domain/analysisSnapshot.ts) and its CLI fast path explain how a matching record can be returned without starting the provider. The CLI returned the original Evidence without the explicit snapshot-replay limitation observed in the earlier MCP control, so the captured output alone does not establish which path was used. Target, selected-project file hashes, and the old snapshot hash remained unchanged through the CLI checks; the new snapshot was caller-selected test output. The target was not executed. This is one macOS/Ghidra selected-project case, not a cross-platform or performance measurement.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real MCP snapshot-destination conflict and recovery control on the same source-owned, unexecuted Mach-O with REA b43c329, Ghidra 12.1.4 and Kuna 1.986:

After collecting a complete function dossier, close_binary({snapshot_path:<existing owned file>}) returned invalid_request with reason exists and advice to choose another path or allow overwrite. The existing file's hash was unchanged.
For each provider, binary_session still reported the selected provider open with no cleanup pending, and a subsequent procedure_address query succeeded.
Closing to a new path succeeded. Reopening the same target/provider with that snapshot and inspecting the original dossier by Evidence ID returned its pseudocode. Ghidra's receipt reported one primitive entry and two Evidence records; Kuna's reported zero primitive entries and two Evidence records. The latter proves retained dossier retrieval, not that a future live Kuna query will be served from the snapshot.

Both sessions closed successfully and the target hash remained unchanged. This checks one existing-file conflict and explicit fresh-path recovery per provider; it does not test overwrite, permissions, interrupted writes, edited Ghidra sessions, or provider-independent snapshot replay.

@morluto commented on GitHub (Oct 31, 2026): Real MCP snapshot-destination conflict and recovery control on the same source-owned, unexecuted Mach-O with REA `b43c329`, Ghidra 12.1.4 and Kuna 1.986: - After collecting a complete function dossier, `close_binary({snapshot_path:<existing owned file>})` returned `invalid_request` with reason `exists` and advice to choose another path or allow overwrite. The existing file's hash was unchanged. - For each provider, `binary_session` still reported the selected provider open with no cleanup pending, and a subsequent `procedure_address` query succeeded. - Closing to a new path succeeded. Reopening the same target/provider with that snapshot and inspecting the original dossier by Evidence ID returned its pseudocode. Ghidra's receipt reported one primitive entry and two Evidence records; Kuna's reported zero primitive entries and two Evidence records. The latter proves retained dossier retrieval, not that a future live Kuna query will be served from the snapshot. Both sessions closed successfully and the target hash remained unchanged. This checks one existing-file conflict and explicit fresh-path recovery per provider; it does not test overwrite, permissions, interrupted writes, edited Ghidra sessions, or provider-independent snapshot replay.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real cross-provider snapshot controls on the same unexecuted Mach-O, REA b43c329, Ghidra 12.1.4 and Kuna 1.986:

With Ghidra active, explicitly requesting Kuna while supplying the Ghidra snapshot returned evidence_integrity_mismatch / profile_mismatch. The Ghidra binding stayed open, cleanup was not pending, and a Ghidra session rename remained resolvable.
With Kuna active, requesting Ghidra while supplying the Kuna snapshot returned the same mismatch class. Kuna remained bound and its known procedure remained resolvable.
An attempt to import the matching Ghidra snapshot into the already edited Ghidra session was separately refused with a message that metadata may have changed; the renamed selector still resolved afterward. The error advises exporting observations if needed, closing without a snapshot, and reopening. Evidence export does not restore an edited Ghidra database.
After explicit close, each provider successfully reopened its own matching snapshot and resolved the known function. Both original snapshot hashes and the binary hash were unchanged; all sessions closed.

These verify early refusal and active-session preservation for the tested mismatch and dirty-session cases. The profile-mismatch message intentionally lists target/provider/profile possibilities, so this is not a claim that it identifies the exact differing field. No cross-provider cache reuse or broader compatibility is established. Related switch-retirement discussion: #8289.

@morluto commented on GitHub (Oct 31, 2026): Real cross-provider snapshot controls on the same unexecuted Mach-O, REA `b43c329`, Ghidra 12.1.4 and Kuna 1.986: - With Ghidra active, explicitly requesting Kuna while supplying the Ghidra snapshot returned `evidence_integrity_mismatch` / `profile_mismatch`. The Ghidra binding stayed open, cleanup was not pending, and a Ghidra session rename remained resolvable. - With Kuna active, requesting Ghidra while supplying the Kuna snapshot returned the same mismatch class. Kuna remained bound and its known procedure remained resolvable. - An attempt to import the matching Ghidra snapshot into the already edited Ghidra session was separately refused with a message that metadata may have changed; the renamed selector still resolved afterward. The error advises exporting observations if needed, closing without a snapshot, and reopening. Evidence export does not restore an edited Ghidra database. - After explicit close, each provider successfully reopened its own matching snapshot and resolved the known function. Both original snapshot hashes and the binary hash were unchanged; all sessions closed. These verify early refusal and active-session preservation for the tested mismatch and dirty-session cases. The profile-mismatch message intentionally lists target/provider/profile possibilities, so this is not a claim that it identifies the exact differing field. No cross-provider cache reuse or broader compatibility is established. Related switch-retirement discussion: #8289.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Packed-program admission and explicit recovery

I tested the existing .gzf-to-expanded-project workflow on REA b43c329df779708e43ae297f498a844e45d9c923, Ghidra 12.1.4, and the owned packed program described in #8165.

Setting REA_GHIDRA_PROJECT_PATH to that .gzf made scoped rea doctor --provider ghidra --target <target> --format json report config_drift, and rea decompile return configuration_invalid. The guidance requested an expanded .gpr marker and said to restore .grez archives first; it did not describe importing a .gzf packed program. With the same configuration, the MCP SDK could not initialize because the REA process exited 1; its stderr contained the same configuration error. This is a startup configuration refusal, not a failed open_binary call or a claim that direct .gzf is supported.

For a concrete manual recovery, I ran Ghidra analyzeHeadless <owned-dir> restored -import <packed.gzf> -noanalysis into a newly created disposable project, then configured REA with its restored.gpr and exact program name. A real rea decompile returned the earlier caller-authored function name, comment and typed structure accesses. The original executable, packed file and the expanded project's regular-file hashes stayed unchanged during the REA command; neither binary ran. Ghidra's own import created the new project before that check. This is one single-program, same-version control, not a full project archive, Windows, compatibility, concurrent import or performance test.

The existing .gpr route is therefore usable after an explicit external import, but a caller who has only a .gzf may not discover that from the error. Any future direct support should preserve the distinction between packed programs and .grez project archives, exact program/target identity, caller-selected storage, and existing read-only/cleanup semantics. No implicit import or project mutation is proposed by this observation.

@morluto commented on GitHub (Oct 31, 2026): ### Packed-program admission and explicit recovery I tested the existing `.gzf`-to-expanded-project workflow on REA `b43c329df779708e43ae297f498a844e45d9c923`, Ghidra 12.1.4, and the owned packed program described in #8165. Setting `REA_GHIDRA_PROJECT_PATH` to that `.gzf` made scoped `rea doctor --provider ghidra --target <target> --format json` report `config_drift`, and `rea decompile` return `configuration_invalid`. The guidance requested an expanded `.gpr` marker and said to restore `.grez` archives first; it did not describe importing a `.gzf` packed program. With the same configuration, the MCP SDK could not initialize because the REA process exited 1; its stderr contained the same configuration error. This is a startup configuration refusal, not a failed `open_binary` call or a claim that direct `.gzf` is supported. For a concrete manual recovery, I ran Ghidra `analyzeHeadless <owned-dir> restored -import <packed.gzf> -noanalysis` into a newly created disposable project, then configured REA with its `restored.gpr` and exact program name. A real `rea decompile` returned the earlier caller-authored function name, comment and typed structure accesses. The original executable, packed file and the expanded project's regular-file hashes stayed unchanged during the REA command; neither binary ran. Ghidra's own import created the new project before that check. This is one single-program, same-version control, not a full project archive, Windows, compatibility, concurrent import or performance test. The existing `.gpr` route is therefore usable after an explicit external import, but a caller who has only a `.gzf` may not discover that from the error. Any future direct support should preserve the distinction between packed programs and `.grez` project archives, exact program/target identity, caller-selected storage, and existing read-only/cleanup semantics. No implicit import or project mutation is proposed by this observation.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Follow-up on the two-program control above: current main 60149282 now gives the actionable missing-program error, and list_documents correctly lists only the already selected program. Direct Ghidra wildcard processing of a disposable copy found both root programs with the same imported digest, while each had distinct authored analysis. I opened #8344 to discuss bounded candidate discovery before selection and its copy/cleanup costs; it does not propose auto-selecting by digest or changing the existing read-only authority.

@morluto commented on GitHub (Oct 31, 2026): Follow-up on the two-program control above: current main `60149282` now gives the actionable missing-program error, and `list_documents` correctly lists only the already selected program. Direct Ghidra wildcard processing of a disposable copy found both root programs with the same imported digest, while each had distinct authored analysis. I opened #8344 to discuss bounded candidate discovery before selection and its copy/cleanup costs; it does not propose auto-selecting by digest or changing the existing read-only authority.

bsakaguchi commented

2026-11-01 21:10:55 +00:00

Author

Owner

@morluto commented on GitHub (Nov 1, 2026):

Same-project program handoff through existing Evidence export

A follow-up on the two-program fixture tests the current recovery path without proposing a silent project switch. With real Ghidra 12.1.4/JDK 21 and REA PR #8437 head 22072c787566e8aa4bbc7eac8659c820a2fc3c08 (the PR changes only prompt wording/tests), I used two separate stdio MCP server processes selecting the same owned project with REA_GHIDRA_PROGRAM_NAME=program_alpha and then program_beta:

In alpha, analyze_function returned a summary and retained parent. Adding a hypothetical program_name: "program_beta" to open_binary returned invalid_request; alpha remained usable. The advertised open_binary input schema does not contain that field. This is the currently unsupported request, not a promised API regression.
Exported the alpha Evidence bundle and closed alpha. In the new beta-configured server, opened the target and analyzed selected_beta; its analysis-profile digest differed from alpha's.
Imported the alpha bundle while beta was active. inspect_analysis_view resolved alpha's original parent ID, and the imported full parent retained alpha's profile digest. A subsequent beta procedure lookup still succeeded.

The target and all regular files of the disposable source project had unchanged content hashes; the target was not executed. This is one project, one macOS host and two explicitly configured server processes. It transfers historical Evidence, not an editable project or live annotations. It supports a possible per-open selector discussion while showing the existing launch-time configuration plus export/import alternative; it does not establish that restarting is always harmless or that cross-profile comparison is semantically equivalent.

@morluto commented on GitHub (Nov 1, 2026): ### Same-project program handoff through existing Evidence export A follow-up on the two-program fixture tests the current recovery path without proposing a silent project switch. With real Ghidra 12.1.4/JDK 21 and REA PR #8437 head `22072c787566e8aa4bbc7eac8659c820a2fc3c08` (the PR changes only prompt wording/tests), I used two separate stdio MCP server processes selecting the same owned project with `REA_GHIDRA_PROGRAM_NAME=program_alpha` and then `program_beta`: 1. In alpha, `analyze_function` returned a summary and retained parent. Adding a hypothetical `program_name: "program_beta"` to `open_binary` returned `invalid_request`; alpha remained usable. The advertised `open_binary` input schema does not contain that field. This is the currently unsupported request, not a promised API regression. 2. Exported the alpha Evidence bundle and closed alpha. In the new beta-configured server, opened the target and analyzed `selected_beta`; its analysis-profile digest differed from alpha's. 3. Imported the alpha bundle while beta was active. `inspect_analysis_view` resolved alpha's original parent ID, and the imported full parent retained alpha's profile digest. A subsequent beta procedure lookup still succeeded. The target and all regular files of the disposable source project had unchanged content hashes; the target was not executed. This is one project, one macOS host and two explicitly configured server processes. It transfers historical Evidence, **not** an editable project or live annotations. It supports a possible per-open selector discussion while showing the existing launch-time configuration plus export/import alternative; it does not establish that restarting is always harmless or that cross-profile comparison is semantically equivalent.

bsakaguchi commented

2026-11-04 04:56:15 +00:00

Author

Owner

@XinwangTao commented on GitHub (Nov 4, 2026):

morluto 的 follow-up 非常有价值。特别是 #8819 这个缺口：caller 手里只有 .gzf 时，从现有错误信息根本发现不了外部导入恢复这条路（只有 stderr 里的配置错误 + 进程退出 1），诊断成本很高。赞同不直接支持隐式导入、保持 packed program 与 .grez 项目归档的区分，但建议至少在错误消息里显式给出恢复路径（如 "import the .gzf with analyzeHeadless into a disposable project, then configure REA_GHIDRA_PROJECT/REA_GHIDRA_PROGRAM_NAME"），而不是让调用者自己去排查。对 #8821 的配置失败隔离也支持——把启动期配置错误变成 open_binary 可读的明确错误，比让 MCP SDK 初始化直接失败更容易让客户端正确降级。希望看到这两个 design issue 落地。

@XinwangTao commented on GitHub (Nov 4, 2026): morluto 的 follow-up 非常有价值。特别是 #8819 这个缺口：caller 手里只有 .gzf 时，从现有错误信息根本发现不了外部导入恢复这条路（只有 stderr 里的配置错误 + 进程退出 1），诊断成本很高。赞同不直接支持隐式导入、保持 packed program 与 .grez 项目归档的区分，但建议至少在错误消息里显式给出恢复路径（如 "import the .gzf with analyzeHeadless into a disposable project, then configure REA_GHIDRA_PROJECT/REA_GHIDRA_PROGRAM_NAME"），而不是让调用者自己去排查。对 #8821 的配置失败隔离也支持——把启动期配置错误变成 open_binary 可读的明确错误，比让 MCP SDK 初始化直接失败更容易让客户端正确降级。希望看到这两个 design issue 落地。

Rows
Columns

[GH-ISSUE #3751] Ghidra provider: open an existing analysed project instead of re-importing every session (large native binaries) #1456

Summary

Measurements (rea-agents 6.5.0, Ghidra 12.1.4, JDK 21, Windows 11 x64, 63 GB RAM)

Proposal

Snapshot control after changing the same selected project

Real CLI refusal and recovery control

Packed-program admission and explicit recovery

Same-project program handoff through existing Evidence export