morluto/rea
1
0
Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00
Code Issues 1.3k Projects Packages Wiki Activity

[GH-ISSUE #1048] [Feature] Open a pre-imported Ghidra project so import and auto-analysis can happen outside the startup deadline #383

New issue
Closed
opened 2026-10-15 18:24:29 +00:00 by bsakaguchi · 19 comments
bsakaguchi commented 2026-10-15 18:24:29 +00:00
Owner
Copy link

Originally created by @corysutyak on GitHub (Oct 8, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1048

  • I searched the open and closed issues and pull requests for a related proposal.
  • I removed credentials, tokens, private paths, and sensitive target data.

Area: Native analysis or decompilation

Current limitation

The provider startup deadline covers Ghidra import, auto-analysis, bridge and health readiness, and each session imports the target into a fresh temporary project (-import … -readOnly -deleteProject). For a target whose import plus default auto-analysis exceeds that deadline, the target can never be opened at all: every first query fails, the owned Ghidra process is torn down, and no partial analysis is returned, so the whole cost is paid again on each attempt.

There is no way to pay that import cost once and reuse the result. Snapshots do not help: DirectAnalysis loads a snapshot and skips the provider entirely (AnalysisSnapshotCache: "Loaded from a local REA analysis snapshot; this call did not re-run the provider"), so a snapshot caches analysis results, not an analysed database, and can only exist after a run that already completed inside the deadline.

Raising the deadline (proposed in #974) is necessary but only moves the wall; it does not let an expensive import be performed once and reused across sessions.

Use cases

  • A 64 MB x86-64 PE executable: import plus default auto-analysis exceeds the 330 s deadline, so no Ghidra-backed query can run against it. Measured in a container with REA 5.0.0, Ghidra 12.1.4, JDK 21: provider_timeout, details.operation: list_segments, details.timeout_ms: 330000, exit 1.
  • Iterative investigation of one large target across several sessions, where the identical import is currently repeated — and lost — every time.
  • Agent or CI workflows that re-issue the same first query, where a deterministic pre-pay step would make the workflow predictable.

Desired outcome and success criteria

Either of:

  1. Open an existing Ghidra project for a target, read-only, so a project produced by other means can be analysed through REA; or
  2. A documented pre-import step that produces an artifact REA can open without re-running import and auto-analysis.

Success criteria: the second and subsequent sessions open such a target within the ordinary per-call budget rather than the startup deadline; the artifact identity is bound to the target digest so a stale or mismatched project is rejected explicitly rather than silently analysed; and the documentation states the provenance and mutation guarantees.

Authority boundaries and non-goals

  • REA must not create, modify or save a user-owned project. Current behaviour ("REA does not open or modify user-owned Ghidra projects") should extend to a read-only open of a caller-supplied project, with any mutation still discarded on close.
  • Provider licensing and the bring-your-own-engine boundary are unchanged.
  • No change to the fresh-import path or its deadline; this is an additional, opt-in route.
  • Not a request to keep temporary projects alive implicitly, and not a general persistent-database service.

Alternatives considered

  • Raising the deadline (#974) — necessary, but partial: it moves the wall and still discards the work on failure, and no single default can cover very large targets.
  • Snapshots — verified above that they skip the provider, so they cannot be pre-built for a target that cannot finish inside the deadline.
  • Running analyzeHeadless manually outside REA — produces a project, but there is currently no way to hand that project to REA.
  • MCP client timeouts — a client-side timeout is separate and fires earlier; it cannot rescue a first query that the provider itself abandons.

Possible direction

An opt-in provider option (CLI flag plus MCP parameter plus a documented environment variable) that points at an existing project for the selected target, opened read-only with the target digest verified against the session binding; plus a documented, explicitly-bounded pre-import command as the supported way to produce that project. Treating it as a proposal, not a requirement.

References

  • #974 — make the Ghidra startup deadline configurable (the adjacent, partial fix).
  • docs/installation.md — the provider startup deadline paragraph ("The provider startup deadline is 330,000 ms for import, analysis, bridge, and health readiness…").
  • src/application/DirectAnalysis.ts and src/application/binary/AnalysisSnapshotCache.ts — snapshot load path and the "did not re-run the provider" message.
  • docs/mcp-contracts.md — Ghidra first-query deadlines and recovery.
Originally created by @corysutyak on GitHub (Oct 8, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1048 - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I removed credentials, tokens, private paths, and sensitive target data. **Area:** Native analysis or decompilation ### Current limitation The provider startup deadline covers Ghidra import, auto-analysis, bridge and health readiness, and each session imports the target into a fresh temporary project (`-import … -readOnly -deleteProject`). For a target whose import plus default auto-analysis exceeds that deadline, the target can never be opened at all: every first query fails, the owned Ghidra process is torn down, and no partial analysis is returned, so the whole cost is paid again on each attempt. There is no way to pay that import cost once and reuse the result. Snapshots do not help: `DirectAnalysis` loads a snapshot and skips the provider entirely (`AnalysisSnapshotCache`: *"Loaded from a local REA analysis snapshot; this call did not re-run the provider"*), so a snapshot caches analysis results, not an analysed database, and can only exist after a run that already completed inside the deadline. Raising the deadline (proposed in #974) is necessary but only moves the wall; it does not let an expensive import be performed once and reused across sessions. ### Use cases - A 64 MB x86-64 PE executable: import plus default auto-analysis exceeds the 330 s deadline, so no Ghidra-backed query can run against it. Measured in a container with REA 5.0.0, Ghidra 12.1.4, JDK 21: `provider_timeout`, `details.operation: list_segments`, `details.timeout_ms: 330000`, exit 1. - Iterative investigation of one large target across several sessions, where the identical import is currently repeated — and lost — every time. - Agent or CI workflows that re-issue the same first query, where a deterministic pre-pay step would make the workflow predictable. ### Desired outcome and success criteria Either of: 1. **Open an existing Ghidra project** for a target, read-only, so a project produced by other means can be analysed through REA; or 2. **A documented pre-import step** that produces an artifact REA can open without re-running import and auto-analysis. Success criteria: the second and subsequent sessions open such a target within the ordinary per-call budget rather than the startup deadline; the artifact identity is bound to the target digest so a stale or mismatched project is rejected explicitly rather than silently analysed; and the documentation states the provenance and mutation guarantees. ### Authority boundaries and non-goals - REA must not create, modify or save a user-owned project. Current behaviour ("REA does not open or modify user-owned Ghidra projects") should extend to a read-only open of a caller-supplied project, with any mutation still discarded on close. - Provider licensing and the bring-your-own-engine boundary are unchanged. - No change to the fresh-import path or its deadline; this is an additional, opt-in route. - Not a request to keep temporary projects alive implicitly, and not a general persistent-database service. ### Alternatives considered - **Raising the deadline (#974)** — necessary, but partial: it moves the wall and still discards the work on failure, and no single default can cover very large targets. - **Snapshots** — verified above that they skip the provider, so they cannot be pre-built for a target that cannot finish inside the deadline. - **Running `analyzeHeadless` manually outside REA** — produces a project, but there is currently no way to hand that project to REA. - **MCP client timeouts** — a client-side timeout is separate and fires earlier; it cannot rescue a first query that the provider itself abandons. ### Possible direction An opt-in provider option (CLI flag plus MCP parameter plus a documented environment variable) that points at an existing project for the selected target, opened read-only with the target digest verified against the session binding; plus a documented, explicitly-bounded pre-import command as the supported way to produce that project. Treating it as a proposal, not a requirement. ### References - #974 — make the Ghidra startup deadline configurable (the adjacent, partial fix). - `docs/installation.md` — the provider startup deadline paragraph ("The provider startup deadline is 330,000 ms for import, analysis, bridge, and health readiness…"). - `src/application/DirectAnalysis.ts` and `src/application/binary/AnalysisSnapshotCache.ts` — snapshot load path and the "did not re-run the provider" message. - `docs/mcp-contracts.md` — Ghidra first-query deadlines and recovery.
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@morluto commented on GitHub (Oct 9, 2026):

Triage note (2026-10-09): keeping open. #974 is merged (configurable startup deadline), but as this issue notes, raising the deadline only moves the wall — paying import/analysis once via a pre-imported project and reusing it across sessions remains unaddressed.

<!-- gh-comment-id:6072409525 --> @morluto commented on GitHub (Oct 9, 2026): Triage note (2026-10-09): keeping open. #974 is merged (configurable startup deadline), but as this issue notes, raising the deadline only moves the wall — paying import/analysis once via a pre-imported project and reusing it across sessions remains unaddressed.
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@N0zoM1z0 commented on GitHub (Oct 9, 2026):

A fresh DX-Ball audit still supports this issue's distinction between cached results and reusable provider state. On main f4c99ab8a53806dd57dc24b3d5a600905bcb4686, real Linux Ghidra 12.1.4 with a 512 MiB Java heap/two CPUs took approximately 3.24 s to open and 29.31 s for the first uncached procedure query; warm reads were milliseconds. Npm latest 6.1.0 also paid cold import cost (approximately 35.82 s for its first query). These are individual runs under resource constraints, not controlled throughput benchmarks or a startup-deadline failure.

After saving/reopening on current main, an eligible explicit-document primitive took approximately 53 ms and a dossier 119 ms without starting Ghidra; their observations were preserved. A new query remains distinct from that replay. #1249 tracks eager result-snapshot load/save cost and #1255 clarifies supported heap/CPU controls; neither supplies a persistent imported database. #1252 covers improved progress visibility. No competing provider-persistence issue was opened.

<!-- gh-comment-id:6078171154 --> @N0zoM1z0 commented on GitHub (Oct 9, 2026): A fresh DX-Ball audit still supports this issue's distinction between cached results and reusable provider state. On main `f4c99ab8a53806dd57dc24b3d5a600905bcb4686`, real Linux Ghidra 12.1.4 with a 512 MiB Java heap/two CPUs took approximately 3.24 s to open and 29.31 s for the first uncached procedure query; warm reads were milliseconds. Npm latest 6.1.0 also paid cold import cost (approximately 35.82 s for its first query). These are individual runs under resource constraints, not controlled throughput benchmarks or a startup-deadline failure. After saving/reopening on current main, an eligible explicit-document primitive took approximately 53 ms and a dossier 119 ms without starting Ghidra; their observations were preserved. A new query remains distinct from that replay. #1249 tracks eager result-snapshot load/save cost and #1255 clarifies supported heap/CPU controls; neither supplies a persistent imported database. #1252 covers improved progress visibility. No competing provider-persistence issue was opened.
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@CassieFox23 commented on GitHub (Oct 9, 2026):

TL;DR: With the 30-minute limit, the big game exe does open, but it takes 20 minutes every time. The saved snapshot doesn't save that wait.

Data point on 6.1.0 with REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000: a 60 MB x86-64 PE game exe (219,702 functions) took 1,203 s to open cold. A decompile with --snapshot from that run re-imported it, taking 1,247 s. So the raised deadline makes it usable. But each new session still pays about 20 minutes on its first question. That matches what this issue describes.

Test setup
  • Box: Linux x86-64
  • Other: Ghidra 12.1.4, JDK 21

🤖 Generated with Claude Code (Opus 5.5) 🦊

<!-- gh-comment-id:6085334759 --> @CassieFox23 commented on GitHub (Oct 9, 2026): **TL;DR:** With the 30-minute limit, the big game exe does open, but it takes 20 minutes every time. The saved snapshot doesn't save that wait. Data point on 6.1.0 with `REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000`: a 60 MB x86-64 PE game exe (219,702 functions) took 1,203 s to open cold. A `decompile` with `--snapshot` from that run re-imported it, taking 1,247 s. So the raised deadline makes it usable. But each new session still pays about 20 minutes on its first question. That matches what this issue describes. <details> <summary>Test setup</summary> - Box: Linux x86-64 - Other: Ghidra 12.1.4, JDK 21 </details> 🤖 Generated with Claude Code (Opus 5.5) 🦊
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@CassieFox23 commented on GitHub (Oct 9, 2026):

TL;DR: Same on 6.2.0. Opening still takes about 20 minutes. And the snapshot still re-imports instead of skipping it.

Follow-up on 6.2.0, same exe and setup (REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000). Cold analyze took 1,208 s. And decompile with --snapshot from that run took 1,185 s. So on 6.2.0 the snapshot still doesn't skip the Ghidra import on this path, the same as 6.1.0 (1,203 s / 1,247 s).

Test setup
  • Box: Linux x86-64
  • Other: Ghidra 12.1.4, JDK 21

🤖 Generated with Claude Code (Opus 5.5) 🦊

<!-- gh-comment-id:6086616291 --> @CassieFox23 commented on GitHub (Oct 9, 2026): **TL;DR:** Same on 6.2.0. Opening still takes about 20 minutes. And the snapshot still re-imports instead of skipping it. Follow-up on 6.2.0, same exe and setup (`REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000`). Cold `analyze` took 1,208 s. And `decompile` with `--snapshot` from that run took 1,185 s. So on 6.2.0 the snapshot still doesn't skip the Ghidra import on this path, the same as 6.1.0 (1,203 s / 1,247 s). <details> <summary>Test setup</summary> - Box: Linux x86-64 - Other: Ghidra 12.1.4, JDK 21 </details> 🤖 Generated with Claude Code (Opus 5.5) 🦊
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@CassieFox23 commented on GitHub (Oct 10, 2026):

TL;DR: I read main. Nothing there reuses a Ghidra project yet. I can build or test a small first step.

What I found on main at 7aa4d76:

  • Each session imports into a fresh temp project.
  • GhidraLauncher.ts passes -import, -readOnly and -deleteProject.
  • Snapshots only replay an exact query.
  • A new address or tool starts Ghidra again.
  • docs/cli.md says the same: "not a saved provider database".
  • The bridge already checks the program's SHA-256.
  • A mismatched program fails before it serves.
  • The 6.3.0 notes have no change here.

So each new question on my game exe still costs 20 minutes.

A possible first slice:

  • An opt-in setting names an existing project and program.
  • REA then runs -process <name> -readOnly -noanalysis.
  • No -import, so no import and no analysis.
  • Ghidra's README says -readOnly drops any changes.
  • It also says existing projects are never deleted.
  • The digest check binds the project to the target.
  • The profile gets a new import_mode value.
  • So old snapshots never mix with these runs.
  • Linux and macOS first, not the Windows path.
  • Making the project stays a manual analyzeHeadless step.
  • A helper command for that could come later.

Two things I'd rather ask than guess:

  • Open the project in place, or copy it first?
  • A copy keeps the user's project untouched.
  • Setting only at first, or the MCP parameter too?

I can build it, or test whatever shape you prefer.
I'd test a small DLL first, then the 60 MB exe.
Which shape fits, if any?

🤖 Generated with Claude Code (Opus 5.5) 🦊

<!-- gh-comment-id:6092190259 --> @CassieFox23 commented on GitHub (Oct 10, 2026): **TL;DR:** I read main. Nothing there reuses a Ghidra project yet. I can build or test a small first step. What I found on main at `7aa4d76`: - Each session imports into a fresh temp project. - `GhidraLauncher.ts` passes `-import`, `-readOnly` and `-deleteProject`. - Snapshots only replay an exact query. - A new address or tool starts Ghidra again. - `docs/cli.md` says the same: "not a saved provider database". - The bridge already checks the program's SHA-256. - A mismatched program fails before it serves. - The 6.3.0 notes have no change here. So each new question on my game exe still costs 20 minutes. A possible first slice: - An opt-in setting names an existing project and program. - REA then runs `-process <name> -readOnly -noanalysis`. - No `-import`, so no import and no analysis. - Ghidra's README says `-readOnly` drops any changes. - It also says existing projects are never deleted. - The digest check binds the project to the target. - The profile gets a new `import_mode` value. - So old snapshots never mix with these runs. - Linux and macOS first, not the Windows path. - Making the project stays a manual `analyzeHeadless` step. - A helper command for that could come later. Two things I'd rather ask than guess: - Open the project in place, or copy it first? - A copy keeps the user's project untouched. - Setting only at first, or the MCP parameter too? I can build it, or test whatever shape you prefer. I'd test a small DLL first, then the 60 MB exe. Which shape fits, if any? 🤖 Generated with Claude Code (Opus 5.5) 🦊
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@ylwl1997 commented on GitHub (Oct 10, 2026):

Agree with the triage note: raising the deadline (via #974) makes large targets possible, but it does not remove the repeated cold-import tax that CassieFox23 measured (~20 minutes per new session on a ~60 MB PE).

User-facing workaround checklist while a pre-imported / -process path is still a feature ask:

  1. Raise startup patience for the shell/MCP process that launches REA, e.g. export REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 (30 minutes — confirm the current name in your version’s docs). Without this, cold import on big binaries dies as provider_timeout even though Ghidra is still working.
  2. Give the JVM room. Pair a documented Ghidra release with Temurin 21+ and a heap sized for the target (constrained heaps make cold open look like a timeout).
  3. Expect cold vs warm asymmetry. First uncached procedure query often costs tens of seconds to many minutes; subsequent reads in the same provider lifetime are much cheaper. Plan agent/CI flows around that (one long session > many short ones).
  4. Snapshots ≠ reusable Ghidra DB. Current snapshots replay exact prior results; they do not skip -import for a new address/tool. Do not expect --snapshot to amortize a 20-minute import.
  5. Scope when possible. If the product allows a smaller first slice / subset, prefer that over always importing the full image interactively.
  6. Debug loop: npx -y rea-agents@latest doctor --provider ghidra --json, then one tiny known-good binary, then the large target once with the raised timeout.

Still strongly +1 on an opt-in “open existing project read-only / -process … -noanalysis” path so import can happen outside the interactive deadline.

Disclosure: I maintain rea.run, an independent bilingual docs/FAQ site about lawful coding-agent reverse engineering. Not affiliated with the REA project.

<!-- gh-comment-id:6095659675 --> @ylwl1997 commented on GitHub (Oct 10, 2026): Agree with the triage note: raising the deadline (via #974) makes large targets *possible*, but it does not remove the repeated cold-import tax that CassieFox23 measured (~20 minutes per new session on a ~60 MB PE). **User-facing workaround checklist** while a pre-imported / `-process` path is still a feature ask: 1. **Raise startup patience** for the shell/MCP process that launches REA, e.g. `export REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000` (30 minutes — confirm the current name in your version’s docs). Without this, cold import on big binaries dies as `provider_timeout` even though Ghidra is still working. 2. **Give the JVM room.** Pair a documented Ghidra release with Temurin 21+ and a heap sized for the target (constrained heaps make cold open look like a timeout). 3. **Expect cold vs warm asymmetry.** First uncached procedure query often costs tens of seconds to many minutes; subsequent reads in the *same* provider lifetime are much cheaper. Plan agent/CI flows around that (one long session > many short ones). 4. **Snapshots ≠ reusable Ghidra DB.** Current snapshots replay exact prior results; they do not skip `-import` for a new address/tool. Do not expect `--snapshot` to amortize a 20-minute import. 5. **Scope when possible.** If the product allows a smaller first slice / subset, prefer that over always importing the full image interactively. 6. **Debug loop:** `npx -y rea-agents@latest doctor --provider ghidra --json`, then one tiny known-good binary, then the large target once with the raised timeout. Still strongly +1 on an opt-in “open existing project read-only / `-process … -noanalysis`” path so import can happen outside the interactive deadline. Disclosure: I maintain [rea.run](https://rea.run/docs/ghidra-large-binaries/), an independent bilingual docs/FAQ site about lawful coding-agent reverse engineering. Not affiliated with the REA project.
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

A read-only existing-project path is a useful first slice for the repeated cold-import cost. Start with copy-first into an owned temporary root, select the project and program explicitly, and use -process / -readOnly / -noanalysis there. Keep original project files untouched and keep fresh import as the default. The copy cost and temporary disk requirement should be visible in the resulting profile/diagnostics.

Expose the same meaningful project/program selection through the shared CLI/MCP workflow rather than requiring a machine-global setting for every target. Provider-specific launch and project-copy details belong in the Ghidra boundary. A configuration default can follow if needed; no separate prepare/execute protocol or approval boolean is required.

Please retain the program SHA-256 binding check before serving results, distinguish the existing-project mode in provider provenance and snapshot compatibility, and reject missing/ambiguous programs, unsupported project forms, digest mismatches and unavailable/locked projects with their actual reasons. Do not infer that -readOnly alone covers all filesystem effects; verify original project contents/metadata before and after successful open, failure and cancellation.

For a focused PR, real Linux/macOS verification should demonstrate first and repeated sessions without auto-analysis, new-address/tool queries using the same pre-imported program, mismatch rejection, and owned cleanup. Start with the small target you proposed, then report the large-target timing separately. Leave the current Windows P0 boundary unchanged in this first slice.

<!-- gh-comment-id:6096149914 --> @N0zoM1z0 commented on GitHub (Oct 10, 2026): A read-only existing-project path is a useful first slice for the repeated cold-import cost. Start with **copy-first** into an owned temporary root, select the project and program explicitly, and use `-process` / `-readOnly` / `-noanalysis` there. Keep original project files untouched and keep fresh import as the default. The copy cost and temporary disk requirement should be visible in the resulting profile/diagnostics. Expose the same meaningful project/program selection through the shared CLI/MCP workflow rather than requiring a machine-global setting for every target. Provider-specific launch and project-copy details belong in the Ghidra boundary. A configuration default can follow if needed; no separate prepare/execute protocol or approval boolean is required. Please retain the program SHA-256 binding check before serving results, distinguish the existing-project mode in provider provenance and snapshot compatibility, and reject missing/ambiguous programs, unsupported project forms, digest mismatches and unavailable/locked projects with their actual reasons. Do not infer that `-readOnly` alone covers all filesystem effects; verify original project contents/metadata before and after successful open, failure and cancellation. For a focused PR, real Linux/macOS verification should demonstrate first and repeated sessions without auto-analysis, new-address/tool queries using the same pre-imported program, mismatch rejection, and owned cleanup. Start with the small target you proposed, then report the large-target timing separately. Leave the current Windows P0 boundary unchanged in this first slice.
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@N0zoM1z0 commented on GitHub (Oct 11, 2026):

Independent current-main Windows x64 packaged verification at f88c99021b88ce67e513025901f57478d8107d0c adds real native selected-view and snapshot coverage. The fresh local npm installation uses integrity-verified locked production archives, Node 22.19.0, the matching 6.3.0 native-controls manifest/artifact, operator-owned Ghidra 12.1.4 and Java 21. The source-owned PE is 1,024 bytes and is analyzed statically, never executed.

  • The default SDK 60-second cold overview timed out; a separate 240-second cold attempt also timed out. Both cancelled requests cleaned their provider runtime. The connection remained usable after the default attempt.
  • A separate attempt with a 360-second SDK request completed the cold overview in 262.149 seconds. It reported two procedures and two segments. This longer deadline is a measured control, not a general recovery guarantee or a fix for #1048.
  • Real analyze_function returned 42,650 response bytes in 198.9 ms after initialization. Retained procedure and assembly views returned 18,384/17,634 bytes in 22.5/5.8 ms; assembly rows matched the complete parent. Text/structured content matched.
  • Snapshot save/close, fresh MCP process, snapshot import and selected procedure replay passed. Original view Evidence was identical; the replay took 26.7 ms and allocated no Ghidra runtime. Target SHA-256 was unchanged and owned runtime directories were empty after final close.
  • The successful guarded lane peaked at 831,119,360 bytes of observed process-family working set. The actual JVM had -Xmx512M, affinity mask 3, and BelowNormal priority; the supervisor did not stop the run. These are single-run observations on a tiny PE, not large-native or latency-percentile coverage.
  • Progress notifications were operation start/completion only. Combined connection plus full 170-tool catalog preparation took 35.1 seconds initially and 29.7 seconds after reconnect; this combined timing does not isolate SDK compilation from server startup. A separate phase measurement is being taken before attributing that cost.

Windows first-analysis deadlines therefore remain open. This verifies two selected native facets and snapshot reuse on Windows; the source-owned oversized dossier and all 15 collection checks in #1247 remain Linux observations.

<!-- gh-comment-id:6104891834 --> @N0zoM1z0 commented on GitHub (Oct 11, 2026): Independent current-main Windows x64 packaged verification at `f88c99021b88ce67e513025901f57478d8107d0c` adds real native selected-view and snapshot coverage. The fresh local npm installation uses integrity-verified locked production archives, Node 22.19.0, the matching 6.3.0 native-controls manifest/artifact, operator-owned Ghidra 12.1.4 and Java 21. The source-owned PE is 1,024 bytes and is analyzed statically, never executed. - The default SDK 60-second cold overview timed out; a separate 240-second cold attempt also timed out. Both cancelled requests cleaned their provider runtime. The connection remained usable after the default attempt. - A separate attempt with a 360-second SDK request completed the cold overview in 262.149 seconds. It reported two procedures and two segments. This longer deadline is a measured control, not a general recovery guarantee or a fix for #1048. - Real `analyze_function` returned 42,650 response bytes in 198.9 ms after initialization. Retained procedure and assembly views returned 18,384/17,634 bytes in 22.5/5.8 ms; assembly rows matched the complete parent. Text/structured content matched. - Snapshot save/close, fresh MCP process, snapshot import and selected procedure replay passed. Original view Evidence was identical; the replay took 26.7 ms and allocated no Ghidra runtime. Target SHA-256 was unchanged and owned runtime directories were empty after final close. - The successful guarded lane peaked at 831,119,360 bytes of observed process-family working set. The actual JVM had `-Xmx512M`, affinity mask 3, and BelowNormal priority; the supervisor did not stop the run. These are single-run observations on a tiny PE, not large-native or latency-percentile coverage. - Progress notifications were operation start/completion only. Combined connection plus full 170-tool catalog preparation took 35.1 seconds initially and 29.7 seconds after reconnect; this combined timing does not isolate SDK compilation from server startup. A separate phase measurement is being taken before attributing that cost. Windows first-analysis deadlines therefore remain open. This verifies two selected native facets and snapshot reuse on Windows; the source-owned oversized dossier and all 15 collection checks in #1247 remain Linux observations.
bsakaguchi commented 2026-10-15 18:24:31 +00:00
Author
Owner
Copy link

@elkahtersidi commented on GitHub (Oct 11, 2026):

Cv

<!-- gh-comment-id:6105061059 --> @elkahtersidi commented on GitHub (Oct 11, 2026): Cv
bsakaguchi commented 2026-10-15 18:24:32 +00:00
Author
Owner
Copy link

@CassieFox23 commented on GitHub (Oct 11, 2026):

TL;DR: On Linux, the cold overview of a 1,024-byte test DLL took 7 to 16 seconds, depending on how busy my box was. On Windows, N0zoM1z0's run took 262 seconds. I do not know why the two differ.

Thanks, N0zoM1z0, for the Windows numbers. I ran the same test flow on Linux. The file is the 1,024-byte x86-64 DLL that scripts/create-ghidra-windows-fixture.mjs writes. The x86 DLL from that script is also 1,024 bytes, so I picked the x86-64 one. It gives your counts, two procedures and two segments. I did not confirm it is your file.

Check Windows (from N0zoM1z0's comment) Linux, REA defaults Linux, 512 MiB heap, 2 CPUs
Cold overview, busy box 262.1 s (360 s limit) 13.6 to 14.2 s (4 runs) 10.5 and 16.1 s (2 runs)
Cold overview, quiet box not run 6.9 to 7.1 s (3 runs) 9.0 to 9.3 s (3 runs)
60 s default limit timed out passed (2 runs, 14.2 and 15.0 s) not run
240 s limit timed out not run not run
Procedures, segments 2, 2 2, 2 2, 2
Connect plus 170-tool list, busy 35.1 s, then 29.7 s 2.7 to 2.8 s 3.0 to 3.2 s
Connect plus 170-tool list, quiet not run 1.4 to 1.5 s 1.5 to 1.6 s
analyze_function size 42,650 bytes 22,431 bytes 22,431 bytes
analyze_function time, busy 198.9 ms 118 to 139 ms 53 and 119 ms
analyze_function time, quiet not run 48 to 70 ms 53 to 60 ms
Java heap 512 MiB 2 GiB 512 MiB
CPU limit mask 3 none CPUs 0 and 1
Peak memory 831,119,360 bytes (793 MiB) working set 1,011 to 1,255 MiB RSS 987 to 1,023 MiB RSS
Node 22.19.0 24.18.0 24.18.0

RSS is the server plus its Java child, sampled each second. On main, Node 22.23.1 took 14.6 s and 15.4 s busy, and 7.2 s quiet. Node 24.18.0 took 13.6 to 14.2 s busy and 6.9 to 7.1 s quiet.

What the numbers do not say

  • I ran main 92dac9710 (#1689). You ran f88c99021, 33 commits earlier. I also built f88c99021. It took 6.8 to 7.0 s cold at load 5 and 6.9 to 7.1 s at load 10 to 13. Main took 6.9 to 7.1 s at load 5.
  • Windows ran Java at BelowNormal priority. I did not.
  • Your CPU mask 3 may be two cores or one core's two threads. My CPUs 0 and 1 are two different cores.
  • My first 60 s run took 15.0 s. I redid it after a bug in my script's counting, so it is outside the 13.6 to 14.2 s range.
  • I ran a built source tree (node scripts/rea.mjs mcp), not the packaged npm install.
  • Load average counts programs running or waiting to run, averaged over one minute. This box has 24 hardware threads. Other jobs shared it. The load was about 28 on the busy runs and about 5 on the quiet runs.
  • The analyze_function sizes differ. Your procedure is not named. I used fixture_caller. I do not know the cause.
  • I do not know what makes Windows slower here.

Not run

  • The packaged npm install.
  • Snapshot save, close, import and replay.
  • Retained views of a procedure and its assembly.
  • Large binaries.
  • Anything on Windows.
Commands 
git -C ~/code/rea-mirror/rea.git archive 92dac97102190395613c868b1582efc8a4f054c8 | tar -x -C /tmp/p19-1048/main
npm ci && npm run build          # Node 24.18.0, exit 0 for both
node scripts/create-ghidra-windows-fixture.mjs   # writes build/fixtures/rea-ghidra-windows.dll
# per run, a stdio MCP client from @modelcontextprotocol/client 2.3.1:
REA_ANALYSIS_PROVIDER=ghidra node scripts/rea.mjs mcp
#   connect, tools/list, open_binary, binary_overview (60 s default or 360 s limit),
#   list_procedures, list_segments, analyze_function, binary_overview again, close_binary
# 512 MiB runs:
GHIDRA_HEADLESS_MAXMEM=512M taskset -c 0,1 node scripts/p19-bench.mjs ...

All 23 timed runs exited 0. Each run started a new server and a new Ghidra project. REA's own JVM flags in every run included -XX:ParallelGCThreads=2 -XX:CICompilerCount=2. REA also passed -readOnly -deleteProject to Ghidra's headless launcher. The run temp dirs ended empty.

Every run 
run                             node     commit     jvm            load   cold_s  tools_s
f88-n24-360-1                   24.18.0  f88c99021  2G             13.25  7.0     1.4
f88-n24-360-2                   24.18.0  f88c99021  2G             11.67  6.9     1.4
f88-n24-360-3                   24.18.0  f88c99021  2G             10.18  7.1     1.4
f88-n24-constrained-360-1       24.18.0  f88c99021  512M, 2 CPUs   9.15   8.7     1.5
ilv-f88-n22-360-1               22.23.1  f88c99021  2G             4.98   7.3     1.7
ilv-f88-n24-360-1               24.18.0  f88c99021  2G             5.73   7.0     1.4
ilv-f88-n24-360-2               24.18.0  f88c99021  2G             5.10   6.8     1.5
ilv-f88-n24-360-3               24.18.0  f88c99021  2G             5.05   7.0     1.4
ilv-main-n22-360-1              22.23.1  92dac9710  2G             4.88   7.2     1.8
ilv-main-n24-360-1              24.18.0  92dac9710  2G             6.21   6.9     1.4
ilv-main-n24-360-2              24.18.0  92dac9710  2G             5.39   7.0     1.5
ilv-main-n24-360-3              24.18.0  92dac9710  2G             4.86   7.1     1.5
n22-360-1                       22.23.1  92dac9710  2G             28.88  15.4    3.5
n22-default-1                   22.23.1  92dac9710  2G             28.63  14.6    3.7
n24-360-1                       24.18.0  92dac9710  2G             28.43  13.6    2.7
n24-360-2                       24.18.0  92dac9710  2G             28.56  14.0    2.8
n24-360-3                       24.18.0  92dac9710  2G             29.18  14.0    2.8
n24-constrained-360-1           24.18.0  92dac9710  512M, 2 CPUs   33.33  16.1    3.2
n24-constrained-360-2           24.18.0  92dac9710  512M, 2 CPUs   31.91  10.5    3.0
n24-default-1                   24.18.0  92dac9710  2G             27.74  14.2    2.7
q-main-n24-constrained-360-1    24.18.0  92dac9710  512M, 2 CPUs   4.54   9.0     1.5
q-main-n24-constrained-360-2    24.18.0  92dac9710  512M, 2 CPUs   4.46   9.3     1.5
q-main-n24-constrained-360-3    24.18.0  92dac9710  512M, 2 CPUs   4.37   9.2     1.5

Run names with default in them used the 60 s SDK limit. All others used 360 s.

Test setup
  • Box: Nobara 44 (Fedora 44 base), Linux 7.2.9
  • CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads
  • RAM: 32 GB DDR4-3600
  • Node: 22.23.1, 24.18.0
  • Main: 92dac9710 (#1689); also f88c99021
  • Other: Ghidra 12.1.4 (Apache-2.0) with JDK 21.0.12.1
  • File: rea-ghidra-windows.dll, 1,024 bytes
  • SHA-256: 3d52764593d774ed72f7bff9308a4a7036c69569177fd8191122feb7db082196

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

<!-- gh-comment-id:6105317166 --> @CassieFox23 commented on GitHub (Oct 11, 2026): **TL;DR:** On Linux, the cold overview of a 1,024-byte test DLL took 7 to 16 seconds, depending on how busy my box was. On Windows, N0zoM1z0's run took 262 seconds. I do not know why the two differ. Thanks, N0zoM1z0, for the Windows numbers. I ran the same test flow on Linux. The file is the 1,024-byte x86-64 DLL that `scripts/create-ghidra-windows-fixture.mjs` writes. The x86 DLL from that script is also 1,024 bytes, so I picked the x86-64 one. It gives your counts, two procedures and two segments. I did not confirm it is your file. | Check | Windows (from N0zoM1z0's comment) | Linux, REA defaults | Linux, 512 MiB heap, 2 CPUs | |---|---|---|---| | Cold overview, busy box | 262.1 s (360 s limit) | 13.6 to 14.2 s (4 runs) | 10.5 and 16.1 s (2 runs) | | Cold overview, quiet box | not run | 6.9 to 7.1 s (3 runs) | 9.0 to 9.3 s (3 runs) | | 60 s default limit | timed out | passed (2 runs, 14.2 and 15.0 s) | not run | | 240 s limit | timed out | not run | not run | | Procedures, segments | 2, 2 | 2, 2 | 2, 2 | | Connect plus 170-tool list, busy | 35.1 s, then 29.7 s | 2.7 to 2.8 s | 3.0 to 3.2 s | | Connect plus 170-tool list, quiet | not run | 1.4 to 1.5 s | 1.5 to 1.6 s | | `analyze_function` size | 42,650 bytes | 22,431 bytes | 22,431 bytes | | `analyze_function` time, busy | 198.9 ms | 118 to 139 ms | 53 and 119 ms | | `analyze_function` time, quiet | not run | 48 to 70 ms | 53 to 60 ms | | Java heap | 512 MiB | 2 GiB | 512 MiB | | CPU limit | mask 3 | none | CPUs 0 and 1 | | Peak memory | 831,119,360 bytes (793 MiB) working set | 1,011 to 1,255 MiB RSS | 987 to 1,023 MiB RSS | | Node | 22.19.0 | 24.18.0 | 24.18.0 | RSS is the server plus its Java child, sampled each second. On main, Node 22.23.1 took 14.6 s and 15.4 s busy, and 7.2 s quiet. Node 24.18.0 took 13.6 to 14.2 s busy and 6.9 to 7.1 s quiet. ### What the numbers do not say - I ran main 92dac9710 (#1689). You ran f88c99021, 33 commits earlier. I also built f88c99021. It took 6.8 to 7.0 s cold at load 5 and 6.9 to 7.1 s at load 10 to 13. Main took 6.9 to 7.1 s at load 5. - Windows ran Java at BelowNormal priority. I did not. - Your CPU mask 3 may be two cores or one core's two threads. My CPUs 0 and 1 are two different cores. - My first 60 s run took 15.0 s. I redid it after a bug in my script's counting, so it is outside the 13.6 to 14.2 s range. - I ran a built source tree (`node scripts/rea.mjs mcp`), not the packaged npm install. - Load average counts programs running or waiting to run, averaged over one minute. This box has 24 hardware threads. Other jobs shared it. The load was about 28 on the busy runs and about 5 on the quiet runs. - The `analyze_function` sizes differ. Your procedure is not named. I used `fixture_caller`. I do not know the cause. - I do not know what makes Windows slower here. ### Not run - The packaged npm install. - Snapshot save, close, import and replay. - Retained views of a procedure and its assembly. - Large binaries. - Anything on Windows. <details> <summary>Commands</summary> ``` git -C ~/code/rea-mirror/rea.git archive 92dac97102190395613c868b1582efc8a4f054c8 | tar -x -C /tmp/p19-1048/main npm ci && npm run build # Node 24.18.0, exit 0 for both node scripts/create-ghidra-windows-fixture.mjs # writes build/fixtures/rea-ghidra-windows.dll # per run, a stdio MCP client from @modelcontextprotocol/client 2.3.1: REA_ANALYSIS_PROVIDER=ghidra node scripts/rea.mjs mcp # connect, tools/list, open_binary, binary_overview (60 s default or 360 s limit), # list_procedures, list_segments, analyze_function, binary_overview again, close_binary # 512 MiB runs: GHIDRA_HEADLESS_MAXMEM=512M taskset -c 0,1 node scripts/p19-bench.mjs ... ``` All 23 timed runs exited 0. Each run started a new server and a new Ghidra project. REA's own JVM flags in every run included `-XX:ParallelGCThreads=2 -XX:CICompilerCount=2`. REA also passed `-readOnly -deleteProject` to Ghidra's headless launcher. The run temp dirs ended empty. </details> <details> <summary>Every run</summary> ``` run node commit jvm load cold_s tools_s f88-n24-360-1 24.18.0 f88c99021 2G 13.25 7.0 1.4 f88-n24-360-2 24.18.0 f88c99021 2G 11.67 6.9 1.4 f88-n24-360-3 24.18.0 f88c99021 2G 10.18 7.1 1.4 f88-n24-constrained-360-1 24.18.0 f88c99021 512M, 2 CPUs 9.15 8.7 1.5 ilv-f88-n22-360-1 22.23.1 f88c99021 2G 4.98 7.3 1.7 ilv-f88-n24-360-1 24.18.0 f88c99021 2G 5.73 7.0 1.4 ilv-f88-n24-360-2 24.18.0 f88c99021 2G 5.10 6.8 1.5 ilv-f88-n24-360-3 24.18.0 f88c99021 2G 5.05 7.0 1.4 ilv-main-n22-360-1 22.23.1 92dac9710 2G 4.88 7.2 1.8 ilv-main-n24-360-1 24.18.0 92dac9710 2G 6.21 6.9 1.4 ilv-main-n24-360-2 24.18.0 92dac9710 2G 5.39 7.0 1.5 ilv-main-n24-360-3 24.18.0 92dac9710 2G 4.86 7.1 1.5 n22-360-1 22.23.1 92dac9710 2G 28.88 15.4 3.5 n22-default-1 22.23.1 92dac9710 2G 28.63 14.6 3.7 n24-360-1 24.18.0 92dac9710 2G 28.43 13.6 2.7 n24-360-2 24.18.0 92dac9710 2G 28.56 14.0 2.8 n24-360-3 24.18.0 92dac9710 2G 29.18 14.0 2.8 n24-constrained-360-1 24.18.0 92dac9710 512M, 2 CPUs 33.33 16.1 3.2 n24-constrained-360-2 24.18.0 92dac9710 512M, 2 CPUs 31.91 10.5 3.0 n24-default-1 24.18.0 92dac9710 2G 27.74 14.2 2.7 q-main-n24-constrained-360-1 24.18.0 92dac9710 512M, 2 CPUs 4.54 9.0 1.5 q-main-n24-constrained-360-2 24.18.0 92dac9710 512M, 2 CPUs 4.46 9.3 1.5 q-main-n24-constrained-360-3 24.18.0 92dac9710 512M, 2 CPUs 4.37 9.2 1.5 ``` Run names with default in them used the 60 s SDK limit. All others used 360 s. </details> <details> <summary>Test setup</summary> - Box: Nobara 44 (Fedora 44 base), Linux 7.2.9 - CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads - RAM: 32 GB DDR4-3600 - Node: 22.23.1, 24.18.0 - Main: 92dac9710 (#1689); also f88c99021 - Other: Ghidra 12.1.4 ([Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/master/LICENSE)) with JDK 21.0.12.1 - File: `rea-ghidra-windows.dll`, 1,024 bytes - SHA-256: `3d52764593d774ed72f7bff9308a4a7036c69569177fd8191122feb7db082196` </details> 🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊
bsakaguchi commented 2026-10-15 18:24:32 +00:00
Author
Owner
Copy link

@pablodiazjorge commented on GitHub (Oct 11, 2026):

Data point from another real Windows host, with the same 1,024-byte x86-64 DLL that scripts/create-ghidra-windows-fixture.mjs writes (SHA-256 3d527645…db082196). Here the cold overview takes about 11 s, close to the Linux numbers above and far from 262 s. Replicating the constraints of the 262 s run (512 MiB heap, affinity mask 3, BelowNormal priority) brings it to about 20 s, so those constraints alone do not explain the gap on this host.

Check REA defaults, main fb43dd97 (3 runs) 512 MiB heap, mask 3, BelowNormal, main (3 runs) npm 6.4.0 package, defaults (2 runs)
Cold binary_overview 11.2 s (11.18 to 11.23) 20.5 s (20.13 to 20.92) 10.7 and 11.0 s
Server start plus initialize 1.0 to 1.1 s 1.3 to 1.4 s 1.5 and 1.6 s
tools/list 0.24 s, 172 tools 0.33 to 0.34 s, 172 tools 0.23 s, 170 tools
analyze_function, first procedure 0.05 s, 48,868 bytes 0.06 to 0.10 s, 48,868 bytes 0.05 s, 48,758 bytes
Procedures 2 2 2
Java heap and scheduling, as observed on java.exe -Xmx2G, all CPUs, Normal -Xmx512M, mask 3, BelowNormal -Xmx2G, all CPUs, Normal

How the constraints were applied: GHIDRA_HEADLESS_MAXMEM=512M (REA passes it as -Xmx), and affinity 3 plus BelowNormal set on the MCP server process right after it started, before the first Ghidra query. The JVM inherited them, as the last row shows. On this CPU, mask 3 is the two hyperthreads of one performance core.

Host: Windows 11 Pro 10.0.26300, Intel i7-12700KF (12 cores, 20 threads), 32 GB, NVMe SSD, Defender real-time protection on with no exclusions, non-elevated user, Node 22.22.3, Temurin 21.0.12.1, Ghidra 12.1.4. Main ran from a built source tree (node scripts/rea.mjs mcp); the last column is the packaged npm 6.4.0. Every run started a fresh MCP server over stdio and measured each request from the client. The first run of the session, not in the table, took 12.3 s for the cold overview and 3.9 s to initialize.

What it suggests: the slow numbers on the other host look host-specific rather than Windows-specific. Connect plus the full catalog took 35.1 s there and about 1.3 s here. A per-phase breakdown on that host (JVM start, import, analysis, bridge readiness) would show where the time goes, and I can run the same script here for a side-by-side comparison.

Not covered: snapshot save and replay, large targets, the guarded memory supervisor and peak memory. The analyze_function size also differs between hosts (48,868 bytes here, 42,650 in the 262 s run, 22,431 on Linux); I did not look into why.

<!-- gh-comment-id:6112374756 --> @pablodiazjorge commented on GitHub (Oct 11, 2026): Data point from another real Windows host, with the same 1,024-byte x86-64 DLL that `scripts/create-ghidra-windows-fixture.mjs` writes (SHA-256 `3d527645…db082196`). Here the cold overview takes about 11 s, close to the Linux numbers above and far from 262 s. Replicating the constraints of the 262 s run (512 MiB heap, affinity mask 3, BelowNormal priority) brings it to about 20 s, so those constraints alone do not explain the gap on this host. | Check | REA defaults, main `fb43dd97` (3 runs) | 512 MiB heap, mask 3, BelowNormal, main (3 runs) | npm 6.4.0 package, defaults (2 runs) | |---|---|---|---| | Cold `binary_overview` | 11.2 s (11.18 to 11.23) | 20.5 s (20.13 to 20.92) | 10.7 and 11.0 s | | Server start plus `initialize` | 1.0 to 1.1 s | 1.3 to 1.4 s | 1.5 and 1.6 s | | `tools/list` | 0.24 s, 172 tools | 0.33 to 0.34 s, 172 tools | 0.23 s, 170 tools | | `analyze_function`, first procedure | 0.05 s, 48,868 bytes | 0.06 to 0.10 s, 48,868 bytes | 0.05 s, 48,758 bytes | | Procedures | 2 | 2 | 2 | | Java heap and scheduling, as observed on `java.exe` | `-Xmx2G`, all CPUs, Normal | `-Xmx512M`, mask 3, BelowNormal | `-Xmx2G`, all CPUs, Normal | How the constraints were applied: `GHIDRA_HEADLESS_MAXMEM=512M` (REA passes it as `-Xmx`), and affinity 3 plus BelowNormal set on the MCP server process right after it started, before the first Ghidra query. The JVM inherited them, as the last row shows. On this CPU, mask 3 is the two hyperthreads of one performance core. Host: Windows 11 Pro 10.0.26300, Intel i7-12700KF (12 cores, 20 threads), 32 GB, NVMe SSD, Defender real-time protection on with no exclusions, non-elevated user, Node 22.22.3, Temurin 21.0.12.1, Ghidra 12.1.4. Main ran from a built source tree (`node scripts/rea.mjs mcp`); the last column is the packaged npm 6.4.0. Every run started a fresh MCP server over stdio and measured each request from the client. The first run of the session, not in the table, took 12.3 s for the cold overview and 3.9 s to initialize. What it suggests: the slow numbers on the other host look host-specific rather than Windows-specific. Connect plus the full catalog took 35.1 s there and about 1.3 s here. A per-phase breakdown on that host (JVM start, import, analysis, bridge readiness) would show where the time goes, and I can run the same script here for a side-by-side comparison. Not covered: snapshot save and replay, large targets, the guarded memory supervisor and peak memory. The `analyze_function` size also differs between hosts (48,868 bytes here, 42,650 in the 262 s run, 22,431 on Linux); I did not look into why.
bsakaguchi commented 2026-10-15 18:24:32 +00:00
Author
Owner
Copy link

@akushonkamen commented on GitHub (Oct 12, 2026):

Taking this on. I confirmed the root cause on current main: GhidraLauncher.ghidraHeadlessArguments always emits -import <target> (src/ghidra/GhidraLauncher.ts :539) together with -readOnly -deleteProject (:590-591) into a per-session fresh project root (join(runtimeRoot, "project")), so import/analysis cost can never be reused across sessions and a big target blows the startup deadline budget in GhidraClientStartup (deadline.remainingMs() → "Ghidra startup deadline elapsed").

Following @N0zoM1z0's first-slice spec: an existing-project branch that runs -process <program> with explicit project location/name (keeping -readOnly), copy-first into an owned temporary root so the user's original project files are never touched, fresh import stays the default. Tests in src/ghidra/GhidraLauncher*.test.ts covering: no -import/-deleteProject in the existing-project branch and copy-before-launch.

Disclosure: I'm an AI coding agent (Claude), working under my operator's account; the PR/patches are AI-generated.

<!-- gh-comment-id:6122866541 --> @akushonkamen commented on GitHub (Oct 12, 2026): Taking this on. I confirmed the root cause on current main: `GhidraLauncher.ghidraHeadlessArguments` always emits `-import <target>` (`src/ghidra/GhidraLauncher.ts` ~:539) together with `-readOnly -deleteProject` (~:590-591) into a per-session fresh project root (`join(runtimeRoot, "project")`), so import/analysis cost can never be reused across sessions and a big target blows the startup deadline budget in `GhidraClientStartup` (`deadline.remainingMs()` → "Ghidra startup deadline elapsed"). Following @N0zoM1z0's first-slice spec: an existing-project branch that runs `-process <program>` with explicit project location/name (keeping `-readOnly`), **copy-first** into an owned temporary root so the user's original project files are never touched, fresh import stays the default. Tests in `src/ghidra/GhidraLauncher*.test.ts` covering: no `-import`/`-deleteProject` in the existing-project branch and copy-before-launch. Disclosure: I'm an AI coding agent (Claude), working under my operator's account; the PR/patches are AI-generated.
bsakaguchi commented 2026-10-15 18:24:32 +00:00
Author
Owner
Copy link

@akushonkamen commented on GitHub (Oct 12, 2026):

Status update on the first slice: the launcher mechanism (+copy-first, fixed rea-project project-name contract with explicit pre-launch rejection, tests) is ready locally; wiring, digest binding, docs, and real-provider verification are tracked in #2147 — the slice is fixture/simulated-tested only, no real Ghidra lane has run yet. PR to follow with that disclosure in the description.

🤖 Generated with Claude Code (Opus 5.5) 🦊

<!-- gh-comment-id:6123305986 --> @akushonkamen commented on GitHub (Oct 12, 2026): Status update on the first slice: the launcher mechanism (+copy-first, fixed `rea-project` project-name contract with explicit pre-launch rejection, tests) is ready locally; wiring, digest binding, docs, and real-provider verification are tracked in #2147 — the slice is fixture/simulated-tested only, no real Ghidra lane has run yet. PR to follow with that disclosure in the description. 🤖 Generated with Claude Code (Opus 5.5) 🦊
bsakaguchi commented 2026-10-15 18:24:32 +00:00
Author
Owner
Copy link

@akushonkamen commented on GitHub (Oct 12, 2026):

TL;DR: PR is up: #2150 (branch fix/1048-existing-project-process).

First slice as scoped in my claim comment: existingProject branch in ghidraHeadlessArguments (-process <programName> -noanalysis + fixed rea-project name, -readOnly kept, -import/-deleteProject dropped), import-time selections rejected with reasons, and copy-first into the owned runtime project root so the original directory is never written. Profile/CLI selection surface is intentionally left for a follow-up slice.

Evidence on the rebased tip: baseline 22 passed | 1 skipped → red 14 failed | 22 passed | 1 skipped → green 36 passed | 1 skipped (tests/boundary/providers/ghidra/ghidraLauncher.test.ts); src/ghidra 364 passed; src/ghidra + tests/boundary/providers/ghidra 440 passed | 1 skipped; tsc --noEmit clean; oxlint 6 warnings | 0 errors (unchanged from baseline), oxfmt clean.

<!-- gh-comment-id:6123511460 --> @akushonkamen commented on GitHub (Oct 12, 2026): **TL;DR:** PR is up: #2150 (branch `fix/1048-existing-project-process`). First slice as scoped in my claim comment: `existingProject` branch in `ghidraHeadlessArguments` (`-process <programName> -noanalysis` + fixed `rea-project` name, `-readOnly` kept, `-import`/`-deleteProject` dropped), import-time selections rejected with reasons, and copy-first into the owned runtime project root so the original directory is never written. Profile/CLI selection surface is intentionally left for a follow-up slice. Evidence on the rebased tip: baseline 22 passed | 1 skipped → red 14 failed | 22 passed | 1 skipped → green 36 passed | 1 skipped (`tests/boundary/providers/ghidra/ghidraLauncher.test.ts`); `src/ghidra` 364 passed; `src/ghidra` + `tests/boundary/providers/ghidra` 440 passed | 1 skipped; `tsc --noEmit` clean; oxlint 6 warnings | 0 errors (unchanged from baseline), oxfmt clean.
bsakaguchi commented 2026-10-15 18:24:32 +00:00
Author
Owner
Copy link

@pablodiazjorge commented on GitHub (Oct 12, 2026):

@akushonkamen I ran the argument set from this PR against real Ghidra 12.1.4 (Windows 11, Temurin 21.0.12.1). I called analyzeHeadless directly, since the PR rejects the win32 launch path inside REA. The target is a small x64 PE built from source here, analyzed statically. One run per step:

Step Result
-import with analysis into rea-project, project kept 22.4 s. The directory holds rea-project.gpr and rea-project.rep/
Full copy, -process <program> -noanalysis -readOnly (this PR's arguments) Exit 0 in 4.9 s. A post-script sees 97 functions, so the earlier analysis is reused
Copy with only rea-project.rep/ Exit 1: NotFoundException: Project marker file not found: <copy>/rea-project.gpr

The argument shape works. The layout check, though, doesn't match the project Ghidra wrote: copyExistingGhidraProject accepts a directory that holds rea-project.rep or rea-project.grez. A directory with only .rep passes the check and then fails inside Ghidra with the marker error above, which is the opaque failure the check is meant to prevent.

One possible fix is to require both rea-project.gpr and rea-project.rep. The test fixtures would then create those two instead of .grez and .rep/marker. Your call, of course.

Two smaller notes:

  • The original project was unchanged after these runs (file hashes before and after). That reflects my manual copy, not the PR's copy code.
  • currentProgram.getExecutableSHA256() was available on the opened program without re-importing, which may help with the digest binding tracked in #2147.
<!-- gh-comment-id:6123910792 --> @pablodiazjorge commented on GitHub (Oct 12, 2026): @akushonkamen I ran the argument set from this PR against real Ghidra 12.1.4 (Windows 11, Temurin 21.0.12.1). I called `analyzeHeadless` directly, since the PR rejects the win32 launch path inside REA. The target is a small x64 PE built from source here, analyzed statically. One run per step: | Step | Result | |---|---| | `-import` with analysis into `rea-project`, project kept | 22.4 s. The directory holds `rea-project.gpr` and `rea-project.rep/` | | Full copy, `-process <program> -noanalysis -readOnly` (this PR's arguments) | Exit 0 in 4.9 s. A post-script sees 97 functions, so the earlier analysis is reused | | Copy with only `rea-project.rep/` | Exit 1: `NotFoundException: Project marker file not found: <copy>/rea-project.gpr` | The argument shape works. The layout check, though, doesn't match the project Ghidra wrote: `copyExistingGhidraProject` accepts a directory that holds `rea-project.rep` or `rea-project.grez`. A directory with only `.rep` passes the check and then fails inside Ghidra with the marker error above, which is the opaque failure the check is meant to prevent. One possible fix is to require both `rea-project.gpr` and `rea-project.rep`. The test fixtures would then create those two instead of `.grez` and `.rep/marker`. Your call, of course. Two smaller notes: - The original project was unchanged after these runs (file hashes before and after). That reflects my manual copy, not the PR's copy code. - `currentProgram.getExecutableSHA256()` was available on the opened program without re-importing, which may help with the digest binding tracked in #2147.
bsakaguchi commented 2026-10-15 18:24:33 +00:00
Author
Owner
Copy link

@CassieFox23 commented on GitHub (Oct 12, 2026):

TL;DR: On main today, REA keeps your renames only while the file stays open in one MCP session. Closing the file or starting a new server wipes them. Snapshots and evidence bundles do not bring them back.

This is a related data point for a reusable project. One user had an agent name every function, then lost the names. I tested what REA keeps on Linux with Ghidra, on two mains: 538f8cc72 (#2037), then the newer 83a4445f4 (#2034).

What you do What happens
Rename, then open the same file again without closing Names stay.
Rename, then close the file and open it again Names are gone. The old names return.
Rename, then start a new server Names are gone.
Rename, then save a snapshot Refused with evidence_integrity_mismatch. The session stays open and the names stay.
Save a snapshot with no edits, then open from it Works. It holds only the original names.
Export the evidence bundle The new names are inside, as a log of what REA saw.
Import that bundle into a clean session The file loads. No names come back.
Read the names out of the bundle, then send them to set_addresses_names Names return. This worked in the same server and in a fresh one.
Call comment or inline_comment capability_unavailable on Ghidra.

Comments set with annotate_native_function show in analyze_function. The entry comment also shows in procedure_pseudo_code. They vanish on close.

One change between the two mains

Commit 1c64fe3ef is the likely cause, since it changed when REA blocks snapshots. I did not bisect. A rename that fails now also blocks the snapshot save, where 538f8cc72 (#2037) saved one. The refusal text changed too. Exact messages are in the first fold below.

Why this matters for #1048

A reusable project could let names survive a close. Today the Ghidra project is temporary and deleted on close. So a snapshot cannot hold names.

Question for akushonkamen: does the project opened by #2150 keep edits, or open read-only? The update on #2150 says -readOnly stays. Ghidra's README says -readOnly drops changes, as I quoted earlier in this thread. I have not tested #2150. This is a question, not a finding.

Not run

  • Windows, macOS, and any target but a Linux ELF with Ghidra.
  • Hopper and IDA.
  • Two renames of one address, to see which name wins from a bundle.
  • Failures other than an unmapped address.
  • The REA test suite.

Thanks, akushonkamen, for tracing the cause and taking this on.

Exact errors and messages

Snapshot save after a rename, close_binary with snapshot_path, on 538f8cc72:

code: evidence_integrity_mismatch
category: integrity_mismatch
retryable: false
message: Analysis snapshots are unavailable after analysis metadata mutations. Export session observations through export_evidence_bundle if needed. Close the active target without saving a snapshot, then reopen it before importing or saving a snapshot.

On 83a4445f4, same code, category and retryable. Only the start of the message changed:

message: Analysis snapshots are unavailable because analysis metadata may have changed. Export session observations through export_evidence_bundle if needed. Close the active target without saving a snapshot, then reopen it before importing or saving a snapshot.

In both, the session stayed open, the names stayed, and no snapshot file was written.

Opening from a snapshot while an edited session is open gives the same code. The wording changed the same way as the save refusal. The old text said "after analysis metadata mutations". The new text says "because analysis metadata may have changed".

The failed rename was set_address_name on address 0x1. Both mains gave invalid_request, "Analysis input is invalid.", with the detail "Address is not mapped: 0x1". The name at 0x4004a0 stayed add_numbers. Then I ran close_binary with snapshot_path. The older main saved a file of 4,406 bytes. The newer main refused with evidence_integrity_mismatch and wrote no file.

comment and inline_comment on Ghidra:

code: capability_unavailable
category: unsupported_provider
message: This analysis is unavailable for the current target. Choose another analysis or target.
details: provider_id ghidra, reason "operation is not declared by this provider"
Round trip by hand
  1. Open the target. Rename with set_address_name, set_addresses_names and annotate_native_function. Read back with list_procedures, address_name and list_names.
  2. Run export_evidence_bundle. It wrote one JSON file: 112,958 bytes, 11 records. The new names sit in the edit records. The old names sit in the earlier list_procedures record.
  3. Run close_binary with no snapshot_path. Open the target again. The old names are back.
  4. Run import_evidence_bundle. It reports imported 10. The old names stay.
  5. Write the names from the bundle into one map, like {"0x4004a0": "q1_sum_two_ints"}. Send it to set_addresses_names. Every address returns true, and the new names show again.
  6. Send the comments again with annotate_native_function.
  7. Repeat steps 3 to 6 in a fresh server. Same result.

Bundle records carry no timestamp or order field. A double rename of one address would not say which name came last. I did not test that.

Every edit record in the bundle carries a limits note. It ends with: "this is not a saved Ghidra project."

Commands 
# build, Node 24.18.0, in a memory-capped scope
systemd-run --user --scope -p MemoryMax=8G sh -c 'npm ci && npm run build && npm run test:prepare'
gcc -O1 -g0 -o target target.c
# one stdio MCP client per phase, against:
REA_ANALYSIS_PROVIDER=ghidra node scripts/rea.mjs mcp
# calls: open_binary, list_procedures, set_address_name, set_addresses_names,
# annotate_native_function, address_name, list_names, close_binary,
# export_evidence_bundle, import_evidence_bundle, comment, inline_comment

Each server was stopped after its phase. No REA or Ghidra process was left running.

Test setup
  • Box: Nobara 44 (Fedora 44 base), Linux 7.2.9
  • CPU: AMD Ryzen 9 5900X
  • RAM: 32 GB
  • Node: 24.18.0
  • Main: 538f8cc72 (#2037, older run) and 83a4445f4 (#2034, newer run)
  • Other: Ghidra 12.1.4 (Apache-2.0), JDK 21
  • Target: a small C program built with gcc -O1, x86-64 ELF, our own code

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

<!-- gh-comment-id:6127456580 --> @CassieFox23 commented on GitHub (Oct 12, 2026): **TL;DR:** On main today, REA keeps your renames only while the file stays open in one MCP session. Closing the file or starting a new server wipes them. Snapshots and evidence bundles do not bring them back. This is a related data point for a reusable project. One user had an agent name every function, then lost the names. I tested what REA keeps on Linux with Ghidra, on two mains: 538f8cc72 (#2037), then the newer 83a4445f4 (#2034). | What you do | What happens | |---|---| | Rename, then open the same file again without closing | Names stay. | | Rename, then close the file and open it again | Names are gone. The old names return. | | Rename, then start a new server | Names are gone. | | Rename, then save a snapshot | Refused with `evidence_integrity_mismatch`. The session stays open and the names stay. | | Save a snapshot with no edits, then open from it | Works. It holds only the original names. | | Export the evidence bundle | The new names are inside, as a log of what REA saw. | | Import that bundle into a clean session | The file loads. No names come back. | | Read the names out of the bundle, then send them to `set_addresses_names` | Names return. This worked in the same server and in a fresh one. | | Call `comment` or `inline_comment` | `capability_unavailable` on Ghidra. | Comments set with `annotate_native_function` show in `analyze_function`. The entry comment also shows in `procedure_pseudo_code`. They vanish on close. ### One change between the two mains Commit 1c64fe3ef is the likely cause, since it changed when REA blocks snapshots. I did not bisect. A rename that fails now also blocks the snapshot save, where 538f8cc72 (#2037) saved one. The refusal text changed too. Exact messages are in the first fold below. ### Why this matters for #1048 A reusable project could let names survive a close. Today the Ghidra project is temporary and deleted on close. So a snapshot cannot hold names. Question for akushonkamen: does the project opened by #2150 keep edits, or open read-only? The update on #2150 says `-readOnly` stays. Ghidra's README says `-readOnly` drops changes, as I quoted earlier in this thread. I have not tested #2150. This is a question, not a finding. ### Not run - Windows, macOS, and any target but a Linux ELF with Ghidra. - Hopper and IDA. - Two renames of one address, to see which name wins from a bundle. - Failures other than an unmapped address. - The REA test suite. Thanks, akushonkamen, for tracing the cause and taking this on. <details> <summary>Exact errors and messages</summary> Snapshot save after a rename, `close_binary` with `snapshot_path`, on 538f8cc72: ``` code: evidence_integrity_mismatch category: integrity_mismatch retryable: false message: Analysis snapshots are unavailable after analysis metadata mutations. Export session observations through export_evidence_bundle if needed. Close the active target without saving a snapshot, then reopen it before importing or saving a snapshot. ``` On 83a4445f4, same code, category and retryable. Only the start of the message changed: ``` message: Analysis snapshots are unavailable because analysis metadata may have changed. Export session observations through export_evidence_bundle if needed. Close the active target without saving a snapshot, then reopen it before importing or saving a snapshot. ``` In both, the session stayed open, the names stayed, and no snapshot file was written. Opening from a snapshot while an edited session is open gives the same code. The wording changed the same way as the save refusal. The old text said "after analysis metadata mutations". The new text says "because analysis metadata may have changed". The failed rename was `set_address_name` on address 0x1. Both mains gave `invalid_request`, "Analysis input is invalid.", with the detail "Address is not mapped: 0x1". The name at 0x4004a0 stayed add_numbers. Then I ran `close_binary` with `snapshot_path`. The older main saved a file of 4,406 bytes. The newer main refused with `evidence_integrity_mismatch` and wrote no file. `comment` and `inline_comment` on Ghidra: ``` code: capability_unavailable category: unsupported_provider message: This analysis is unavailable for the current target. Choose another analysis or target. details: provider_id ghidra, reason "operation is not declared by this provider" ``` </details> <details> <summary>Round trip by hand</summary> 1. Open the target. Rename with `set_address_name`, `set_addresses_names` and `annotate_native_function`. Read back with `list_procedures`, `address_name` and `list_names`. 2. Run `export_evidence_bundle`. It wrote one JSON file: 112,958 bytes, 11 records. The new names sit in the edit records. The old names sit in the earlier `list_procedures` record. 3. Run `close_binary` with no `snapshot_path`. Open the target again. The old names are back. 4. Run `import_evidence_bundle`. It reports `imported 10`. The old names stay. 5. Write the names from the bundle into one map, like `{"0x4004a0": "q1_sum_two_ints"}`. Send it to `set_addresses_names`. Every address returns `true`, and the new names show again. 6. Send the comments again with `annotate_native_function`. 7. Repeat steps 3 to 6 in a fresh server. Same result. Bundle records carry no timestamp or order field. A double rename of one address would not say which name came last. I did not test that. Every edit record in the bundle carries a limits note. It ends with: "this is not a saved Ghidra project." </details> <details> <summary>Commands</summary> ``` # build, Node 24.18.0, in a memory-capped scope systemd-run --user --scope -p MemoryMax=8G sh -c 'npm ci && npm run build && npm run test:prepare' gcc -O1 -g0 -o target target.c # one stdio MCP client per phase, against: REA_ANALYSIS_PROVIDER=ghidra node scripts/rea.mjs mcp # calls: open_binary, list_procedures, set_address_name, set_addresses_names, # annotate_native_function, address_name, list_names, close_binary, # export_evidence_bundle, import_evidence_bundle, comment, inline_comment ``` Each server was stopped after its phase. No REA or Ghidra process was left running. </details> <details> <summary>Test setup</summary> - Box: Nobara 44 (Fedora 44 base), Linux 7.2.9 - CPU: AMD Ryzen 9 5900X - RAM: 32 GB - Node: 24.18.0 - Main: 538f8cc72 (#2037, older run) and 83a4445f4 (#2034, newer run) - Other: Ghidra 12.1.4 (Apache-2.0), JDK 21 - Target: a small C program built with gcc -O1, x86-64 ELF, our own code </details> 🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊
bsakaguchi commented 2026-10-15 18:24:33 +00:00
Author
Owner
Copy link

@matbrgz commented on GitHub (Oct 13, 2026):

Data point from a terminal-only workflow: on a 7.1 MB Delphi 7 PE32 (AUTORIZADOR.exe, sha256 1398d55a…8fa363, public DATASUS installer AUTORIZADOR_VER6630.exe), every rea decompile|function|analyze --provider ghidra CLI call re-imports and re-analyzes for ~70 s (Ghidra 12.1.4, macOS arm64, 1,610 seeds). --snapshot only helps for exact repeats. Our workaround is a small stdio MCP client that opens the binary once and runs a JSON list of tool calls (the queries after the import take milliseconds). A pre-imported project (this issue) or a CLI batch mode (#2927) would remove the need for it.

<!-- gh-comment-id:6151085620 --> @matbrgz commented on GitHub (Oct 13, 2026): Data point from a terminal-only workflow: on a 7.1 MB Delphi 7 PE32 (`AUTORIZADOR.exe`, sha256 `1398d55a…8fa363`, public DATASUS installer `AUTORIZADOR_VER6630.exe`), every `rea decompile|function|analyze --provider ghidra` CLI call re-imports and re-analyzes for ~70 s (Ghidra 12.1.4, macOS arm64, 1,610 seeds). `--snapshot` only helps for exact repeats. Our workaround is a small stdio MCP client that opens the binary once and runs a JSON list of tool calls (the queries after the import take milliseconds). A pre-imported project (this issue) or a CLI batch mode (#2927) would remove the need for it.
bsakaguchi 2026-10-31 00:29:06 +00:00
bsakaguchi commented 2026-10-31 00:29:07 +00:00
Author
Owner
Copy link

@alexlanghart commented on GitHub (Oct 19, 2026):

A working implementation of the "documented pre-import step" direction, for a scope decision. It's on my fork, not opened as a PR: alexlanghart/rea@feat/ghidra-gzf-reuse (61aba7587, 34308b014, cf3f55297).

Design. REA_GHIDRA_PROGRAM_ARCHIVE_DIR names a caller-owned directory. Profile resolution names an archive <target-sha256>-<key>.gzf, where key is the analysis profile digest (seed content, not seed paths, so moving the same seeds keeps the archive), and commits program_archive {mode, path, key} into the profile.

  • create (no matching file): normal import + seeds + auto-analysis + no-return repair, then a post-script packs the program (DomainObjectAdapterDB.saveToPackedFile) into an owner-only temp dir beside the destination, chmod 600, and hard-links it into place. It never replaces an existing file, and a failed save is reported as a limitation, not a session failure.
  • load (file exists): -import <archive> -noanalysis -readOnly -deleteProject, bridge post-script only, still in the usual ephemeral project. The bridge's existing getExecutableSHA256() check binds the archive to the target, and it now also reports the key stored in PROGRAM_INFO; the session fails closed on a mismatch. Session annotations are never written back.
  • Not admitted on Windows P0.

Measurements (Ghidra 12.1.4, JDK 21, WSL2, 9G heap): 60 MB x86-64 PE with 130k seeded functions. Create: 26 min 53 s, 7.5 GB peak RSS, 306 MB archive. Load: 21.6 s, 1.9 GB, with byte-identical decompile output. A second create/load pair with the final config: 24 min / 6.45 GB vs. 21.2 s / 1.8 GB. On a small ELF (jq), create 19 s vs. load 15 s (JVM startup dominates).

Known gaps before this could be integration-ready:

  1. Env-var only; no CLI flag or open_binary parameter yet.
  2. A loaded session still reports import_mode: "ephemeral-source-immutable" alongside program_archive.mode: "load"; it probably needs its own import mode so Evidence says the analysis came from an earlier session.
  3. Archive contents are trusted beyond the executable digest and stored key (no archive hash in the profile); fine for a caller-owned directory, but it should be explicit.
  4. Unit tests plus real-provider runs only: no launcher boundary tests, lifecycle regressions (cancel during save, orphaned temp dir after a JVM kill, concurrent create) or verify: lane yet. Raw/DOS/PSP/MIPS soft-float/NativeAOT targets are untested on the load path.
  5. No eviction or size accounting for the directory.

Per-target seed selection (REA_GHIDRA_SEED_DIR, 61cb3b222) is on the same branch but independent; I'd split it out.

Is this direction (REA writes and reloads its own .gzf, keyed by target + profile) in scope, or would you prefer the read-only "open a caller-supplied project" route from the original proposal? If it's in scope, I can address the gaps above and open a draft PR linked here.

<!-- gh-comment-id:6271226623 --> @alexlanghart commented on GitHub (Oct 19, 2026): A working implementation of the "documented pre-import step" direction, for a scope decision. It's on my fork, not opened as a PR: [`alexlanghart/rea@feat/ghidra-gzf-reuse`](https://github.com/alexlanghart/rea/tree/feat/ghidra-gzf-reuse) (61aba7587, 34308b014, cf3f55297). **Design.** `REA_GHIDRA_PROGRAM_ARCHIVE_DIR` names a caller-owned directory. Profile resolution names an archive `<target-sha256>-<key>.gzf`, where `key` is the analysis profile digest (seed *content*, not seed paths, so moving the same seeds keeps the archive), and commits `program_archive {mode, path, key}` into the profile. - **create** (no matching file): normal import + seeds + auto-analysis + no-return repair, then a post-script packs the program (`DomainObjectAdapterDB.saveToPackedFile`) into an owner-only temp dir beside the destination, `chmod 600`, and hard-links it into place. It never replaces an existing file, and a failed save is reported as a limitation, not a session failure. - **load** (file exists): `-import <archive> -noanalysis -readOnly -deleteProject`, bridge post-script only, still in the usual ephemeral project. The bridge's existing `getExecutableSHA256()` check binds the archive to the target, and it now also reports the key stored in `PROGRAM_INFO`; the session fails closed on a mismatch. Session annotations are never written back. - Not admitted on Windows P0. **Measurements** (Ghidra 12.1.4, JDK 21, WSL2, 9G heap): 60 MB x86-64 PE with 130k seeded functions. Create: 26 min 53 s, 7.5 GB peak RSS, 306 MB archive. Load: **21.6 s, 1.9 GB**, with byte-identical `decompile` output. A second create/load pair with the final config: 24 min / 6.45 GB vs. 21.2 s / 1.8 GB. On a small ELF (jq), create 19 s vs. load 15 s (JVM startup dominates). **Known gaps** before this could be integration-ready: 1. Env-var only; no CLI flag or `open_binary` parameter yet. 2. A loaded session still reports `import_mode: "ephemeral-source-immutable"` alongside `program_archive.mode: "load"`; it probably needs its own import mode so Evidence says the analysis came from an earlier session. 3. Archive contents are trusted beyond the executable digest and stored key (no archive hash in the profile); fine for a caller-owned directory, but it should be explicit. 4. Unit tests plus real-provider runs only: no launcher boundary tests, lifecycle regressions (cancel during save, orphaned temp dir after a JVM kill, concurrent create) or `verify:` lane yet. Raw/DOS/PSP/MIPS soft-float/NativeAOT targets are untested on the load path. 5. No eviction or size accounting for the directory. Per-target seed selection (`REA_GHIDRA_SEED_DIR`, 61cb3b222) is on the same branch but independent; I'd split it out. Is this direction (REA writes and reloads its own `.gzf`, keyed by target + profile) in scope, or would you prefer the read-only "open a caller-supplied project" route from the original proposal? If it's in scope, I can address the gaps above and open a draft PR linked here.
bsakaguchi commented 2026-10-31 00:29:07 +00:00
Author
Owner
Copy link

@kaoru0822-kitauji commented on GitHub (Oct 26, 2026):

Use case from a Discord user: they want REA to read the provider database they already worked in, so their manual work (renames, types, comments) shows up in REA's results. This is a different goal from this issue's original one, which is moving import cost outside the startup deadline. An opened existing project would serve both.

Here is what each provider does today (checked against main b3818bbba docs and source; nothing was run against real providers):

Provider Opens the user's existing database? Source
Ghidra No. Every session runs a fresh -import … -readOnly -deleteProject. The existing-project -process launcher slice from this issue isn't wired to the product yet. #2147 tracks the wiring, digest binding, docs and real-provider verification. src/ghidra/GhidraLauncher.ts (import args), #2147
IDA Yes, through attached mode. Open the .i64/.idb in the IDA GUI with ida-pro-mcp (or the native plugin), then point REA at the original input binary. REA checks the recorded input SHA-256 and returns live results, so GUI renames and types are visible. Headless mode never adopts a user database: it copies the input and releases with save: false. docs/ida-provider.md ("Open the original input binary…", headless section)
Hopper No on macOS: saved .hop targets are refused, because the native open database route doesn't run REA's bridge. The original executable can be opened, but database-only annotations aren't recovered. Linux keeps its demo-launch route for saved databases. docs/installation.md (saved Hopper database paragraph)

What each provider would need:

  • Ghidra: finish #2147: product selection of a project and program, binding the program to the target digest, and a read-only copy-first open, so the user's annotations are read and never modified. That makes this use case reachable. The .gzf archive design in the comment above would serve it too, if the archive came from the user's own project.
  • IDA: nothing new for reads. Documenting attached mode as the "use my existing database" route would answer this request.
  • Hopper (macOS): an open route that runs the bridge for a .hop, or one that attaches to an already-open Hopper document. #479 covers reuse of existing GUI windows.

Suggested first step: item 1 of #2147, the product wiring. The launcher mechanism already exists. Meanwhile, IDA users can use attached mode today.

<!-- gh-comment-id:6410249964 --> @kaoru0822-kitauji commented on GitHub (Oct 26, 2026): **Use case from a Discord user:** they want REA to read the provider database they already worked in, so their manual work (renames, types, comments) shows up in REA's results. This is a different goal from this issue's original one, which is moving import cost outside the startup deadline. An opened existing project would serve both. Here is what each provider does today (checked against main b3818bbba docs and source; nothing was run against real providers): | Provider | Opens the user's existing database? | Source | |---|---|---| | Ghidra | No. Every session runs a fresh `-import … -readOnly -deleteProject`. The existing-project `-process` launcher slice from this issue isn't wired to the product yet. #2147 tracks the wiring, digest binding, docs and real-provider verification. | `src/ghidra/GhidraLauncher.ts` (import args), #2147 | | IDA | **Yes, through attached mode.** Open the `.i64`/`.idb` in the IDA GUI with ida-pro-mcp (or the native plugin), then point REA at the **original input binary**. REA checks the recorded input SHA-256 and returns live results, so GUI renames and types are visible. Headless mode never adopts a user database: it copies the input and releases with `save: false`. | `docs/ida-provider.md` ("Open the original input binary…", headless section) | | Hopper | No on macOS: saved `.hop` targets are refused, because the native `open database` route doesn't run REA's bridge. The original executable can be opened, but database-only annotations aren't recovered. Linux keeps its demo-launch route for saved databases. | `docs/installation.md` (saved Hopper database paragraph) | What each provider would need: - **Ghidra:** finish #2147: product selection of a project and program, binding the program to the target digest, and a read-only copy-first open, so the user's annotations are read and never modified. That makes this use case reachable. The `.gzf` archive design in the comment above would serve it too, if the archive came from the user's own project. - **IDA:** nothing new for reads. Documenting attached mode as the "use my existing database" route would answer this request. - **Hopper (macOS):** an open route that runs the bridge for a `.hop`, or one that attaches to an already-open Hopper document. #479 covers reuse of existing GUI windows. Suggested first step: item 1 of #2147, the product wiring. The launcher mechanism already exists. Meanwhile, IDA users can use attached mode today.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels   
area: bridge

   
area: bridge

   
area: build

   
area: build

   
area: build

   
area: cli

   
area: domain

   
area: evidence

   
area: mcp

   
area: native

   
area: providers

   
area: runtime

   
area: server

   
area: setup

   
area: tests

   
bug

   
design

   
documentation

   
enhancement

   
enhancement

   
enhancement

   
experiment

   
help wanted

   
i18n

   
invalid

   
P0: critical

   
P1: high

   
P2: medium

   
P2: medium

   
P2: medium

   
P3: low

   
performance

   
provider-request

   
pull-request
Mirrored from GitHub Pull Request

  
question

   
roadmap

   
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#383
No description provided.