[GH-ISSUE #8344] [Design] Discover candidates before selecting an existing Ghidra program #6550

Open
opened 2026-11-01 21:12:09 +00:00 by bsakaguchi · 13 comments
Owner

Originally created by @morluto on GitHub (Oct 31, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8344

Before submitting

  • I searched the open and closed issues and pull requests for a related discussion.
  • I understand the contribution scope must be reviewed before integration; open questions are welcome here.
  • This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Should REA offer a bounded way to discover programs in a caller-selected Ghidra project before requiring the analyst to select one?

Context and evidence

On REA main 60149282, macOS arm64, and Ghidra 12.1.4, a source-built binary had two root programs in one caller-owned project, program_alpha and program_beta. They recorded the same imported executable SHA-256, but had distinct analyst-authored function names and comments.

With REA_GHIDRA_PROJECT_PATH set and REA_GHIDRA_PROGRAM_NAME unset, open_binary bound the target lazily; the first list_documents failed with the current actionable configuration error because the binary basename was not a project program. Setting either exact program name in a new REA server worked, and list_documents then returned only the selected name. Decompilation showed the corresponding distinct authored name and comment. This is consistent with the advertised one-program session contract; the issue is candidate discovery before selection, not a claim that list_documents is incorrect.

As a separate direct Ghidra control, I copied the project to a disposable location and ran headless -process '*' -noanalysis -readOnly with a script reporting each program's path, recorded executable digest and language. It reported both names and the same digest. The original project and binary content digests were unchanged; the temporary project copy's bookkeeping files did change. The target was not executed. Ghidra's headless documentation documents wildcard processing; this experiment is not evidence that processing every program is cheap or a suitable REA implementation.

Nested selection integration control

On macOS arm64, an uncommitted temporary merge of #3817 f8abf28c with main 46c13b998 resolved two conflicts locally; this was not the contributor’s unmodified PR head. With Ghidra 12.1.4, an owned project containing workflow under /group, and explicit folder and program settings, real REA stdio MCP in compact and full profiles opened the target, listed three procedures and returned an analyze_function summary for evaluate. The analysis profile committed the exact /group folder and selected program. The target SHA-256 ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d and the eight regular files in the project .rep tree were unchanged; both sessions closed. The two focused project/launcher suites passed 80 tests with 3 skipped. This demonstrates explicit nested selection in that local merge, not discovery of candidates, verification of every project file or a clean #3817 run.

Same digest in two folders

A separate owned project imported the same unexecuted Mach-O as workflow under both /group and /other. Headless Ghidra scripts assigned different function names (audit_alpha and audit_beta) to the same function in those saved programs. With the same temporary conflict-resolved #3817/main merge described above, also containing the unrelated missing-program guidance prototype from #8579, real REA/Ghidra 12.1.4 compact and full MCP sessions selected each folder explicitly and returned its corresponding name. The selected project path and program name were otherwise identical; the analysis profile digests differed by folder. The target and all regular files in the owned .gpr/.rep project tree retained their hashes across REA use, and each session closed.

This is one controlled project, not proof of candidate-discovery behavior or automatic selection. It demonstrates why equal imported executable digests cannot choose between analysis programs: their saved names and other analysis may differ. Discovery should preserve each folder/name identity and leave the choice to the caller.

Cross-folder snapshot control

Using the same two-folder project and temporary integration checkout, real compact and full MCP controls saved a snapshot after listing the /group program. A fresh server configured for /other and the same target rejected that snapshot with evidence_integrity_mismatch and a structured binding_mismatch.profile_digest; the rejected open left no target selected. Opening /other without the snapshot then returned audit_beta. Target and .gpr/.rep file hashes were unchanged. This verifies one cross-folder replay boundary; it does not establish a general cache or project-discovery policy.

A subsequent control started with /other already open and its audit_beta analysis available. In both MCP profiles, trying to reopen the same target with the /group snapshot returned the same structured mismatch; the active analysis run identity and listed procedures remained unchanged, and close_binary succeeded. Target and project file hashes were unchanged. This tests failed-open session preservation for this fixture and temporary integration only.

Recursive Ghidra discovery control

On the owned two-folder project above, direct Ghidra 12.1.4 headless on a disposable project copy with -process "*" -recursive -noanalysis -readOnly and a probe script reported /group/workflow and /other/workflow. Both recorded the same imported SHA-256, while the saved function names were audit_alpha and audit_beta. The original target and all 12 regular original project files matched their earlier hashes after the run. This was direct Ghidra, not a REA discovery operation, and the project copy may change internally.

The two-row control establishes that recursive processing can enumerate this fixture and retain path identity; it does not establish boundedness, partial-failure coverage, cancellation behavior, cost, or safe behavior for arbitrary project contents. A discovery design should report those limits and avoid silently choosing a same-digest result.

Recursive processing partial-failure control

On a fresh disposable copy of the same two-folder project, a direct Ghidra 12.1.4 headless probe intentionally threw an exception while processing /group/workflow and emitted a row for /other/workflow. Ghidra logged REPORT SCRIPT ERROR, continued to the second program, and exited with code 0. The original project and target hashes remained unchanged. This is an injected script failure, not a naturally corrupted project and not a REA discovery run.

A future discovery workflow cannot treat exit code zero or one emitted row as proof that every candidate was inspected. It needs per-item completion/failure evidence and truthful aggregate coverage, as well as cancellation and output bounds. Do not silently label the failed candidate absent or mismatch.

Expected interaction with targets

Read existing caller-selected project and binary files; launch the local Ghidra provider and run an REA-owned inspection script against an owned copy if that approach is chosen. Do not execute the target binary or modify the caller's project. Exact effects and authority need design review.

Boundary details (optional)

Keep user choice explicit: equal imported digests do not establish identical analysis or select a program. Preserve exact project path, project fingerprint, program path/name, known/unknown digest and completeness; never turn an uninspected or failed item into a mismatch. Do not silently fall back to a fresh import or mutate the source project. Current REA selection supports root program names; nested-project paths are a separate scope decision.

Options and tradeoffs

Consider a project-specific discovery operation or a focused setup/doctor handoff before opening a target, versus documenting a safe external Ghidra workflow. Compare bounded metadata discovery with per-program opening, including copy cost, cancellation, unrecognized/non-program entries, identity verification, concurrent edits, pagination and cleanup. Avoid exposing a generic arbitrary-script tool or automatically selecting the first digest match.

Verification available so far

I have a prototype or observed results

Success criteria and open questions

Exercise MCP and CLI with zero, one and multiple candidates; same-digest programs with different annotations; missing/unknown identities; nested and non-program items if supported; cancellation, partial inventory, copy failure and source-project preservation. Make inventory scope and failure/coverage explicit. The direct Ghidra controls include two root programs and, separately, two nested programs on one POSIX host. Nested selection was exercised only in a temporary REA integration; REA candidate discovery was not implemented or tested. Windows and performance remain unverified.

References

Existing-project request: #3751. Missing-program recovery already fixed: #8216 / #8222. Nested program selection PR: #3817. Digest mismatch handling: #8169. The checked list_documents contract is in src/contracts/officialToolContracts.ts and intentionally describes one Ghidra Program.

@gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Host readiness versus selected-program validity

On #8820 head a4b7dfcd1, macOS arm64 and Ghidra 12.1.4, I selected an owned expanded project containing a root program named program, but set REA_GHIDRA_PROGRAM_NAME=not_a_program. Scoped rea doctor --provider ghidra --target <target> reported healthy; a subsequent rea decompile rejected the selection with configuration_invalid, named REA_GHIDRA_PROGRAM_NAME, and advised selecting an exact root program without modifying the project or falling back to a fresh import. Selecting program instead returned the expected function. The target, original packed input and every regular file in the expanded project kept their hashes during REA use; none was executed.

This is consistent with the documented doctor contract: its provider checks are host-level and do not attest that a particular project program exists or matches the target. It is not evidence that the installation check is wrong. It does show that project discovery or readiness UX must keep host readiness, selected project shape, exact program selection, target identity and successful analysis separate; finding a project file or receiving a healthy doctor result does not establish the later facts. No project-discovery operation was implemented in this control.

Single renamed program with spaces

On #8820 head f3c0336c5, I used Ghidra 12.1.4 headless to import a copy of an owned Mach-O named renamed sample into an expanded project whose directory and project name contained spaces. The admitted REA target was a separate same-digest file named arm64. In compact/full MCP, leaving REA_GHIDRA_PROGRAM_NAME unset admitted the target lazily, then the first pseudocode query returned configuration_invalid, explaining that the default name arm64 was not a root program and asking for the exact name. Restarting with REA_GHIDRA_PROGRAM_NAME=renamed sample successfully decompiled the expected function in both profiles. The target SHA-256 0e0ecf82c96e4918ee2ca1287dc25513d8fd13d5ab7fa5b98db00ad342f03930 and regular original project files remained unchanged. Ghidra's import log supplied the name in this control; REA itself did not enumerate it. This verifies exact-name selection and preservation for one program and host, not candidate discovery or automatic same-digest selection.

Originally created by @morluto on GitHub (Oct 31, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8344 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Should REA offer a bounded way to discover programs in a caller-selected Ghidra project before requiring the analyst to select one? ### Context and evidence On REA main `60149282`, macOS arm64, and Ghidra 12.1.4, a source-built binary had two root programs in one caller-owned project, `program_alpha` and `program_beta`. They recorded the same imported executable SHA-256, but had distinct analyst-authored function names and comments. With `REA_GHIDRA_PROJECT_PATH` set and `REA_GHIDRA_PROGRAM_NAME` unset, `open_binary` bound the target lazily; the first `list_documents` failed with the current actionable configuration error because the binary basename was not a project program. Setting either exact program name in a new REA server worked, and `list_documents` then returned only the selected name. Decompilation showed the corresponding distinct authored name and comment. This is consistent with the advertised one-program session contract; the issue is candidate discovery before selection, not a claim that `list_documents` is incorrect. As a separate direct Ghidra control, I copied the project to a disposable location and ran headless `-process '*' -noanalysis -readOnly` with a script reporting each program's path, recorded executable digest and language. It reported both names and the same digest. The original project and binary content digests were unchanged; the temporary project copy's bookkeeping files did change. The target was not executed. Ghidra's [headless documentation](https://github.com/NationalSecurityAgency/ghidra/blob/master/Ghidra/RuntimeScripts/support/analyzeHeadlessREADME.md#-process-project_file) documents wildcard processing; this experiment is not evidence that processing every program is cheap or a suitable REA implementation. ### Nested selection integration control On macOS arm64, an uncommitted temporary merge of #3817 `f8abf28c` with main `46c13b998` resolved two conflicts locally; this was not the contributor’s unmodified PR head. With Ghidra 12.1.4, an owned project containing `workflow` under `/group`, and explicit folder and program settings, real REA stdio MCP in compact and full profiles opened the target, listed three procedures and returned an `analyze_function` summary for `evaluate`. The analysis profile committed the exact `/group` folder and selected program. The target SHA-256 `ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d` and the eight regular files in the project `.rep` tree were unchanged; both sessions closed. The two focused project/launcher suites passed 80 tests with 3 skipped. This demonstrates explicit nested selection in that local merge, not discovery of candidates, verification of every project file or a clean #3817 run. ### Same digest in two folders A separate owned project imported the same unexecuted Mach-O as `workflow` under both `/group` and `/other`. Headless Ghidra scripts assigned different function names (`audit_alpha` and `audit_beta`) to the same function in those saved programs. With the same temporary conflict-resolved #3817/main merge described above, also containing the unrelated missing-program guidance prototype from #8579, real REA/Ghidra 12.1.4 compact and full MCP sessions selected each folder explicitly and returned its corresponding name. The selected project path and program name were otherwise identical; the analysis profile digests differed by folder. The target and all regular files in the owned `.gpr`/`.rep` project tree retained their hashes across REA use, and each session closed. This is one controlled project, not proof of candidate-discovery behavior or automatic selection. It demonstrates why equal imported executable digests cannot choose between analysis programs: their saved names and other analysis may differ. Discovery should preserve each folder/name identity and leave the choice to the caller. ### Cross-folder snapshot control Using the same two-folder project and temporary integration checkout, real compact and full MCP controls saved a snapshot after listing the `/group` program. A fresh server configured for `/other` and the same target rejected that snapshot with `evidence_integrity_mismatch` and a structured `binding_mismatch.profile_digest`; the rejected open left no target selected. Opening `/other` without the snapshot then returned `audit_beta`. Target and `.gpr`/`.rep` file hashes were unchanged. This verifies one cross-folder replay boundary; it does not establish a general cache or project-discovery policy. A subsequent control started with `/other` already open and its `audit_beta` analysis available. In both MCP profiles, trying to reopen the same target with the `/group` snapshot returned the same structured mismatch; the active analysis run identity and listed procedures remained unchanged, and `close_binary` succeeded. Target and project file hashes were unchanged. This tests failed-open session preservation for this fixture and temporary integration only. ### Recursive Ghidra discovery control On the owned two-folder project above, direct Ghidra 12.1.4 headless on a disposable project copy with `-process "*" -recursive -noanalysis -readOnly` and a probe script reported `/group/workflow` and `/other/workflow`. Both recorded the same imported SHA-256, while the saved function names were `audit_alpha` and `audit_beta`. The original target and all 12 regular original project files matched their earlier hashes after the run. This was direct Ghidra, not a REA discovery operation, and the project copy may change internally. The two-row control establishes that recursive processing can enumerate this fixture and retain path identity; it does not establish boundedness, partial-failure coverage, cancellation behavior, cost, or safe behavior for arbitrary project contents. A discovery design should report those limits and avoid silently choosing a same-digest result. ### Recursive processing partial-failure control On a fresh disposable copy of the same two-folder project, a direct Ghidra 12.1.4 headless probe intentionally threw an exception while processing `/group/workflow` and emitted a row for `/other/workflow`. Ghidra logged `REPORT SCRIPT ERROR`, continued to the second program, and exited with code 0. The original project and target hashes remained unchanged. This is an injected script failure, not a naturally corrupted project and not a REA discovery run. A future discovery workflow cannot treat exit code zero or one emitted row as proof that every candidate was inspected. It needs per-item completion/failure evidence and truthful aggregate coverage, as well as cancellation and output bounds. Do not silently label the failed candidate absent or mismatch. ### Expected interaction with targets Read existing caller-selected project and binary files; launch the local Ghidra provider and run an REA-owned inspection script against an owned copy if that approach is chosen. Do not execute the target binary or modify the caller's project. Exact effects and authority need design review. ### Boundary details (optional) Keep user choice explicit: equal imported digests do not establish identical analysis or select a program. Preserve exact project path, project fingerprint, program path/name, known/unknown digest and completeness; never turn an uninspected or failed item into a mismatch. Do not silently fall back to a fresh import or mutate the source project. Current REA selection supports root program names; nested-project paths are a separate scope decision. ### Options and tradeoffs Consider a project-specific discovery operation or a focused setup/doctor handoff before opening a target, versus documenting a safe external Ghidra workflow. Compare bounded metadata discovery with per-program opening, including copy cost, cancellation, unrecognized/non-program entries, identity verification, concurrent edits, pagination and cleanup. Avoid exposing a generic arbitrary-script tool or automatically selecting the first digest match. ### Verification available so far I have a prototype or observed results ### Success criteria and open questions Exercise MCP and CLI with zero, one and multiple candidates; same-digest programs with different annotations; missing/unknown identities; nested and non-program items if supported; cancellation, partial inventory, copy failure and source-project preservation. Make inventory scope and failure/coverage explicit. The direct Ghidra controls include two root programs and, separately, two nested programs on one POSIX host. Nested selection was exercised only in a temporary REA integration; REA candidate discovery was not implemented or tested. Windows and performance remain unverified. ### References Existing-project request: #3751. Missing-program recovery already fixed: #8216 / #8222. Nested program selection PR: #3817. Digest mismatch handling: #8169. The checked `list_documents` contract is in `src/contracts/officialToolContracts.ts` and intentionally describes one Ghidra Program. @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 ### Host readiness versus selected-program validity On #8820 head `a4b7dfcd1`, macOS arm64 and Ghidra 12.1.4, I selected an owned expanded project containing a root program named `program`, but set `REA_GHIDRA_PROGRAM_NAME=not_a_program`. Scoped `rea doctor --provider ghidra --target <target>` reported healthy; a subsequent `rea decompile` rejected the selection with `configuration_invalid`, named `REA_GHIDRA_PROGRAM_NAME`, and advised selecting an exact root program without modifying the project or falling back to a fresh import. Selecting `program` instead returned the expected function. The target, original packed input and every regular file in the expanded project kept their hashes during REA use; none was executed. This is consistent with the documented doctor contract: its provider checks are host-level and do not attest that a particular project program exists or matches the target. It is not evidence that the installation check is wrong. It does show that project discovery or readiness UX must keep host readiness, selected project shape, exact program selection, target identity and successful analysis separate; finding a project file or receiving a healthy doctor result does not establish the later facts. No project-discovery operation was implemented in this control. ### Single renamed program with spaces On #8820 head `f3c0336c5`, I used Ghidra 12.1.4 headless to import a copy of an owned Mach-O named `renamed sample` into an expanded project whose directory and project name contained spaces. The admitted REA target was a separate same-digest file named `arm64`. In compact/full MCP, leaving `REA_GHIDRA_PROGRAM_NAME` unset admitted the target lazily, then the first pseudocode query returned `configuration_invalid`, explaining that the default name `arm64` was not a root program and asking for the exact name. Restarting with `REA_GHIDRA_PROGRAM_NAME=renamed sample` successfully decompiled the expected function in both profiles. The target SHA-256 `0e0ecf82c96e4918ee2ca1287dc25513d8fd13d5ab7fa5b98db00ad342f03930` and regular original project files remained unchanged. Ghidra's import log supplied the name in this control; REA itself did not enumerate it. This verifies exact-name selection and preservation for one program and host, not candidate discovery or automatic same-digest selection.
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

A second Ghidra 12.1.4 control narrows the design space. A headless script running against one already-known root program traversed state.getProject().getProjectData().getRootFolder() using DomainFolder.getFiles() / getFolders() and inspected each DomainFile's path, content type and saved metadata. Without opening each candidate as a Program, it reported both /workflow and /group/workflow in the owned nested-project fixture. Both happened to expose Executable SHA256 in saved metadata. Ghidra's own batch-project example shows the folder traversal and distinguishes content types and links.

In a separate owned project whose saved program's executable SHA-256 had previously been cleared, the same metadata traversal omitted the SHA-256 key; the script did not treat this as a mismatch. Both runs used disposable copies and -noanalysis -readOnly; original project and binary hashes remained unchanged. The first copied project's bookkeeping files changed, while the second copy's did not. Neither target ran.

This is not yet a complete solution for an unknown project: the scripts were launched using an exact known root program, so an empty project or a project with no known eligible root program needs a separate bootstrap strategy. Saved metadata is a discovery hint, not proof that the current program database is unmodified or that analysis is complete; exact candidate selection and the existing live imported-digest check still matter. This tests two owned projects on one host, not stale metadata, links, unreadable entries, pagination, cancellation or cost.

<!-- gh-comment-id:6546337046 --> @morluto commented on GitHub (Oct 31, 2026): A second Ghidra 12.1.4 control narrows the design space. A headless script running against **one already-known root program** traversed `state.getProject().getProjectData().getRootFolder()` using `DomainFolder.getFiles()` / `getFolders()` and inspected each `DomainFile`'s path, content type and saved metadata. Without opening each candidate as a `Program`, it reported both `/workflow` and `/group/workflow` in the owned nested-project fixture. Both happened to expose `Executable SHA256` in saved metadata. Ghidra's own [batch-project example](https://github.com/NationalSecurityAgency/ghidra/blob/master/Ghidra/Features/Base/ghidra_scripts/CallAnotherScriptForAllPrograms.java) shows the folder traversal and distinguishes content types and links. In a separate owned project whose saved program's executable SHA-256 had previously been cleared, the same metadata traversal omitted the SHA-256 key; the script did not treat this as a mismatch. Both runs used disposable copies and `-noanalysis -readOnly`; original project and binary hashes remained unchanged. The first copied project's bookkeeping files changed, while the second copy's did not. Neither target ran. This is not yet a complete solution for an unknown project: the scripts were launched using an exact known root program, so an empty project or a project with no known eligible root program needs a separate bootstrap strategy. Saved metadata is a discovery hint, not proof that the current program database is unmodified or that analysis is complete; exact candidate selection and the existing live imported-digest check still matter. This tests two owned projects on one host, not stale metadata, links, unreadable entries, pagination, cancellation or cost.
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

The bootstrap limitation in my previous comment is not inherent to Ghidra's project API. I built a separate, disposable Java GhidraLaunchable against the installed Ghidra 12.1.4: initialize a headless Ghidra application, call GhidraProject.openProject, traverse its root DomainFolder, read each DomainFile's content type/path/saved metadata, and close the project. The helper did not select or open a Program.

On copied fixtures it returned /workflow and /group/workflow with the same recorded SHA-256; a project whose recorded digest was cleared returned that field as absent; a newly created disposable empty project returned zero items successfully. Original source-project file hashes stayed unchanged. Opening the nested disposable copy changed its bookkeeping files; the unknown-digest copy and empty project did not change in these runs. No target was executed. An initial launch attempt with an incomplete classpath failed before opening the projects; adding the installed Ghidra jars allowed the checks above.

This is a feasibility experiment, not an REA implementation or measured improvement: the helper currently uses all installed Ghidra jars, has no bounds, structured error/paging contract, cancellation controls, ownership-aware cleanup, concurrent-edit checks, or supported cross-platform packaging. openProject on an owned copy is not proof of a read-only storage operation, and saved metadata remains a hint rather than a verified selected-program digest. These gaps need to be addressed before considering integration.

<!-- gh-comment-id:6546358285 --> @morluto commented on GitHub (Oct 31, 2026): The bootstrap limitation in my previous comment is not inherent to Ghidra's project API. I built a separate, disposable Java `GhidraLaunchable` against the installed Ghidra 12.1.4: initialize a headless Ghidra application, call [`GhidraProject.openProject`](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/Ghidra/Features/Base/src/main/java/ghidra/base/project/GhidraProject.java), traverse its root `DomainFolder`, read each `DomainFile`'s content type/path/saved metadata, and close the project. The helper did **not** select or open a `Program`. On copied fixtures it returned `/workflow` and `/group/workflow` with the same recorded SHA-256; a project whose recorded digest was cleared returned that field as absent; a newly created disposable empty project returned zero items successfully. Original source-project file hashes stayed unchanged. Opening the nested disposable copy changed its bookkeeping files; the unknown-digest copy and empty project did not change in these runs. No target was executed. An initial launch attempt with an incomplete classpath failed before opening the projects; adding the installed Ghidra jars allowed the checks above. This is a feasibility experiment, not an REA implementation or measured improvement: the helper currently uses all installed Ghidra jars, has no bounds, structured error/paging contract, cancellation controls, ownership-aware cleanup, concurrent-edit checks, or supported cross-platform packaging. `openProject` on an owned copy is not proof of a read-only storage operation, and saved metadata remains a hint rather than a verified selected-program digest. These gaps need to be addressed before considering integration.
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

I exercised link entries in a disposable copy of the nested project. Using Ghidra's project API, I created an internal folder link group_alias to /group and an internal program link program_alias to /workflow. A project-root metadata traversal then reported /workflow as Program, /program_alias as ProgramLink, and /group_alias as FolderLink; getFolders() returned the ordinary /group, not the folder-link entry in this fixture. The original project remained unchanged.

On current REA main 60149282 / Ghidra 12.1.4, selecting workflow worked. Selecting either link name bound the target lazily, then list_documents returned the actionable configuration_invalid missing-root-program outcome. Direct analyzeHeadless -process on another disposable copy showed the same result: workflow processed successfully, while each link name was reported as no matching project program. The linked source project and target hashes remained unchanged during these queries. This is consistent with headless processing, not a claim that REA broke a supported link selection.

For discovery, this means an entry list should preserve actual content type/link status and distinguish a visible link from a selectable program; blindly presenting every DomainFile as a selectable program would give incorrect guidance. Whether and how REA should resolve internal links is a separate design question. No external/network links, broken links, link cycles, other Ghidra versions, or nested selection implementation were tested.

<!-- gh-comment-id:6546379356 --> @morluto commented on GitHub (Oct 31, 2026): I exercised link entries in a **disposable copy** of the nested project. Using Ghidra's project API, I created an internal folder link `group_alias` to `/group` and an internal program link `program_alias` to `/workflow`. A project-root metadata traversal then reported `/workflow` as `Program`, `/program_alias` as `ProgramLink`, and `/group_alias` as `FolderLink`; `getFolders()` returned the ordinary `/group`, not the folder-link entry in this fixture. The original project remained unchanged. On current REA main `60149282` / Ghidra 12.1.4, selecting `workflow` worked. Selecting either link name bound the target lazily, then `list_documents` returned the actionable `configuration_invalid` missing-root-program outcome. Direct `analyzeHeadless -process` on another disposable copy showed the same result: `workflow` processed successfully, while each link name was reported as no matching project program. The linked source project and target hashes remained unchanged during these queries. This is consistent with headless processing, not a claim that REA broke a supported link selection. For discovery, this means an entry list should preserve actual content type/link status and distinguish a visible link from a selectable program; blindly presenting every `DomainFile` as a selectable program would give incorrect guidance. Whether and how REA should resolve internal links is a separate design question. No external/network links, broken links, link cycles, other Ghidra versions, or nested selection implementation were tested.
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

A concrete control for the metadata caveat: the standalone project inventory reported the saved Executable SHA256 of a previously edited owned program as equal to its original binary (a6bd938bf4ae8031a33d270ca7698ee1ced5b78347a9c4fa3750b0e57f6b126c). The program's instruction bytes had been deliberately changed in the project database before this check; the original binary was unchanged.

On current REA main 60149282 with real Ghidra 12.1.4, the caller-selected project and a fresh import both mapped the tested entry to file offset 728. The saved project returned 40058052c0035fd6 and pseudocode returning 0x2a; the fresh import and independent original-file read returned 00008052c0035fd6 and pseudocode returning 0. The selected-project Evidence included the existing prior-edits caveat. inspect_native_load_image's source-file view showed different original/modified source digests for the saved project and equal digests for this fresh control; both views survived export and target-free import unchanged. Original project and target hashes remained unchanged during the REA sessions.

This illustrates why a discovery result must describe a saved import digest as an identity hint, not unchanged program bytes or equivalent analysis. Digest inequality alone also does not establish an analyst edit: a separate earlier fresh-import relocation control is documented in #8232. One intentionally edited fixture, not a prevalence or runtime-execution claim.

<!-- gh-comment-id:6546395289 --> @morluto commented on GitHub (Oct 31, 2026): A concrete control for the metadata caveat: the standalone project inventory reported the saved `Executable SHA256` of a previously edited owned program as equal to its original binary (`a6bd938bf4ae8031a33d270ca7698ee1ced5b78347a9c4fa3750b0e57f6b126c`). The program's instruction bytes had been deliberately changed in the project database before this check; the original binary was unchanged. On current REA main `60149282` with real Ghidra 12.1.4, the caller-selected project and a fresh import both mapped the tested entry to file offset 728. The saved project returned `40058052c0035fd6` and pseudocode returning `0x2a`; the fresh import and independent original-file read returned `00008052c0035fd6` and pseudocode returning `0`. The selected-project Evidence included the existing prior-edits caveat. `inspect_native_load_image`'s source-file view showed different original/modified source digests for the saved project and equal digests for this fresh control; both views survived export and target-free import unchanged. Original project and target hashes remained unchanged during the REA sessions. This illustrates why a discovery result must describe a saved import digest as an identity hint, not unchanged program bytes or equivalent analysis. Digest inequality alone also does not establish an analyst edit: a separate earlier fresh-import relocation control is documented in #8232. One intentionally edited fixture, not a prevalence or runtime-execution claim.
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

A related selection/handoff control using a local conflict resolution of nested-folder PR #3817 onto current main: the selected Ghidra project contained root and nested programs with the same imported artifact and distinct analyst names/comments. Root and a nested folder with spaces both opened through real REA/Ghidra; the source project remained unchanged.

Because selection is still configured on process startup, I exported the root dossier Evidence, started a separately configured nested MCP server, imported that bundle, and compared the root and nested dossiers. The comparison retained both sources and its observed text/comment differences, while keeping function correspondence ambiguous. A combined bundle imported into a targetless server reproduced the exact comparison and Evidence ID. Separate address_to_file_offset and bounded read_bytes calls gave matching mappings/bytes for this constructed pair, with independent Mach-O load-image verification still unsupported; these observations do not establish arbitrary project equivalence.

This is a viable explicit recovery path for comparing saved programs, but candidate discovery alone would not let an agent change the selected program inside the existing server. It does not establish whether per-open selection belongs in REA or justify treating same-digest candidates as interchangeable. More details and the local merge limitations are on PR #3817.

<!-- gh-comment-id:6547985375 --> @morluto commented on GitHub (Oct 31, 2026): A related selection/handoff control using a **local conflict resolution** of nested-folder PR #3817 onto current main: the selected Ghidra project contained root and nested programs with the same imported artifact and distinct analyst names/comments. Root and a nested folder with spaces both opened through real REA/Ghidra; the source project remained unchanged. Because selection is still configured on process startup, I exported the root dossier Evidence, started a separately configured nested MCP server, imported that bundle, and compared the root and nested dossiers. The comparison retained both sources and its observed text/comment differences, while keeping function correspondence ambiguous. A combined bundle imported into a targetless server reproduced the exact comparison and Evidence ID. Separate `address_to_file_offset` and bounded `read_bytes` calls gave matching mappings/bytes for this constructed pair, with independent Mach-O load-image verification still unsupported; these observations do not establish arbitrary project equivalence. This is a viable explicit recovery path for comparing saved programs, but candidate discovery alone would not let an agent change the selected program inside the existing server. It does not establish whether per-open selection belongs in REA or justify treating same-digest candidates as interchangeable. More details and the local merge limitations are on [PR #3817](https://github.com/morluto/rea/pull/3817#issuecomment-6547983494).
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

Broken-link control on macOS arm64 with Ghidra 12.1.4 and REA main d54a4661: in a disposable copy of the earlier project, I created an internal ProgramLink to a nonexistent program and an internal FolderLink to a nonexistent folder. Root metadata enumeration retained both as link entries with no recorded executable SHA-256. For these local links, Ghidra's LinkFileInfo.getLinkStatus reported BROKEN and identified the missing target; the existing valid program and folder links reported INTERNAL, with their targets present. The check skipped external links and did not open a candidate Program.

Through public REA MCP, selecting the actual root program succeeded. Selecting the valid program link, broken program link or broken folder link each produced configuration_invalid on list_documents; close_binary succeeded in each session. REA left the selected source project and binary byte-identical. This is consistent with the current root-program selection contract, not an assertion that REA promises to resolve links.

For any future discovery UX, retain entry type and link-resolution status as separate facts: ProgramLink alone does not establish a selectable program or an existing target. If status is not checked, leave it unknown. This probe does not establish behavior or side effects for external/network links, cycles, permission failures, concurrent edits or other Ghidra versions; it is not an argument to follow external links automatically.

<!-- gh-comment-id:6549092828 --> @morluto commented on GitHub (Oct 31, 2026): Broken-link control on macOS arm64 with Ghidra 12.1.4 and REA main `d54a4661`: in a disposable copy of the earlier project, I created an internal `ProgramLink` to a nonexistent program and an internal `FolderLink` to a nonexistent folder. Root metadata enumeration retained both as link entries with no recorded executable SHA-256. For these **local** links, Ghidra's `LinkFileInfo.getLinkStatus` reported `BROKEN` and identified the missing target; the existing valid program and folder links reported `INTERNAL`, with their targets present. The check skipped external links and did not open a candidate `Program`. Through public REA MCP, selecting the actual root program succeeded. Selecting the valid program link, broken program link or broken folder link each produced `configuration_invalid` on `list_documents`; `close_binary` succeeded in each session. REA left the selected source project and binary byte-identical. This is consistent with the current root-program selection contract, not an assertion that REA promises to resolve links. For any future discovery UX, retain entry type and link-resolution status as separate facts: `ProgramLink` alone does not establish a selectable program or an existing target. If status is not checked, leave it unknown. This probe does not establish behavior or side effects for external/network links, cycles, permission failures, concurrent edits or other Ghidra versions; it is not an argument to follow external links automatically.
Author
Owner

@morluto commented on GitHub (Nov 1, 2026):

Current-main nested-selection control

On clean main 6e9ee97a, macOS arm64 and Ghidra 12.1.4, I retested the owned project from #3817. Direct analyzeHeadless on a disposable copy selected /workflow and /group/workflow separately using the project folder in its project argument. Both programs recorded the same imported binary SHA-256, while their authored function names differ. This agrees with Ghidra's headless folder syntax.

With current REA and REA_GHIDRA_PROGRAM_NAME=workflow, the session selected the root program: list_documents returned workflow, root-authored pseudocode succeeded, and the nested-only name was unavailable. Setting REA_GHIDRA_PROGRAM_NAME=group/workflow prevented MCP startup: SDK connect observed the server close and stderr explicitly said to select one root program name without paths.

I also checked REA_GHIDRA_PROGRAM_FOLDER=/group, the configuration name used by the still-open, conflicting #3817 branch. That setting is not supported by current main: it was ignored, and REA again selected the root program with the same analysis-profile digest and root-authored pseudocode. This is a control for someone trying the unmerged branch's configuration, not a claim that main promises folder support. If nested selection is implemented, preserve the folder as part of the selected program identity/profile and fail explicitly on invalid selection; the source binary digest alone cannot distinguish these two saved analyses.

The source project regular-file hashes and binary SHA-256 were unchanged during the REA calls; direct Ghidra operated on a disposable copy. Successful MCP responses were checked against fetched compact schemas. This is one constructed project on one host, not a claim about other folder forms, existing user configurations or agent behavior.

A follow-up left REA_GHIDRA_PROGRAM_NAME unset. Because the target basename was workflow, REA again selected the root program and produced the same profile digest and root-only pseudocode; it did not enumerate the nested same-named program. This matches the current root-name default, but reinforces that the reported selected name is not a project-wide uniqueness check. Original project and target hashes stayed unchanged.

<!-- gh-comment-id:6551251118 --> @morluto commented on GitHub (Nov 1, 2026): ### Current-main nested-selection control On clean main `6e9ee97a`, macOS arm64 and Ghidra 12.1.4, I retested the owned project from #3817. Direct `analyzeHeadless` on a disposable copy selected `/workflow` and `/group/workflow` separately using the project folder in its project argument. Both programs recorded the same imported binary SHA-256, while their authored function names differ. This agrees with Ghidra's [headless folder syntax](https://github.com/NationalSecurityAgency/ghidra/blob/master/Ghidra/RuntimeScripts/support/analyzeHeadlessREADME.md#project_namefolder_path). With current REA and `REA_GHIDRA_PROGRAM_NAME=workflow`, the session selected the root program: `list_documents` returned `workflow`, root-authored pseudocode succeeded, and the nested-only name was unavailable. Setting `REA_GHIDRA_PROGRAM_NAME=group/workflow` prevented MCP startup: SDK connect observed the server close and stderr explicitly said to select one root program name without paths. I also checked `REA_GHIDRA_PROGRAM_FOLDER=/group`, the configuration name used by the still-open, conflicting #3817 branch. **That setting is not supported by current main**: it was ignored, and REA again selected the root program with the same analysis-profile digest and root-authored pseudocode. This is a control for someone trying the unmerged branch's configuration, not a claim that main promises folder support. If nested selection is implemented, preserve the folder as part of the selected program identity/profile and fail explicitly on invalid selection; the source binary digest alone cannot distinguish these two saved analyses. The source project regular-file hashes and binary SHA-256 were unchanged during the REA calls; direct Ghidra operated on a disposable copy. Successful MCP responses were checked against fetched compact schemas. This is one constructed project on one host, not a claim about other folder forms, existing user configurations or agent behavior. A follow-up left `REA_GHIDRA_PROGRAM_NAME` unset. Because the target basename was `workflow`, REA again selected the root program and produced the same profile digest and root-only pseudocode; it did not enumerate the nested same-named program. This matches the current root-name default, but reinforces that the reported selected name is not a project-wide uniqueness check. Original project and target hashes stayed unchanged.
Author
Owner

@morluto commented on GitHub (Nov 2, 2026):

Direct Ghidra no-match control

On a disposable copy of the same owned two-folder project, Ghidra 12.1.4 headless on macOS arm64 with -process '*' -recursive -noanalysis -readOnly ran the identity script on both programs and exited 0. Repeating with a nonmatching exact name, then a nonmatching wildcard, returned exit 1, no script rows, and Requested project program file(s) not found in both cases. All 11 regular files in the source copy retained their hashes; only a fresh disposable project copy was processed. The target was not run.

For this fixture, an empty script output from a selective -process invocation must therefore be interpreted together with Ghidra's reported selection failure, not silently promoted to a successful empty inventory. Together with the earlier injected per-program failure that exited 0, this shows that neither nonzero nor zero process exit alone describes complete candidate coverage. This is direct Ghidra behavior for one project and two selectors, not an implemented REA discovery operation or a claim about empty projects and other error modes.

<!-- gh-comment-id:6584633004 --> @morluto commented on GitHub (Nov 2, 2026): ### Direct Ghidra no-match control On a disposable copy of the same owned two-folder project, Ghidra 12.1.4 headless on macOS arm64 with `-process '*' -recursive -noanalysis -readOnly` ran the identity script on both programs and exited 0. Repeating with a nonmatching exact name, then a nonmatching wildcard, returned exit 1, no script rows, and `Requested project program file(s) not found` in both cases. All 11 regular files in the source copy retained their hashes; only a fresh disposable project copy was processed. The target was not run. For this fixture, an empty script output from a selective `-process` invocation must therefore be interpreted together with Ghidra's reported selection failure, not silently promoted to a successful empty inventory. Together with the earlier injected per-program failure that exited 0, this shows that neither nonzero nor zero process exit alone describes complete candidate coverage. This is direct Ghidra behavior for one project and two selectors, not an implemented REA discovery operation or a claim about empty projects and other error modes.
Author
Owner

@morluto commented on GitHub (Nov 2, 2026):

Empty-project control

On macOS arm64 with Ghidra 12.1.4, I created a new disposable local project by importing an owned fixture with -noanalysis and a headless post-script requesting CONTINUE_THEN_DELETE. The import exited 0 and Ghidra logged that it discarded the only imported program. Processing that project with -process '*' -recursive -noanalysis -readOnly then exited 1 with Requested project program file(s) not found: * and produced no identity-script rows. The source fixture was not executed and its SHA-256 remained 9fc02faea0c3723a094c209adbe62a279011df4d508298625bd766fa20ed7c46.

Thus, in this one verified empty project, headless reports an empty wildcard selection through the same broad failure form seen for a nonmatching selector in a populated project. A prospective REA inventory needs independent evidence about the selected project and enumeration scope before converting that result into a complete empty list; it should preserve genuine project/processing errors. This is direct Ghidra behavior only, not a claim that all projects or Ghidra versions behave alike.

<!-- gh-comment-id:6584666013 --> @morluto commented on GitHub (Nov 2, 2026): ### Empty-project control On macOS arm64 with Ghidra 12.1.4, I created a new disposable local project by importing an owned fixture with `-noanalysis` and a headless post-script requesting `CONTINUE_THEN_DELETE`. The import exited 0 and Ghidra logged that it discarded the only imported program. Processing that project with `-process '*' -recursive -noanalysis -readOnly` then exited 1 with `Requested project program file(s) not found: *` and produced no identity-script rows. The source fixture was not executed and its SHA-256 remained `9fc02faea0c3723a094c209adbe62a279011df4d508298625bd766fa20ed7c46`. Thus, in this one verified empty project, headless reports an empty wildcard selection through the same broad failure form seen for a nonmatching selector in a populated project. A prospective REA inventory needs independent evidence about the selected project and enumeration scope before converting that result into a complete empty list; it should preserve genuine project/processing errors. This is direct Ghidra behavior only, not a claim that all projects or Ghidra versions behave alike.
Author
Owner

@morluto commented on GitHub (Nov 2, 2026):

Program-independent project inventory control

Ghidra 12.1.4's bundled headless documentation describes running program-independent scripts without -import or -process. I tested that route on disposable copies of both owned projects from the controls above, using -readOnly and a pre-script that recursively visits state.getProject().getProjectData().getRootFolder(), DomainFolder.getFiles() and getFolders(). It does not open a program or invoke analysis. The populated project returned /group/workflow and /other/workflow as Program entries; their DomainFile.getMetadata() maps each reported the same imported SHA-256. The independently constructed empty project returned an explicit completion marker with count zero. Both invocations exited 0; source-project regular-file hashes remained unchanged.

This offers a more direct candidate-discovery direction than processing every program, and can represent an observed empty project in these controls. Metadata is a recorded project observation, not proof of current target identity, project immutability or equivalent saved analysis: the two same-digest programs have different annotations. The experiment does not establish bounds, ordering, lazy metadata cost, non-program handling, cancellation, partial failures, concurrent changes, cross-version behavior, or whether -readOnly avoids changes to the disposable copy's bookkeeping. Any REA implementation still needs explicit caller selection, preservation of unknown metadata and per-scope completion, source-project ownership, and the applicable authority decision. This is a direct Ghidra experiment, not an REA capability.

<!-- gh-comment-id:6584688576 --> @morluto commented on GitHub (Nov 2, 2026): ### Program-independent project inventory control Ghidra 12.1.4's bundled headless documentation describes running program-independent scripts without `-import` or `-process`. I tested that route on disposable copies of both owned projects from the controls above, using `-readOnly` and a pre-script that recursively visits `state.getProject().getProjectData().getRootFolder()`, `DomainFolder.getFiles()` and `getFolders()`. It does not open a program or invoke analysis. The populated project returned `/group/workflow` and `/other/workflow` as `Program` entries; their `DomainFile.getMetadata()` maps each reported the same imported SHA-256. The independently constructed empty project returned an explicit completion marker with count zero. Both invocations exited 0; source-project regular-file hashes remained unchanged. This offers a more direct candidate-discovery direction than processing every program, and can represent an observed empty project in these controls. Metadata is a recorded project observation, not proof of current target identity, project immutability or equivalent saved analysis: the two same-digest programs have different annotations. The experiment does not establish bounds, ordering, lazy metadata cost, non-program handling, cancellation, partial failures, concurrent changes, cross-version behavior, or whether `-readOnly` avoids changes to the disposable copy's bookkeeping. Any REA implementation still needs explicit caller selection, preservation of unknown metadata and per-scope completion, source-project ownership, and the applicable authority decision. This is a direct Ghidra experiment, not an REA capability.
Author
Owner

@morluto commented on GitHub (Nov 2, 2026):

A direct Ghidra 12.1.4 program-independent script on a disposable copy of the owned two-program project emitted one entry at a caller-selected limit of one and explicitly reported has_more=true after encountering the second. It exited 0 and the 11 regular files of the source project copy retained their hashes. This tests one output-limit control, not a resumable cursor, performance, or cancellation.

The bundled Ghidra 12.1.4 DomainFolder API documents that getFiles() and getFolders() return arrays for an entire folder and may use cached data without forcing a full refresh. Therefore, bounding emitted rows alone does not prove bounded per-folder materialization or fresh live inventory. The same bundled source describes linked-folder/file semantics; ProjectDataUtils.descendantFiles(folder) ignores link files by default, whereas its filtered overload can follow selected links. A design should state its link scope and avoid treating ignored or inaccessible linked content as inspected; do not follow external links merely to make an inventory appear complete. These are source-derived API observations, not measurements or a test of a linked project. Limits, partial completion, ordering and project identity across continuation still need a concrete design and runtime checks.

<!-- gh-comment-id:6584720643 --> @morluto commented on GitHub (Nov 2, 2026): ### Bounded project inventory and link scope A direct Ghidra 12.1.4 program-independent script on a disposable copy of the owned two-program project emitted one entry at a caller-selected limit of one and explicitly reported `has_more=true` after encountering the second. It exited 0 and the 11 regular files of the source project copy retained their hashes. This tests one output-limit control, not a resumable cursor, performance, or cancellation. The bundled Ghidra 12.1.4 `DomainFolder` API documents that `getFiles()` and `getFolders()` return arrays for an entire folder and may use cached data without forcing a full refresh. Therefore, bounding emitted rows alone does not prove bounded per-folder materialization or fresh live inventory. The same bundled source describes linked-folder/file semantics; `ProjectDataUtils.descendantFiles(folder)` ignores link files by default, whereas its filtered overload can follow selected links. A design should state its link scope and avoid treating ignored or inaccessible linked content as inspected; do not follow external links merely to make an inventory appear complete. These are source-derived API observations, not measurements or a test of a linked project. Limits, partial completion, ordering and project identity across continuation still need a concrete design and runtime checks.
Author
Owner

@morluto commented on GitHub (Nov 2, 2026):

Linked-folder control

On a disposable copy of the owned two-program project, a program-independent Ghidra 12.1.4 script created an internal folder link /audit_link pointing to /group. A separate headless -readOnly script saw /audit_link in the root's getFiles() as FolderLink, with isLink=true and isFolderLink=true; root getFolders() returned /group and /other, not the link. ProjectDataUtils.descendantFiles(root) returned the two original program paths and omitted the link. Both scripts completed, and the original project's regular-file hashes remained unchanged. Creation intentionally modified only the disposable copy.

An inventory using the default descendant iterator can therefore omit even the existence of a link in this controlled case; a non-following inventory that needs to disclose skipped links must separately observe them. This is one internal link on macOS, not an external/broken-link, permission, cycle, or network test. It does not imply that link targets should be followed or counted as candidates.

<!-- gh-comment-id:6584746065 --> @morluto commented on GitHub (Nov 2, 2026): ### Linked-folder control On a disposable copy of the owned two-program project, a program-independent Ghidra 12.1.4 script created an internal folder link `/audit_link` pointing to `/group`. A separate headless `-readOnly` script saw `/audit_link` in the root's `getFiles()` as `FolderLink`, with `isLink=true` and `isFolderLink=true`; root `getFolders()` returned `/group` and `/other`, not the link. `ProjectDataUtils.descendantFiles(root)` returned the two original program paths and omitted the link. Both scripts completed, and the original project's regular-file hashes remained unchanged. Creation intentionally modified only the disposable copy. An inventory using the default descendant iterator can therefore omit even the existence of a link in this controlled case; a non-following inventory that needs to disclose skipped links must separately observe them. This is one internal link on macOS, not an external/broken-link, permission, cycle, or network test. It does not imply that link targets should be followed or counted as candidates.
Author
Owner

@morluto commented on GitHub (Nov 2, 2026):

Read-only inventory copy check

For the program-independent project inventory control, I repeated Ghidra 12.1.4 -readOnly against a fresh disposable copy of the two-program project and hashed every regular file in both the copied project and its source before and after. The script completed and reported both candidates; the copy had the same 11 regular-file paths and SHA-256 values afterward, with none added or removed. The source project likewise remained unchanged. This narrows the earlier uncertainty about copy bookkeeping for this fixture and one invocation; it does not check timestamps, directory metadata, transient writes, other project states or platforms, and does not establish that using a caller-owned project directly is safe.

<!-- gh-comment-id:6584769128 --> @morluto commented on GitHub (Nov 2, 2026): ### Read-only inventory copy check For the program-independent project inventory control, I repeated Ghidra 12.1.4 `-readOnly` against a fresh disposable copy of the two-program project and hashed every regular file in both the copied project and its source before and after. The script completed and reported both candidates; the copy had the same 11 regular-file paths and SHA-256 values afterward, with none added or removed. The source project likewise remained unchanged. This narrows the earlier uncertainty about copy bookkeeping for this fixture and one invocation; it does not check timestamps, directory metadata, transient writes, other project states or platforms, and does not establish that using a caller-owned project directly is safe.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6550
No description provided.