mirror of
https://github.com/morluto/rea.git
synced 2026-11-05 06:45:22 +00:00
[GH-ISSUE #8165] [Design] Preserve and reopen owned Ghidra analysis with a packed-program checkpoint #4691
Open
opened 2026-10-31 00:52:31 +00:00 by bsakaguchi
·
7 comments
No Branch/Tag specified
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
Mirrored from GitHub Pull Request
question
roadmap
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
morluto/rea#4691
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8165
Before submitting
Decision needed
Should REA support an explicit, caller-directed checkpoint and later reopening of an owned Ghidra program, including session edits, without modifying the selected source project or executable? Is a Ghidra packed program (
.gzf) an appropriate first format?Context and evidence
At REA commit
47fa6ab3795d547207001ffb01643c0aa39dcad8, fresh-import Ghidra sessions are temporary. Metadata edits are discarded on close, and cancellation of an active operation ends the session. Existing-project mode copies a caller-selected.gpr/.repinto an owned runtime, validates the program's recorded import SHA-256, and opens it read-only without reanalysis; REA does not allow edits in that mode. An Evidence bundle or REA analysis snapshot is not a resumable Ghidra database.Observed on macOS arm64, Ghidra 12.1.4 and JDK 21: a small self-built Mach-O was imported with
analyzeHeadless -readOnly; a post-script renamed one function and calledcurrentProgram.saveToPackedFileafter ending the script transaction. A second headless invocation imported the.gzfinto a new project with-noanalysis: the renamed function was present, two functions were reported, and the recorded executable SHA-256 matched the fixture. The initial attempt without ending the script transaction failed withUnable to lock due to active transaction. That script error was logged even though the headless process exited zero. REA's bridge already ends its script transaction before serving requests.A real stdio MCP session then selected the restored project via the existing environment settings:
procedure_pseudo_codereturned the renamed function;annotate_native_functionreturnedcapability_unavailablewith reasonExisting Ghidra project sessions are read-only. The packed export itself was not produced by REA. No large-target, cross-version, cancellation or failure-recovery results were measured.Expected interaction with targets
Read existing files; launch the user-supplied Ghidra process; mutate metadata in an owned analysis program; write a caller-selected new artifact. The target executable and source project would remain unchanged. Other effects and applicable platform scope need review.
Boundary details (optional)
Preserve the selected target and provider identity, source project fingerprint, Ghidra version, explicit unknown analysis completeness and the origin of prior edits. Avoid implying that a
.gzfis a full project archive. Do not conflate Evidence export or REA snapshots with an editable engine database. Windows P0 currently has no database-mutation authority.Options and tradeoffs
Investigate exporting a single packed program after edits, with exclusive destination creation and a verified reopen/import path. An alternative is a new owned project or exporting/replaying only analyst annotations (#6405). Decide whether a private copy of an existing project may be edited and checkpointed, and how to select it for an individual CLI/MCP session; #3751 already discusses project and
.gzfselection. Direct writes to the original project and arbitrary script execution are separate scope decisions.Verification available so far
I have a prototype or observed results
Success criteria and open questions
Establish real Ghidra CLI/MCP round trips for fresh and existing-project sessions, including comments, names, function/type data and import identity. Verify version and format compatibility, concurrency/transaction boundaries, failed or interrupted writes, cancellation during finalization, destination conflicts, and cleanup retry. Determine what cannot be preserved and when the latest edits are at risk. Measure a representative repeated investigation before claiming time savings. A maintainer scope decision is required before substantial implementation intended for integration.
References
.gzfinput), #6405 (annotation export and replay), #3136 (reuse benefit research), #5271 (long-running analysis), #3509 (transport failure).Additional cancellation observation
On a self-built Mach-O with a separate slow function, a real REA/Ghidra 12.1.4 stdio MCP session renamed
fast_auditand added an entry comment; pseudocode showed both. Cancelling a different 1,200-branch decompile after 300 ms returned an SDK timeout; a later query returned the original function name without the comment, andclose_binarysucceeded. This establishes lost unsaved annotations on one active-operation cancellation, not every cancellation/finalization path.A second run exported an Evidence bundle after cancellation and before querying again. That bundle contained the completed annotation record and pseudocode with the prior name/comment, even though the next live Ghidra query returned the original name. This preserves historical observations, not a resumable edited Ghidra program; it can mitigate loss of recorded context.
Existing immutable snapshot behavior verified
In a real Ghidra 12.1.4 stdio MCP session on a self-owned Mach-O,
analyze_functionreturned a summary with a retained complete Evidence ID.close_binary(snapshot_path=...)succeeded without session edits and reported one primitive cache entry and two Evidence records. In a fresh MCP server,open_binary(snapshot_path=...)succeeded for the same target/profile;inspect_analysis_viewreturned the retained dossier's pseudocode using the original Evidence ID before any new decompile request. This demonstrates a useful existing way to carry completed observations forward. It does not save an editable Ghidra database or establish that a later provider operation is served without recomputation. The separate edited-session snapshot refusal and packed-program experiment described above still apply.@morluto commented on GitHub (Oct 30, 2026):
A follow-up headless round trip on macOS arm64/Ghidra 12.1.4 exercised more than a rename. On the same owned Mach-O (
ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d), a post-script renamed_evaluate, set an entry comment, created/audit_checkpoint_recordwith three fields, applied a pointer parameter and return type, ended the script transaction, and wrote a.gzf. A second headless import with-noanalysisinto a new owned project observed the same name, comment, signature, structure and original executable SHA-256; its decompiler usedrecord->flagsandrecord->id.With that restored project selected through
REA_GHIDRA_PROJECT_PATHand its exact root program name, a real REA MCP session opened the original binary, returned the renamed/commented typed pseudocode, andinspect_native_data_typereturned the saved structure and fields.annotate_native_functionwas refused because existing-project sessions are read-only. A before/after content digest inventory of the selected project's 10 regular files matched, and the source binary SHA-256 remained unchanged.The
.gzfwas created and imported by separate user-owned headless scripts, not by REA. The initial Ghidra import logged unrecoverable DWARF expressions even though the manually applied facts survived; this is not proof that all debug information, project state, formats, versions or cleanup-failure cases round-trip. Ghidra's script log was checked for the explicit success markers rather than relying on exit status alone.@morluto commented on GitHub (Oct 30, 2026):
A related manual recovery control is recorded in #6405: completed annotation Evidence can be imported into a fresh session and a caller-selected name/comment can be reapplied after target and profile checks, but importing it does not restore the Ghidra database. Two sessions applying the same two edits in opposite orders produced identical annotation Evidence IDs in identical canonical bundle order despite ending with different live names/comments. Evidence alone is therefore not a chronological edit log. This narrows the existing mitigation; it does not replace an explicit checkpoint or prove general replay safety.
@morluto commented on GitHub (Oct 31, 2026):
Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.
@morluto commented on GitHub (Oct 31, 2026):
Real MCP/TypeScript SDK close-finalization control on a source-owned, unexecuted Mach-O with Ghidra 12.1.4 and REA
b43c329: after collecting a complete function dossier, callclose_binary({snapshot_path:<new path>}). A test-only scheduling barrier in a disposable built runtime held the operation after the Ghidra client reported successful retirement, but before the remaining session/file/candidate cleanup and record clearing finished. At the barrier the snapshot file already existed.In the cancellation run, the SDK aborted the close request and rejected with the caller's cancellation; the barrier was then released. A subsequent
binary_sessionreportedopen:falseandcleanup_pending:false. Reopening the same target with the saved snapshot succeeded, andinspect_analysis_viewretrieved the original dossier's pseudocode by its retained Evidence ID. A separate no-cancellation barrier control returned the snapshot receipt (one primitive entry, one Evidence record) and recovered the same view. Both snapshot files had the same SHA-256 in this fixture, both sessions closed, and the target hash remained unchanged.This is a positive result for one specific, deliberately delayed finalization boundary. The cancelled client received no close receipt, and the test does not establish whether the server transmitted a final response, cancellation behavior before snapshot publication, concurrent cleanup failures, or natural timing. The snapshot carries completed Evidence; it is not an edited Ghidra project. This complements the existing selected-project and packed-program continuity discussion.
@morluto commented on GitHub (Oct 31, 2026):
Pre-publication counterpart to the earlier close-finalization control: a disposable built-runtime scheduling barrier held
close_binary({snapshot_path:<new path>})after snapshot capture but immediately before its write. The destination did not yet exist when the TypeScript MCP SDK cancelled the request. After releasing the barrier, the SDK had already rejected with cancellation, but REA continued the close and published the snapshot. In the recorded run, status first reportedopen:false, cleanup_pending:true, then a later poll reportedopen:false, cleanup_pending:false. Reopening the target with that snapshot succeeded and its original function Evidence was retrievable. A no-cancellation control produced the same snapshot SHA-256; all closes succeeded and the unexecuted target hash was unchanged.This is real SDK request cancellation at a test-delayed boundary, not proof of natural timing or a claim that cancellation always yields a snapshot. The close operation's existing implementation deliberately owns cleanup independently of the caller signal; the test also shows the snapshot write may complete after the caller stops waiting. A prior immediate sample observed
open:true, cleanup_pending:falsewhile the close was still in progress, so neither a single status sample nor the client rejection establishes the final write outcome. No server close receipt was available to the cancelled caller. This may inform recovery guidance and commit-boundary design without treating a REA Evidence snapshot as an editable Ghidra checkpoint.@morluto commented on GitHub (Oct 31, 2026):
Explicit snapshot overwrite and edited-session refusal
Real REA
b43c329stdio MCP on an owned, unexecuted Mach-O, Ghidra 12.1.4 and Kuna 1.986:snapshot_pathpointing to an existing disposable test marker andoverwrite:true. Both closes succeeded and replaced their markers with valid snapshots. Reopening each with the same provider and target allowedinspect_analysis_viewto retrieve the original dossier by Evidence ID. Ghidra's receipt had one primitive entry; Kuna's had zero, so the Kuna result proves retained Evidence retrieval, not live-query cache reuse.evidence_integrity_mismatchexplaining that snapshots are unavailable after metadata changes. The marker's hash stayed unchanged,binary_sessionstill reported open/no cleanup pending, and the edited name still resolved. Closing without a snapshot then succeeded.The target hash remained unchanged and no target was executed. This tests an authorized overwrite and an early dirty-session refusal, not interrupted writes, permissions, cleanup failures or a resumable edited Ghidra project. Together with the earlier no-overwrite conflict control, it shows the practical recovery choice for these cases without implying snapshot support for live annotations.
@morluto commented on GitHub (Oct 31, 2026):
Destination-permission and directory recovery controls
Real REA
b43c329stdio MCP on the owned Mach-O with Ghidra 12.1.4 and Kuna 1.986, running as an unprivileged macOS user: after collecting a dossier, I removed write permission from a disposable destination directory and tried both an existing marker withoverwrite:trueand a new filename in that directory. Both requests returned a file-access error advising a permission check; the existing marker hash was unchanged and the new file was absent. For both providers,binary_sessionremained open with no cleanup pending after each failure, and a procedure lookup still succeeded. Closing to a different writable path then succeeded; reopening the snapshot retrieved the original dossier. Ghidra recorded one primitive entry, Kuna zero, so the latter is Evidence retrieval rather than proof of provider-query replay. Directory permissions were restored afterward.A separate Ghidra control selected an existing directory as the destination with
overwrite:true. REA returned the distinctinvalid_request/not-fileerror, preserved the directory's marker, kept the session usable, and closed successfully to a normal file. The target hash stayed unchanged throughout and no target was executed.These are positive recovery checks for one host's directory-permission and destination-type failures. They do not cover disk-full, interrupted publication, cleanup failure, ACLs, Windows, or all I/O causes. The permission result's structured reason was the general
io; no claim is made that every such error is a permission problem.