mirror of
https://github.com/morluto/rea.git
synced 2026-11-05 06:45:22 +00:00
[GH-ISSUE #8232] [Design] Compare selected Ghidra project memory with bounded original bytes #6501
Open
opened 2026-11-01 21:11:42 +00:00 by bsakaguchi
·
7 comments
No Branch/Tag specified
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
Mirrored from GitHub Pull Request
question
roadmap
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
morluto/rea#6501
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8232
Before submitting
Decision needed
Should an analyst be able to compare a bounded address range in a caller-selected Ghidra project's analyzed image with its mapped original artifact bytes, while keeping relocation effects and unknown causes explicit?
Context and evidence
On macOS arm64 / Ghidra 12.1.4, I imported an owned Mach-O into a new disposable project, used a separate headless script to change one instruction in the program database, and saved it. The executable itself was unchanged. The saved program's recorded import SHA-256 still matched the original executable, as expected: that is import identity, not a claim that its database was never edited.
A real REA MCP session selected this project. At
0x1000002d8,read_bytesreturned40058052c0035fd6and pseudocode returned0x2a. A separate REA fresh-import session for the same target returned00008052c0035fd6and pseudocode returned zero; direct reading of the original file at the Ghidra-reported offset 728 matched the fresh bytes. Both sessions mapped the address to 728. The existing-project Evidence records the project fingerprint, its import mode and a limitation that previous edits may exist; the installation guide correctly explains that matching import digests do not prove an unedited database. Original target and all ten selected project regular-file digests remained unchanged during REA's sessions. REA did not make the edit.inspect_native_load_image(detail: summary)followed by aninspect_analysis_viewnative-load-image/source_filespage exposed different original and modified source-file SHA-256 values for the edited project. The same view of this particular fresh-import fixture showed equal values. However, a different fresh-import Mach-O with a loader-applied relocation showed different original/modified source digests too, with anAPPLIEDrelocation. Therefore digest inequality alone does not identify a user patch. Both examples returnedverification: unsupportedfor Mach-O; an empty relocation collection in the edited example is only the provider's reported collection, not proof that all possible causes were excluded.Expected interaction with targets
Read selected local artifact and provider observations; launch the user-supplied analysis provider; no target execution, byte mutation, or write to the source project is proposed.
Boundary details (optional)
Bind each observation to the exact target, selected project fingerprint/program, address space, mapping and byte range. Preserve uninitialized, unmapped, overlay, nonunique and unavailable cases. Imported file identity, Ghidra file-byte digest, analyzed memory, loader relocations and analyst edits are distinct observations; unknown cause must remain unknown. Do not infer a user patch from digest inequality, assume a complete relocation inventory, or treat an address-to-file-offset mapping as proof of byte equality.
Options and tradeoffs
Compare documenting the existing multi-step workflow, enriching an existing bounded bytes/load-image view with independently read original bytes and coverage, or a retained comparison composed from the existing contracts. A whole-image comparison may be expensive; an agent asking about a specific instruction can select a bounded range. Reuse existing
read_bytes,address_to_file_offset, andinspect_native_load_imagerather than adding a provider-only API wrapper. Consider an explicit source/project origin in the result alongside existing profile and limitations, without weakening Evidence identity.Verification available so far
I have a prototype or observed results
Success criteria and open questions
Exercise a saved byte-edited project, a no-edit project, fresh imports with and without relocations, mapped/unmapped and uninitialized ranges, multiple source files, overlaps/overlays, changed source digest and partial reads. Confirm correct provenance and that relocation or unknown cause is not labeled a patch. Large projects, CLI, other operating systems, Ghidra versions, and actual agent decisions remain untested. Establish resource bounds and cleanup behavior before any integration.
References
@morluto commented on GitHub (Oct 30, 2026):
An overlay control adds an important limitation to digest-based comparison. In a new owned Ghidra project, I added a 16-byte initialized overlay at the same numeric address as the file-backed entry, without changing the original file-backed bytes. REA/Ghidra's source-files view reported equal original and modified source digests, while its mappings view separately exposed an initialized overlay in
AUDIT_OVERLAYwithsource_file_index: nullandfile_offset: null.read_bytesreturned different bytes for the explicit overlay and default address-space coordinates; mapping the overlay to a file offset was refused. Equal source-file digests therefore do not establish that all program memory is file-backed or that an overlay has matching original bytes.An explicitly bounded
search_bytesover the overlay found all nine overlapping matches across three pages. An unbounded search omitted it and placed it underskipped_not_loadedeven though the load-image mapping reported it as loaded; the provider guide documents this combined omission, and #8233 tracks the misleading category and MCP discoverability. This test used one authored overlay; it does not establish behavior for every overlay or provider. Target and project file contents stayed unchanged during REA sessions; the target was not executed.@morluto commented on GitHub (Oct 31, 2026):
Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.
@morluto commented on GitHub (Oct 31, 2026):
Explicit source offset with an edited saved program
I revisited the controlled edited-project fixture through the actual file-offset workflow. On REA
b43c329df779708e43ae297f498a844e45d9c923and Ghidra 12.1.4,procedure_pseudo_code({procedure:"file-offset:0x2d8"})selected source offset728, mapped address0x1000002d8, and the same procedure entry in both a fresh import and the caller-selected, read-only edited project. The fresh result returned0; the selected-project result returned0x2a.An independent read of the target file at offset 728 was
00008052c0035fd6. Real REA/Ghidraread_bytesreturned those bytes in the fresh import, and40058052c0035fd6in the previously edited project. The selected-project Evidence recorded its content fingerprint and warned that prior edits may be present.inspect_native_load_imagealso retained different original/modified source-file digests, with independent Mach-O verification unsupported. The target SHA-256 and ten project-file hashes stayed unchanged during the REA sessions; the target was not executed.This confirms a useful boundary for #8178 and this issue: a valid target-file coordinate can identify the source location while the selected analysis program contains different bytes. The test does not imply that every original/modified digest difference is a patch; the earlier loader-relocation control remains relevant.
@morluto commented on GitHub (Oct 31, 2026):
Rechecked the earlier byte-edited project against current REA main
60149282with real Ghidra 12.1.4. At the same mapped address/file offset, the selected saved project still returned edited instruction bytes andreturn 0x2a; a fresh import returned the original bytes andreturn 0, matching an independent file read. Both source-file digest views survived Evidence export and target-free import exactly. Target and source project hashes remained unchanged during the checks.The project-discovery experiment in #8344 independently found the original executable SHA-256 in the edited program's saved
DomainFilemetadata. This is an import-identity observation, not proof that program memory is unchanged. The previous relocation control remains relevant: different source digests alone do not identify the cause. No target executed; this is one owned example, not broad format or provider coverage.@morluto commented on GitHub (Oct 31, 2026):
A bounded file-to-memory comparison needs to stop at a mapping boundary, even when the input file has more bytes. I exercised a new self-built, unexecuted x86-64 ELF through real REA main
60149282/ Ghidra 12.1.4 MCP. Independentllvm-readelfidentified an eight-byte.datafile mapping immediately followed in virtual memory by eight-byte.bss(NOBITS); Ghidra placed them at0x103360..0x103367and0x103368..0x10336f, respectively.read_bytes({address:"0x103364",length:8})returned only55667788,returned_bytes: 4,complete: false. The start mapped to file offset 868, but the next virtual address0x103368produced an explicitfile_mapping: uninitialized;0x103370was explicitlyunmapped. A direct file read at offsets 868–875 gives556677884c696e6b: the latter four bytes are from the unrelated on-disk.commentsection, not a source for the BSS. The.dataeight-byte read was complete and matched its source bytes.This is a working negative control for the proposed comparison, not a new REA bug: the public result retains partial-read status and mapping failure. A future bounded comparison should report the four comparable source bytes and the remainder as unavailable rather than padding, inferring a contiguous file range from the first offset, or labeling the remainder changed. Compact MCP output schemas were fetched before the calls; target SHA-256
44bacc0c5a2321118915bfcf375fe9709dcc9423f99904336bbb3d3d114b55eewas unchanged. This single fresh-import ELF does not exercise edited projects, overlays, relocation effects or other formats.Selected-project control: I separately imported the same ELF into a new owned Ghidra project with
-noanalysis, then selected that program through REA's read-only project mode. The complete.dataread, partial boundary read, uninitialized mapping refusal and unmapped-address refusal produced the same normalized results as the fresh import. The target and all regular-file hashes under the selected source project were unchanged during the REA calls. This adds one unedited selected-project control; it does not replace the earlier saved-byte-edit and relocation controls.@morluto commented on GitHub (Oct 31, 2026):
I exercised a local orchestration experiment, not a new REA tool, against six real Ghidra 12.1.4 cases on main
d54a4661d. For each selected range it readinspect_native_load_image,read_bytesandaddress_to_file_offset, checked the observed source-file digest against the explicitly selected target, and compared only bytes within the one matching initialized provider mapping. Original file bytes were read separately by the audit script; the comparison itself was not REA Evidence..datacrossing into.bss: 4/8 bytes compared and equal; the remaining four stayed unavailable. A wholly uninitialized range and a separately addressed initialized overlay both had no usable source-file mapping, even though the overlay's memory read succeeded.APPLIEDrelocation at that address with the same original and memory bytes. Thus “different” cannot by itself mean analyst patch. An empty relocation list in another case is not proof that all causes are excluded.For the edited and relocation cases, the three individual REA Evidence records (
inspect_native_load_image,read_bytes,address_to_file_offset) survived export and exact import into a fresh targetless session. The audit script's independently read source bytes and comparison were not captured as a composed REA record, so those bundles alone are not a complete offline comparison. The provider's load-image status wasunsupportedfor independent Mach-O/ELF verification; matching its mappings and digest does not upgrade that status to verified.This supports testing a bounded, task-level comparison, while preserving the individual source observations, source-byte identity, address-space qualifier, partial/unavailable portions, and unknown difference cause. It is not production-ready: the experiment stopped at one provider mapping, did not handle mapping overlaps or ranges spanning multiple mappings, and did not establish a stable atomic source snapshot under concurrent changes. My initial harness also failed on a qualified overlay address; the corrected rerun kept the address space separate. All tested target and selected-project file hashes stayed unchanged during the main comparison runs; no target was executed.
@morluto commented on GitHub (Oct 31, 2026):
Additional real MCP search and handoff check on the same owned fixture and byte-edited selected project:
0x1000002d8..0x1000002df,search_bytesfor40 05 80 52 c0 03 5f d6found the address in the selected project and reported file offset 728; the original-byte pattern returned no match in that range.read_bytesreturned the edited pattern.00 00 80 52 c0 03 5f d6, did not find the edited pattern in that range, and reported the same address and offset. Direct reading of the original file at offset 728 gave the original pattern.This is expected analyzed-memory behavior, not a search correctness bug. It illustrates why a hit's
file_offsetidentifies a mapping and does not assert the original file has the matching bytes. This project was deliberately edited by a separate headless script before the test; REA's sessions left both the target and every selected-project regular-file digest unchanged. No target was executed. Scope: one ARM64 Mach-O, Ghidra 12.1.4, macOS arm64 and REAd54a4661da20382ad8456564a08569fdd24131c5; other mappings and causes remain open as described above.A separate unexecuted ARM64 Mach-O with a 16-byte
__bsssupplies an uninitialized control. At the BSS address,read_bytesreturned zero bytes andcomplete: false;search_bytesreturned no matches, explicitly listed all 16 bytes asskipped_uninitialized, and setcomplete: truefor the initialized search it performed.address_to_file_offsetrefused the BSS address with an explicit uninitialized-memory reason; code bytes and their file offset were available as a positive control. The fixture SHA-256156085b7dd9ca5e7335011e663a64665a174aafbed0d61eb1c3afc889aad308awas unchanged. Thus the comparison design must carry skipped-range coverage rather than treating the empty match list as an observation of zero bytes.