mirror of
https://github.com/morluto/rea.git
synced 2026-11-05 06:45:22 +00:00
[GH-ISSUE #4246] [Design] Caller-selected Ghidra language and base address for raw-binary imports #1669
Open
opened 2026-10-16 21:08:49 +00:00 by bsakaguchi
·
6 comments
No Branch/Tag specified
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
Mirrored from GitHub Pull Request
question
roadmap
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
morluto/rea#1669
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4246
Before submitting
Decision needed
Should
open_binary format=raw-binaryaccept a caller-selected Ghidra language ID (and optional compiler spec), validated against the installed language catalog and reported as unverified coverage, alongside the current per-family lanes? Separately, should it accept a load base address that differs from the entry point?Context and evidence
Observed in source at the REA baseline below: raw-binary admits only
x86,x86_64,arm,arm64, and6502(src/domain/rawBinary.ts:11-17), each mapped to one fixed little-endian Ghidra language (src/ghidra/GhidraRawBinaryLanguage.ts:16-51).REA_GHIDRA_LANGUAGE_IDis refused for raw-binary with "REA_GHIDRA_LANGUAGE_ID cannot override the admitted raw-binary import." (src/ghidra/GhidraAnalysisProfile.ts:110-118).docs/raw-binary.mdlists big-endian, Thumb-only, Cortex-M, and other processor variants as unsupported, and the singleentry_pointis both the load address of file offset 0 and the entry (src/ghidra/GhidraProviderClient.ts:601-606).Source-derived reasoning: headerless images for MIPS (BE and LE), big-endian ARM, PowerPC, Xtensa, SuperH, and Cortex-M are common in embedded firmware, and Ghidra already ships languages for them. Each family currently needs its own lane (#3532 for SH-4, open PR #4018 for big-endian m68k). A bootloader or kernel image is also often linked at a base address that is not its entry point; with base fixed to entry, absolute references in such an image resolve to the wrong addresses.
Precedent: the PowerPC ELF lane already lets a caller-selected
REA_GHIDRA_LANGUAGE_IDreplace the lane default, admitted against the installed language catalog (docs/ghidra-ppc.md:28-34; PR #1747 exempted PowerPC from the override restriction that MIPS and RISC-V keep).AGENTS.mdrequires unverified coverage to be reported explicitly, which a generic language selection would need for every language without a verified lane.Unknown: whether
ReaGhidraPrepareRaw.javaand load-image verification (src/ghidra/GhidraLoadImageValues.ts:343-380) can obtain the default address space and address width from the installed language catalog instead of the fixed mapping.Expected interaction with targets
Read existing files or captured data
Boundary details (optional)
The change concerns how a caller-selected local file is imported by a caller-configured Ghidra installation; it adds no execution of the target, no network, and no new provider. The evidence invariant to keep: an unverified language must never be reported with the coverage claims of a verified lane, and a refused or unknown language must fail before analysis rather than fall back to a different one.
Options and tradeoffs
language_id(andcompiler_spec_id) for raw-binary, validated against the installed.ldefscatalog, with Evidence and limitations marking the language as unverified unless it matches a verified lane. This matches the PowerPC precedent, at the cost of weaker guarantees for unverified languages.base_address: keeps file offset 0 mapped atbase_addressand seeds the entry atentry_point, with validation that the entry lies inside the mapped image. This changes the persisted analysis profile and snapshot identity.Verification available so far
I can suggest a sample or manual check
Success criteria and open questions
MIPS:BE:32:defaulton a self-compiledobjcopy -O binaryimage), the import succeeds, procedures and pseudocode are readable, and every result carries an explicit unverified-language limitation.REA_GHIDRA_LANGUAGE_IDfor raw-binary, or only an explicitopen_binaryinput? How should snapshot reuse key on the selected language and base?References
mainat0cc56493be(package 6.5.0):docs/raw-binary.md,docs/ghidra-ppc.md,docs/testing.md(raw lanes),src/ghidra/GhidraLauncher.ts:649-681,bridge/ghidra/ReaGhidraPrepareRaw.java.@onionmixer commented on GitHub (Oct 17, 2026):
Thank you for writing this up. I hope a concrete use case is helpful for the second question (a load base that differs from the entry); please disregard it if it is out of scope.
Use case (observed). I have been looking at a 68000 main-CPU program from a Mega-CD (Sega CD) game with REA's raw m68k lane. The image is loaded at
0x200000, and its first 256 bytes are a 68000 vector table; the reset vector points to0x200200. With a singleentry_point, I can only map the file correctly by usingentry_point=0x200000.ReaGhidraPrepareRaw.javathen callsdisassemble()andcreateFunction(..., "entry")at0x200000, so the vector table is decoded as code before auto-analysis and before anyREA_GHIDRA_SEED_FILEseeds are applied. In my case, that early flow reached a large part of the program.Why it mattered here (observed, with a caveat). This program's trap handlers read inline parameter words placed after the
trapinstruction. The early flow from the entry decoded those words as instructions. Because of that, seeds alone could not correct those places afterwards: code seeds refuse addresses that are already inside an instruction. I worked around it locally with a different change, which I describe separately below. I did not isolate how much of the difference comes from the entry seeding alone, so I would not want to overstate it.What a separate base would allow. If
open_binaryaccepted a load base (0x200000) separately from the entry (0x200200), this image could be imported without decoding the vector table as code. The vector table could then be left for seeds or labels. The firmware and bootloader images mentioned in this issue probably have the same layout.Linux run of the m68k lane (observed). In case it is useful for the "Linux has not been exercised" note in
docs/raw-binary.md: on Ubuntu 22.04 x64 with Ghidra 12.1 (not the 12.1.4 verified build) and OpenJDK 21.0.12,node scripts/verify-real-ghidra-m68k.mjsreported"ok":truefor every check. I ran it onrea-agents-6.5.0with #4018 cherry-picked, not on current main.I cannot share the program itself because it is commercial game code. A small synthetic image with a vector table and an inline-operand trap reproduces the same pattern, and I would be glad to prepare one if that would help.
Prepared with help from an AI coding assistant; I have checked the statements above against my local runs.
@Entrpi commented on GitHub (Oct 27, 2026):
To unblock Cortex-M flash dumps (#5744), I'm preparing an implementation of a minimal answer to this issue, for review:
base_address, separate fromentry_point. Targets that use none of the new inputs keep today's behaviour, identity and load image exactly.architecture_variant, naming the M profile:armv6-m,armv7-mandarmv7e-m→ARM:LE:32:Cortex;armv8-m.baseandarmv8-m.main→ARM:LE:32:v8-m.Cortexlacks v8-M encodings such asMSR MSPLIM, so a singlecortex-mvalue (as in #4927) can't map to one language.entry_pointoptional when a base is given. Under these variants, Reset is then read from the vector table, at the base or atvector_table_addressfor prefixed images such as RP2040 boot2 and MCUboot. An explicit odd entry is refused, and the error gives the even address.A new load-image shape for new targets only. Targets that use the new inputs get a minimal, versioned
segmentsload image. That's a new saved-Evidence shape, so it needs your compatibility decision.Language passthrough (option B) could follow later as an explicitly unverified override. I'll credit #4927's author and reuse its sound parts.
Please confirm or redirect any of this; it won't merge until a maintainer does.
@Entrpi commented on GitHub (Oct 27, 2026):
The implementation of the proposal above is ready for design review on my fork: compare with
main. I'm at this repository's open pull request limit, so it will become a draft PR when one of my slots frees.The decision is a single commit,
56f27b3de: types, schemas, the owner table, docs and table-driven tests. The commits after it implement it. Beyond the outline above:base_addressis refused withoutarchitecture_variant, because no verified lane covers a separate base for other families yet.raw_loadin the analysis profile and snapshot target, and a minimalsegmentsload image. Loads without a variant keep today's profile, identity and load image.-loader-applyLabels false).ARMCortex.pspeclabels0x0–0x40and marks every handler slot an entry, which would cover the image at base0x0.On real Ghidra 12.1.4 (Linux), a new
verify:ghidra:cortex-mlane passes all five variants, including a prefixed image read throughvector_table_addressand v8-M.main'sMSPLIM. The existing raw, m68k, 6502, COM, DOL and bridge-source lanes also pass.Please confirm or redirect; it won't merge until a maintainer does.
Update (28 Oct): the branch now also covers
armv8.1-m.main(Cortex-M55/M85), decoded withARM:LE:32:v8-m, and the lane passes all six variants. Ghidra 12.1.4 gets the targets of the ARMv8.1-M low-overhead loop branches (wls,le) wrong, so code after such a loop is not disassembled; results for that variant carry this limitation.@Reid-n0rc commented on GitHub (Oct 27, 2026):
A use case for Option B that the per-family lanes and the M-profile
architecture_variantproposal would not cover: a language that exists only as a community processor module.Use case (observed). Headerless Renesas RX (RXv2) firmware for an RX651-class MCU: a flat ROM image mapped at
0xFFF80000, with the reset vector at the fixed vector table near the top of the address space. Ghidra 12.1.4 ships no RX language. A community SLEIGH module (language IDrx62t:LE:32:default, compiler specdefault) decodes this image well when its files are installed under the Ghidra installation's ownGhidra/Processors/<module>/data/languages/, so the language is present in the installation catalog that REA reads. With plainanalyzeHeadless -processor rx62t:LE:32:default -loader BinaryLoader -loader-baseAddr 0xFFF80000, auto-analysis and decompilation work. Through REA:format=raw-binarycannot select it:architectureis limited to the fixed families, andREA_GHIDRA_LANGUAGE_IDis refused for raw-binary (src/ghidra/GhidraAnalysisProfile.ts).e_machine=EM_RX, 173) is rejected earlier with "unsupported ELF architecture" (elfArchitecturemaps only x86, x86_64, mips, powerpc, arm, hexagon, arm64, riscv).REA_GHIDRA_PROJECT_PATHcannot reuse an analyzed project, because project reuse admits only ELF, PE and Mach-O.So Option B, validated against the installed
.ldefscatalog and reported as unverified coverage, would be the only route for this class of target. It would also cover other community-only languages without a new lane each.A second gap Option B alone doesn't close (question for the design). For MCU firmware, the language is necessary but not sufficient for readable output. The decompile becomes useful only after the analyst adds memory blocks that are not in the file: on-chip RAM, peripheral I/O registers, data flash, the C runtime's
.datacopy-down from ROM into RAM, and the relocatable interrupt vector table as function seeds. Without RAM and I/O blocks, every global and register access shows as an unmapped address.REA_GHIDRA_SEED_FILEcovers the function seeds. Is a caller-supplied memory map for raw-binary (named uninitialized or initialized blocks, plus optional byte-copy ranges) in scope, perhaps as a separate decision from the language selection? Or would raw-binary project reuse be the preferred route, since snapshot identity would then follow the project content as it already does for ELF/PE/Mach-O?Answers to the open questions, from this use case:
open_binaryinput reads better thanREA_GHIDRA_LANGUAGE_ID: the language is a property of the target, and one server may analyze several headerless images with different CPUs..ldefs/.slawould be safer than the ID alone.I can run a self-built test image through a prototype if that helps verification.
@Entrpi commented on GitHub (Oct 28, 2026):
Thanks, this is exactly the case for the language input (option B). I'd make it an explicit
open_binaryinput, checked against the installed catalog and reported as unverified. Identity should cover a digest of the resolved.ldefs/.sla, as you say, not just the id.The memory map is the other half of this design: a caller-declared load plan with named blocks, copy-down ranges and seeds. It changes load identity, so it's part of the refactor question in #7478.
I'll build the language input on my fork and ping you here; a self-built RX image would be a great check.
@Entrpi commented on GitHub (Oct 28, 2026):
D1 is now open as draft #7951, and its decision record there is the current version. A three-model review changed four points since the outline above:
architecture_variantrequiresbase_address, because mapping a dump at its entry would put the vector table at Reset;layout: "segments", and a refused selection names its input indetails.field.Please confirm or redirect on #7951.
tools/listmetadata, and document that DeepSeek Harness does not lazy-load #4208