[PR #6392] fix(ghidra): thin the selected universal Mach-O slice into the private import #7332

Open
opened 2026-11-01 21:47:07 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6392
Author: @cpruijsen
Created: 10/22/2026
Status: 🔄 Open

Base: main ← Head: fix/ghidra-thin-universal-slice


📝 Commits (4)

  • 719823f fix(ghidra): thin the selected universal Mach-O slice into the private import
  • 2bf9696 fix(ghidra): report universal-file offsets for a thinned slice
  • 6d1ce95 Merge branch 'main' into fix/ghidra-thin-universal-slice
  • 124f354 test(ghidra): thin a slice spanning several 64 KiB read chunks

📊 Changes

9 files changed (+336 additions, -28 deletions)

View changed files

📝 bridge/ghidra/ReaGhidraBridge.java (+27 -4)
📝 src/ghidra/GhidraClient.ts (+13 -2)
📝 src/ghidra/GhidraClientTypes.ts (+3 -0)
📝 src/ghidra/GhidraLauncher.ts (+5 -0)
📝 src/ghidra/GhidraProvider.behavior.test.ts (+22 -15)
📝 src/ghidra/GhidraProvider.ts (+6 -2)
📝 src/ghidra/GhidraProviderClient.ts (+8 -0)
📝 src/ghidra/GhidraTargetSnapshot.ts (+41 -5)
➕ src/ghidra/GhidraTargetSnapshotThinning.test.ts (+211 -0)

📄 Description

Summary

The Ghidra provider refused every universal (FAT) Mach-O since 6582b2433, with "analyze a thinned Mach-O slice instead". REA offers no tool that thins a slice, so there was no way to get Ghidra analysis of a typical macOS app through REA. Most macOS apps ship as universal binaries.

This change makes Ghidra import only the selected slice, with the approach #2201 merged for IDA: the private, per-session copy that Ghidra already imports now holds just the admitted slice. Evidence still names the original universal file and its digest, and every file offset in Ghidra results and file-offset: selectors is an offset in that universal file.

No issue is linked. I hit the refusal analyzing a universal macOS app (Elgato Control Center, x86_64 arm64) through REA with Ghidra 12.1.4.

Problem and expected behavior

  • Base: rea search "<universal Mach-O>" elgato --provider ghidra fails with "Ghidra v1 cannot enforce the selected architecture when importing a universal Mach-O; analyze a thinned Mach-O slice instead."
  • Expected: Ghidra analyzes the slice that admission selected (BinaryTarget.universalSlice), as the IDA provider does, and its file coordinates refer to the same universal file that Evidence names.
flowchart LR
  A[universal Mach-O] --> B[admission selects slice<br/>BinaryTarget.universalSlice]
  B --> C[private per-session copy<br/>slice bytes only]
  C --> D[Ghidra import + analysis]
  A -. digest checked .-> C
  D --> E[bridge: slice-local offset + slice offset<br/>= universal-file offset]
  F[file-offset: selector<br/>universal-file offset] --> G[bridge: minus slice offset] --> D
  E --> H[Evidence: universal path + digest]

Change and scope

  • GhidraTargetSnapshot.ts: the existing private copy step copies only [slice.offset, slice.offset + slice.size) for universal targets. The whole-file digest is still checked against the admitted target, and the copy fails closed if the source changed or the slice leaves the file.
  • GhidraClient.ts, GhidraClientTypes.ts: the bridge handshake expects the digest of the slice that Ghidra imports.
  • GhidraProvider.ts: universal targets with an admitted slice are accepted. A universal target whose selected architecture is not one of its slices, and a non-universal Mach-O that declares more than one architecture, are still refused.
  • GhidraLauncher.ts, bridge/ghidra/ReaGhidraBridge.java: the session descriptor carries the slice's universal-file offset (target_file_offset_base, optional, validated, absent for every other target). The bridge adds it to every target file offset it reports (address_to_file_offset, byte-search matches, decompile selections) and subtracts it from file-offset: selectors before it maps them, so an offset outside the slice does not resolve.
  • GhidraProviderClient.ts: a session limitation for universal targets that names the imported slice, its offset and size, and the coordinate rule.

What I chose and the alternative (from baron's review): I normalized the coordinates once, in the bridge, which is where Ghidra's FileBytes coordinates become REA JSON. The other option was to make the slice its own Evidence subject with a container relation. I did not take it because REA already reports universal-file offsets for universal targets elsewhere (for example MachoSliceSelection.ts), so one subject with one coordinate system keeps Evidence uniform. inspect_native_load_image mappings stay raw: each file_offset there is relative to the source file named by source_file_index, and that entry carries the slice digest, so the raw provider fact and its identity stay together.

Not included: choosing a slice other than the one admission selects (it follows the host CPU, as for IDA), and the startup deadline default.

Follow the contribution scope.
Unresolved scope, authority, side effects, or required verification block
integration. Link the required maintainer scope decision for new providers or
capabilities involving target execution, injection, mutation, or protection
recovery; ordinary fixes within existing boundaries may state "not applicable".
Draft PRs may leave the scope decision pending and the readiness and merge checklists
unchecked. Maintainers can help resolve open questions.

Scope decision: not applicable. This is an ordinary fix inside the existing Ghidra provider boundary.

Contract and boundary impact

  • Semantic owner and earliest changed stage: Ghidra target snapshot (private import copy), after admission; Ghidra bridge coordinate output and selector input.
  • Contribution scope fit and linked maintainer decision: not applicable (follows #2201 for IDA and lifts the refusal from 6582b2433).
  • CLI and MCP/tool-catalog contract: universal Mach-O targets now open with --provider ghidra instead of failing. No tool catalog changes.
  • Provider, bridge, target-format, or platform compatibility: Ghidra on macOS and Linux. The bridge descriptor gains one optional key. On Windows the provider returns early to the Windows P0 support check, which refuses every format except PE and raw 6502, so the slice path is not reachable there and Windows behavior does not change.
  • Evidence, artifact, provenance, or reconstruction contract: Evidence keeps the original universal path and container digest. File offsets in results and file-offset: selectors are universal-file offsets. Native load-image mappings stay relative to the source file they name.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): the private copy is REA-owned, read-only to the target, and removed with the session. The original file is only read.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact (CI generates and validates ignored catalogs and skills/; edit authored instructions in .agents/skills/reverse-engineer-anything/ and never commit build outputs): none.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: src/ghidra/GhidraTargetSnapshotThinning.test.ts (new): the private copy equals the selected slice for universal fixtures, the launch session carries the slice offset as targetFileOffsetBase, and a changed source or an out-of-range slice fails closed. GhidraProvider.behavior.test.ts: the old refusal test now checks that universal targets with an admitted slice are accepted and that the remaining ambiguous cases are refused. The existing snapshot boundary suite (tests/boundary/providers/ghidra/ghidraTargetSnapshot.test.ts) covers the unchanged non-universal copy.

  • Base reproduction or other evidence: with every source change reverted to main, 6 of the 172 tests in the two changed test files fail. With the change, 172 of 172 pass.

  • Real Ghidra 12.1.4 (macOS arm64, JDK 21), coordinate round trip on a small universal binary (clang -arch x86_64 -arch arm64; x86_64 slice at 4096, arm64 slice at 16384, size 33456). The marker string is at universal offset 17576, and _marker_fn (0x100000460) at universal offset 17504:

    Call Before this round (e13f16a2f) This PR
    address-to-file-offset <bin> 0x1000004a8 file_offset 1192 (slice-local) file_offset 17576
    address-to-file-offset <bin> 0x100000460 file_offset 1120 (slice-local) file_offset 17504
    decompile <bin> --file-offset 0x4460 fails: maps to 0x100004460, no procedure there _marker_fn, location file-offset 17504
    decompile <bin> --file-offset 0x460 (inside the x86_64 region) _marker_fn not_found: "File offset 0x460 has no loaded source mapping for the admitted target file"
  • Real Ghidra 12.1.4 on the universal Elgato Control Center binary: base gives the refusal above. With the thinning, Ghidra imported the private arm64 slice (AARCH64:LE:64:AppleSilicon:swift), and search ... elgato --provider ghidra returned 331 matches, for example https://lspro.elgato.com at 0x10065b7e0. The private copy was byte-identical to lipo -thin arm64. The app's digest in the Evidence equals the digest measured before the run.

  • User-visible CLI/MCP output (if applicable):

    {"address": "0x100000460", "file_offset": 17504}
    

    address-to-file-offset on the arm64 slice of the test binary above (normalized_result). Session limitation: "Ghidra imported only the arm64 slice of this universal Mach-O, copied from container offset 16384 (33456 bytes). File offsets in results and file-offset selectors are universal-file offsets; native load-image mappings stay relative to the imported slice source file."

  • Remaining proof gaps: Ghidra's own analysis of the 9.7 MB Elgato slice took 373 s, above the 330 s default startup deadline, so that run used REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000. One earlier run died in a JVM SIGBUS inside Ghidra's analyzer (SymbolicPropogator.getInstructionAt) before the REA bridge started, so it is upstream of this change.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • The two changed Ghidra test files: 172 passed. With every source change reverted to main: 6 failed.
  • Every Ghidra test file (src/ghidra/*.test.ts, tests/boundary/providers/ghidra/*.test.ts, tests/boundary/filesystem/ghidra*.test.ts): 48 files, 1143 passed, 3 skipped.
  • npm run check:fast: passed.
  • npm run verify:test-discovery: passed (2341 files).
  • npm run docs:check: passed.
  • Real Ghidra runs above: results as listed.
  • All checks ran on the branch rebased onto main at 4a041ae75, macOS arm64, Node 26.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Targets that were refused now open. Non-universal targets send no new descriptor key, so their coordinates do not change.
  • Real Hopper/Ghidra, browser, or OS coverage: real Ghidra 12.1.4 on macOS arm64, as above.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: the change only narrows what is copied into REA's existing private import directory and adds one validated integer to the private session descriptor.

Author readiness

Human owner: @cpruijsen

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6392 **Author:** [@cpruijsen](https://github.com/cpruijsen) **Created:** 10/22/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/ghidra-thin-universal-slice` --- ### 📝 Commits (4) - [`719823f`](https://github.com/morluto/rea/commit/719823f792c1fa1b7b039a650bf555531bb68a9e) fix(ghidra): thin the selected universal Mach-O slice into the private import - [`2bf9696`](https://github.com/morluto/rea/commit/2bf9696603bc9dcfadbc7e5aec5fe7be538f861b) fix(ghidra): report universal-file offsets for a thinned slice - [`6d1ce95`](https://github.com/morluto/rea/commit/6d1ce95af0d56d3ebf341465fb0e742f55db7a43) Merge branch 'main' into fix/ghidra-thin-universal-slice - [`124f354`](https://github.com/morluto/rea/commit/124f354a9a010c20329c3150437aafdff1bbc31f) test(ghidra): thin a slice spanning several 64 KiB read chunks ### 📊 Changes **9 files changed** (+336 additions, -28 deletions) <details> <summary>View changed files</summary> 📝 `bridge/ghidra/ReaGhidraBridge.java` (+27 -4) 📝 `src/ghidra/GhidraClient.ts` (+13 -2) 📝 `src/ghidra/GhidraClientTypes.ts` (+3 -0) 📝 `src/ghidra/GhidraLauncher.ts` (+5 -0) 📝 `src/ghidra/GhidraProvider.behavior.test.ts` (+22 -15) 📝 `src/ghidra/GhidraProvider.ts` (+6 -2) 📝 `src/ghidra/GhidraProviderClient.ts` (+8 -0) 📝 `src/ghidra/GhidraTargetSnapshot.ts` (+41 -5) ➕ `src/ghidra/GhidraTargetSnapshotThinning.test.ts` (+211 -0) </details> ### 📄 Description ## Summary The Ghidra provider refused every universal (FAT) Mach-O since 6582b2433, with "analyze a thinned Mach-O slice instead". REA offers no tool that thins a slice, so there was no way to get Ghidra analysis of a typical macOS app through REA. Most macOS apps ship as universal binaries. This change makes Ghidra import only the selected slice, with the approach #2201 merged for IDA: the private, per-session copy that Ghidra already imports now holds just the admitted slice. Evidence still names the original universal file and its digest, and every file offset in Ghidra results and `file-offset:` selectors is an offset in that universal file. No issue is linked. I hit the refusal analyzing a universal macOS app (Elgato Control Center, `x86_64 arm64`) through REA with Ghidra 12.1.4. ## Problem and expected behavior - Base: `rea search "<universal Mach-O>" elgato --provider ghidra` fails with "Ghidra v1 cannot enforce the selected architecture when importing a universal Mach-O; analyze a thinned Mach-O slice instead." - Expected: Ghidra analyzes the slice that admission selected (`BinaryTarget.universalSlice`), as the IDA provider does, and its file coordinates refer to the same universal file that Evidence names. ```mermaid flowchart LR A[universal Mach-O] --> B[admission selects slice<br/>BinaryTarget.universalSlice] B --> C[private per-session copy<br/>slice bytes only] C --> D[Ghidra import + analysis] A -. digest checked .-> C D --> E[bridge: slice-local offset + slice offset<br/>= universal-file offset] F[file-offset: selector<br/>universal-file offset] --> G[bridge: minus slice offset] --> D E --> H[Evidence: universal path + digest] ``` ## Change and scope - `GhidraTargetSnapshot.ts`: the existing private copy step copies only `[slice.offset, slice.offset + slice.size)` for universal targets. The whole-file digest is still checked against the admitted target, and the copy fails closed if the source changed or the slice leaves the file. - `GhidraClient.ts`, `GhidraClientTypes.ts`: the bridge handshake expects the digest of the slice that Ghidra imports. - `GhidraProvider.ts`: universal targets with an admitted slice are accepted. A universal target whose selected architecture is not one of its slices, and a non-universal Mach-O that declares more than one architecture, are still refused. - `GhidraLauncher.ts`, `bridge/ghidra/ReaGhidraBridge.java`: the session descriptor carries the slice's universal-file offset (`target_file_offset_base`, optional, validated, absent for every other target). The bridge adds it to every target file offset it reports (`address_to_file_offset`, byte-search matches, decompile selections) and subtracts it from `file-offset:` selectors before it maps them, so an offset outside the slice does not resolve. - `GhidraProviderClient.ts`: a session limitation for universal targets that names the imported slice, its offset and size, and the coordinate rule. What I chose and the alternative (from baron's review): I normalized the coordinates once, in the bridge, which is where Ghidra's `FileBytes` coordinates become REA JSON. The other option was to make the slice its own Evidence subject with a container relation. I did not take it because REA already reports universal-file offsets for universal targets elsewhere (for example `MachoSliceSelection.ts`), so one subject with one coordinate system keeps Evidence uniform. `inspect_native_load_image` mappings stay raw: each `file_offset` there is relative to the source file named by `source_file_index`, and that entry carries the slice digest, so the raw provider fact and its identity stay together. Not included: choosing a slice other than the one admission selects (it follows the host CPU, as for IDA), and the startup deadline default. Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). Unresolved scope, authority, side effects, or required verification block integration. Link the required maintainer scope decision for new providers or capabilities involving target execution, injection, mutation, or protection recovery; ordinary fixes within existing boundaries may state "not applicable". Draft PRs may leave the scope decision pending and the readiness and merge checklists unchecked. Maintainers can help resolve open questions. Scope decision: not applicable. This is an ordinary fix inside the existing Ghidra provider boundary. ## Contract and boundary impact - Semantic owner and earliest changed stage: Ghidra target snapshot (private import copy), after admission; Ghidra bridge coordinate output and selector input. - Contribution scope fit and linked maintainer decision: not applicable (follows #2201 for IDA and lifts the refusal from 6582b2433). - CLI and MCP/tool-catalog contract: universal Mach-O targets now open with `--provider ghidra` instead of failing. No tool catalog changes. - Provider, bridge, target-format, or platform compatibility: Ghidra on macOS and Linux. The bridge descriptor gains one optional key. On Windows the provider returns early to the Windows P0 support check, which refuses every format except PE and raw 6502, so the slice path is not reachable there and Windows behavior does not change. - Evidence, artifact, provenance, or reconstruction contract: Evidence keeps the original universal path and container digest. File offsets in results and `file-offset:` selectors are universal-file offsets. Native load-image mappings stay relative to the source file they name. - Target selection, authority, upstream behavior, and cleanup details (if applicable): the private copy is REA-owned, read-only to the target, and removed with the session. The original file is only read. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs): none. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: `src/ghidra/GhidraTargetSnapshotThinning.test.ts` (new): the private copy equals the selected slice for universal fixtures, the launch session carries the slice offset as `targetFileOffsetBase`, and a changed source or an out-of-range slice fails closed. `GhidraProvider.behavior.test.ts`: the old refusal test now checks that universal targets with an admitted slice are accepted and that the remaining ambiguous cases are refused. The existing snapshot boundary suite (`tests/boundary/providers/ghidra/ghidraTargetSnapshot.test.ts`) covers the unchanged non-universal copy. - Base reproduction or other evidence: with every source change reverted to `main`, 6 of the 172 tests in the two changed test files fail. With the change, 172 of 172 pass. - Real Ghidra 12.1.4 (macOS arm64, JDK 21), coordinate round trip on a small universal binary (`clang -arch x86_64 -arch arm64`; x86_64 slice at 4096, arm64 slice at 16384, size 33456). The marker string is at universal offset 17576, and `_marker_fn` (`0x100000460`) at universal offset 17504: | Call | Before this round (e13f16a2f) | This PR | | --- | --- | --- | | `address-to-file-offset <bin> 0x1000004a8` | `file_offset` 1192 (slice-local) | `file_offset` 17576 | | `address-to-file-offset <bin> 0x100000460` | `file_offset` 1120 (slice-local) | `file_offset` 17504 | | `decompile <bin> --file-offset 0x4460` | fails: maps to `0x100004460`, no procedure there | `_marker_fn`, location `file-offset` 17504 | | `decompile <bin> --file-offset 0x460` (inside the x86_64 region) | `_marker_fn` | `not_found`: "File offset 0x460 has no loaded source mapping for the admitted target file" | - Real Ghidra 12.1.4 on the universal Elgato Control Center binary: base gives the refusal above. With the thinning, Ghidra imported the private arm64 slice (`AARCH64:LE:64:AppleSilicon:swift`), and `search ... elgato --provider ghidra` returned 331 matches, for example `https://lspro.elgato.com` at `0x10065b7e0`. The private copy was byte-identical to `lipo -thin arm64`. The app's digest in the Evidence equals the digest measured before the run. - User-visible CLI/MCP output (if applicable): ```json {"address": "0x100000460", "file_offset": 17504} ``` `address-to-file-offset` on the arm64 slice of the test binary above (`normalized_result`). Session limitation: "Ghidra imported only the arm64 slice of this universal Mach-O, copied from container offset 16384 (33456 bytes). File offsets in results and file-offset selectors are universal-file offsets; native load-image mappings stay relative to the imported slice source file." - Remaining proof gaps: Ghidra's own analysis of the 9.7 MB Elgato slice took 373 s, above the 330 s default startup deadline, so that run used `REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000`. One earlier run died in a JVM SIGBUS inside Ghidra's analyzer (`SymbolicPropogator.getInstructionAt`) before the REA bridge started, so it is upstream of this change. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - The two changed Ghidra test files: 172 passed. With every source change reverted to `main`: 6 failed. - Every Ghidra test file (`src/ghidra/*.test.ts`, `tests/boundary/providers/ghidra/*.test.ts`, `tests/boundary/filesystem/ghidra*.test.ts`): 48 files, 1143 passed, 3 skipped. - `npm run check:fast`: passed. - `npm run verify:test-discovery`: passed (2341 files). - `npm run docs:check`: passed. - Real Ghidra runs above: results as listed. - All checks ran on the branch rebased onto `main` at 4a041ae75, macOS arm64, Node 26. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Targets that were refused now open. Non-universal targets send no new descriptor key, so their coordinates do not change. - Real Hopper/Ghidra, browser, or OS coverage: real Ghidra 12.1.4 on macOS arm64, as above. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the change only narrows what is copied into REA's existing private import directory and adds one validated integer to the private session descriptor. ## Author readiness Human owner: @cpruijsen - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7332
No description provided.