morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #3072] [Bug] Hopper provider on macOS: FAT Mach-O slice selection loads `arm64e.x1`, collapsing analysis of system binaries #1077

New issue

Closed

opened 2026-10-15 18:30:52 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-15 18:30:52 +00:00

Owner

Originally created by @XYishion on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3072

Summary

On Apple-silicon macOS, system binaries (/bin/*, /usr/bin/*) are FAT Mach-O files with three
slices: x86_64, arm64e (cpusubtype 2) and arm64e.x1 (cpusubtype 12).

The Hopper provider is launched as hopper -l FAT --aarch64 -l Mach-O --analysis ..., and the
bridge ends up loading the arm64e.x1 slice. That slice's entry point begins with pacibsppc
(0xdac1a7fe) — an arm64e.x1-only encoding that is an invalid instruction under plain arm64
(whereas the same offset in the arm64e slice holds pacibsp / 0xd503237f, a decodable
HINT-class NOP). Hopper's recursive descent therefore aborts at the first instruction of every
function, and almost no code procedures are created.

The session then fails closed with No procedure exists at the requested address, which is correct
behaviour on REA's side — but the underlying slice choice makes the target effectively unanalyzable.

Environment

macOS: Darwin 27.0.0 (Apple silicon)
rea-agents 6.4.0 (via npx and via pinned rea.mjs)
Hopper Disassembler 6.6.0 (demo), launcher sha256
dad6f886261fad0d4cdd01aee6fcf57439c60d7b5c54e62fc97f0180116690e0
Reproduced through both the MCP session and the headless CLI (rea inspect / rea function)

Reproduction

$ lipo -archs /bin/echo
x86_64 arm64e arm64e.x1

$ dyld_info -function_starts /bin/echo      # ground truth
# 3 real functions: 0x1000004F0, 0x1000007C4, 0x1000007F8

Open /bin/echo with REA (MCP open_binary + binary_overview/list_procedures, or
rea inspect /bin/echo):

binary_overview → procedure_count: 5, all of them import stubs; no sub_* procedures
analyze_function { "procedure": "0x1000004f0" } →
Hopper analyze_function failed (-32000, invalid_request): No procedure exists at the requested address
trace_feature "POSIXLY_CORRECT" → string found at 0x100000913, but no incoming references
(same for every other string in main)

The loaded bytes are the `arm64e.x1` slice

$ lipo -thin arm64e    -output /tmp/e.arm64e    /bin/echo
$ lipo -thin arm64e.x1 -output /tmp/e.arm64e.x1 /bin/echo
$ xxd -s 0x4f0 -l 8 /tmp/e.arm64e        # 7f 23 03 d5 ff 03 02 d1   -> pacibsp
$ xxd -s 0x4f0 -l 8 /tmp/e.arm64e.x1     # fe a7 c1 da ff 03 02 d1   -> pacibsppc

# REA: read_bytes { "address": "0x1000004f0", "length": 32 }
#   bytes_hex = "fea7c1daff0302d1fc6f02a9fa6703a9f85f04a9f65705a9f44f06a9fd7b07a9"

REA's bytes match the arm64e.x1 slice exactly, and do not appear anywhere in the arm64e
slice. Note that REA reports architecture: "arm64" and available_architectures: ["arm64"] for
the FAT file — the arm64e / arm64e.x1 distinction is not surfaced.

The two slices differ by only ~380 bytes overall (the cpusubtype byte, LC_UUID, and scattered PAC
encodings at function entries and returns) — but the entry encodings decide whether analysis works
at all.

Scope — not specific to `/bin/echo`

Using dyld_info -function_starts as ground truth:

Binary	Real function starts	Code procedures REA/Hopper creates
`/bin/echo`	3 (`0x4F0`, `0x7C4`, `0x7F8`)	0
`/bin/pwd`	2 (`0x588`, `0x74C`)	1, and mis-aligned (`sub_10000058c` = entry +4)
`/bin/ln`	several from entry `0x5E8`	1, and not at the entry
`/bin/cat`	several	2

The +4 offset is the signature of the failure: the first instruction is undecodable, so Hopper
only starts a function at the following instruction.

Control — the toolchain itself is fine

/opt/homebrew/bin/bomtool is a thin arm64 binary with symbol names. REA/Hopper returns all
14/14 LC_FUNCTION_STARTS entries at exactly the ground-truth addresses (_main at
0x100000e70 … _parse_long_options at 0x100002408). Its loader arguments are
["-l","Mach-O","--aarch64"] with macho_container: "thin", versus ["-l","FAT","--aarch64","-l","Mach-O"]
with macho_container: "fat32" for the failing cases.

So the problem is specific to FAT + arm64e.x1 + stripped, i.e. Apple's own system binaries.

Workaround (verified)

Thin the file to the normal arm64e slice before handing it to REA:

$ lipo -thin arm64e -output /tmp/echo.arm64e /bin/echo
$ rea function /tmp/echo.arm64e 0x1000004f0

Results on /bin/echo:

	FAT original (loads x1)	`lipo -thin arm64e`
`procedure_count`	5 (all import stubs)	11
`read_bytes 0x1000004f0`	`fea7c1da…` (`pacibsppc`)	`7f2303d5…` (`pacibsp`)
`analyze_function 0x1000004f0`	error, no procedure	`EntryPoint` + real pseudocode
`trace_feature "POSIXLY_CORRECT"`	no xrefs	xref from `0x100000520` to string `0x100000913`

/bin/pwd likewise goes 10 → 11 procedures, with the function boundary corrected to 0x588
(instead of sub_10000058c at entry +4).

Suspected cause / suggested fix

When resolving a FAT Mach-O for --aarch64, the arm64e.x1 slice (cpusubtype 12) is selected
rather than the arm64e slice (cpusubtype 2). Whether the selection happens inside REA or inside
Hopper's -l FAT loader, the outcome is a slice whose entry encodings plain arm64 cannot decode.

Suggestions, in order of preference:

When targeting plain arm64, prefer cpusubtype 2 (arm64e) over cpusubtype 12
(arm64e.x1) during FAT slice selection.
Otherwise, pass an explicit slice/arch selector to the loader so the choice is not implicit.
Optionally surface the selected slice (cpusubtype included) in the analysis profile, since
architecture: "arm64" currently hides a choice that decides whether analysis succeeds.

Happy to test a patch — this is reproducible in one command on any Apple-silicon macOS with Hopper
installed.

Originally created by @XYishion on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3072 ## Summary On Apple-silicon macOS, system binaries (`/bin/*`, `/usr/bin/*`) are FAT Mach-O files with three slices: `x86_64`, `arm64e` (cpusubtype 2) and `arm64e.x1` (cpusubtype 12). The Hopper provider is launched as `hopper -l FAT --aarch64 -l Mach-O --analysis ...`, and the bridge ends up loading the **`arm64e.x1`** slice. That slice's entry point begins with `pacibsppc` (`0xdac1a7fe`) — an arm64e.x1-only encoding that is an **invalid instruction under plain arm64** (whereas the same offset in the `arm64e` slice holds `pacibsp` / `0xd503237f`, a decodable HINT-class NOP). Hopper's recursive descent therefore aborts at the first instruction of every function, and almost no code procedures are created. The session then fails closed with `No procedure exists at the requested address`, which is correct behaviour on REA's side — but the underlying slice choice makes the target effectively unanalyzable. ## Environment - macOS: Darwin 27.0.0 (Apple silicon) - `rea-agents` 6.4.0 (via `npx` and via pinned `rea.mjs`) - Hopper Disassembler 6.6.0 (demo), launcher sha256 `dad6f886261fad0d4cdd01aee6fcf57439c60d7b5c54e62fc97f0180116690e0` - Reproduced through both the MCP session and the headless CLI (`rea inspect` / `rea function`) ## Reproduction ```console $ lipo -archs /bin/echo x86_64 arm64e arm64e.x1 $ dyld_info -function_starts /bin/echo # ground truth # 3 real functions: 0x1000004F0, 0x1000007C4, 0x1000007F8 ``` Open `/bin/echo` with REA (MCP `open_binary` + `binary_overview`/`list_procedures`, or `rea inspect /bin/echo`): - `binary_overview` → `procedure_count: 5`, all of them import stubs; **no `sub_*` procedures** - `analyze_function { "procedure": "0x1000004f0" }` → `Hopper analyze_function failed (-32000, invalid_request): No procedure exists at the requested address` - `trace_feature "POSIXLY_CORRECT"` → string found at `0x100000913`, but **no incoming references** (same for every other string in `main`) ### The loaded bytes are the `arm64e.x1` slice ```console $ lipo -thin arm64e -output /tmp/e.arm64e /bin/echo $ lipo -thin arm64e.x1 -output /tmp/e.arm64e.x1 /bin/echo $ xxd -s 0x4f0 -l 8 /tmp/e.arm64e # 7f 23 03 d5 ff 03 02 d1 -> pacibsp $ xxd -s 0x4f0 -l 8 /tmp/e.arm64e.x1 # fe a7 c1 da ff 03 02 d1 -> pacibsppc # REA: read_bytes { "address": "0x1000004f0", "length": 32 } # bytes_hex = "fea7c1daff0302d1fc6f02a9fa6703a9f85f04a9f65705a9f44f06a9fd7b07a9" ``` REA's bytes match the `arm64e.x1` slice **exactly**, and do not appear anywhere in the `arm64e` slice. Note that REA reports `architecture: "arm64"` and `available_architectures: ["arm64"]` for the FAT file — the `arm64e` / `arm64e.x1` distinction is not surfaced. The two slices differ by only ~380 bytes overall (the cpusubtype byte, `LC_UUID`, and scattered PAC encodings at function entries and returns) — but the entry encodings decide whether analysis works at all. ## Scope — not specific to `/bin/echo` Using `dyld_info -function_starts` as ground truth: | Binary | Real function starts | Code procedures REA/Hopper creates | |---|---|---| | `/bin/echo` | 3 (`0x4F0`, `0x7C4`, `0x7F8`) | **0** | | `/bin/pwd` | 2 (`0x588`, `0x74C`) | 1, and mis-aligned (`sub_10000058c` = entry **+4**) | | `/bin/ln` | several from entry `0x5E8` | 1, and **not at the entry** | | `/bin/cat` | several | 2 | The `+4` offset is the signature of the failure: the first instruction is undecodable, so Hopper only starts a function at the following instruction. ## Control — the toolchain itself is fine `/opt/homebrew/bin/bomtool` is a **thin arm64** binary **with symbol names**. REA/Hopper returns all **14/14** `LC_FUNCTION_STARTS` entries at exactly the ground-truth addresses (`_main` at `0x100000e70` … `_parse_long_options` at `0x100002408`). Its loader arguments are `["-l","Mach-O","--aarch64"]` with `macho_container: "thin"`, versus `["-l","FAT","--aarch64","-l","Mach-O"]` with `macho_container: "fat32"` for the failing cases. So the problem is specific to **FAT + arm64e.x1 + stripped**, i.e. Apple's own system binaries. ## Workaround (verified) Thin the file to the normal `arm64e` slice before handing it to REA: ```console $ lipo -thin arm64e -output /tmp/echo.arm64e /bin/echo $ rea function /tmp/echo.arm64e 0x1000004f0 ``` Results on `/bin/echo`: | | FAT original (loads x1) | `lipo -thin arm64e` | |---|---|---| | `procedure_count` | 5 (all import stubs) | **11** | | `read_bytes 0x1000004f0` | `fea7c1da…` (`pacibsppc`) | `7f2303d5…` (`pacibsp`) | | `analyze_function 0x1000004f0` | error, no procedure | **`EntryPoint`** + real pseudocode | | `trace_feature "POSIXLY_CORRECT"` | no xrefs | xref from `0x100000520` to string `0x100000913` | `/bin/pwd` likewise goes 10 → 11 procedures, with the function boundary corrected to `0x588` (instead of `sub_10000058c` at entry `+4`). ## Suspected cause / suggested fix When resolving a FAT Mach-O for `--aarch64`, the `arm64e.x1` slice (cpusubtype 12) is selected rather than the `arm64e` slice (cpusubtype 2). Whether the selection happens inside REA or inside Hopper's `-l FAT` loader, the outcome is a slice whose entry encodings plain arm64 cannot decode. Suggestions, in order of preference: 1. When targeting plain arm64, prefer cpusubtype **2** (`arm64e`) over cpusubtype **12** (`arm64e.x1`) during FAT slice selection. 2. Otherwise, pass an explicit slice/arch selector to the loader so the choice is not implicit. 3. Optionally surface the selected slice (cpusubtype included) in the analysis profile, since `architecture: "arm64"` currently hides a choice that decides whether analysis succeeds. Happy to test a patch — this is reproducible in one command on any Apple-silicon macOS with Hopper installed.

bsakaguchi

2026-10-15 18:30:52 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:30:53 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Claiming this for a fix.

Scope I am taking: extend REA's existing FAT64 thin-slice preparation path (resolveHopperMachOImage / prepareHopperMachOImage) to FAT32 containers, so an arm64 target on a FAT Mach-O resolves a verified thin image of the cpusubtype-2 (arm64e) slice — preferring cpusubtype 2 over 12 (arm64e.x1) when both are present, per suggestion 1 — and opens it through the already-verified -l Mach-O --aarch64 thin-image route instead of Hopper's implicit -l FAT slice choice (the explicit-selection intent of suggestion 2). The selected slice's cpu_type/cpu_subtype stay surfaced in the prepared-image profile per suggestion 3.

Out of scope, so nobody duplicates: changes to Hopper itself, new user-facing slice-selector flags, Ghidra or other-provider behavior, and the FAT64 ambiguity/fail-closed policy beyond reusing it for the shared selector. Unit-level regression tests will use synthesized FAT32 fixtures, so no Hopper install is needed to verify slice selection; real-Hopper end-to-end stays with the reporter's verified workaround as the reference.

PR to follow on top of current main.

@akushonkamen commented on GitHub (Oct 14, 2026): Claiming this for a fix. Scope I am taking: extend REA's existing FAT64 thin-slice preparation path (`resolveHopperMachOImage` / `prepareHopperMachOImage`) to FAT32 containers, so an arm64 target on a FAT Mach-O resolves a verified thin image of the cpusubtype-2 (`arm64e`) slice — preferring cpusubtype 2 over 12 (`arm64e.x1`) when both are present, per suggestion 1 — and opens it through the already-verified `-l Mach-O --aarch64` thin-image route instead of Hopper's implicit `-l FAT` slice choice (the explicit-selection intent of suggestion 2). The selected slice's `cpu_type`/`cpu_subtype` stay surfaced in the prepared-image profile per suggestion 3. Out of scope, so nobody duplicates: changes to Hopper itself, new user-facing slice-selector flags, Ghidra or other-provider behavior, and the FAT64 ambiguity/fail-closed policy beyond reusing it for the shared selector. Unit-level regression tests will use synthesized FAT32 fixtures, so no Hopper install is needed to verify slice selection; real-Hopper end-to-end stays with the reporter's verified workaround as the reference. PR to follow on top of current main.

bsakaguchi commented

2026-10-15 18:30:53 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Opened PR #3092 with the fix: the existing FAT64 thin-slice preparation now covers FAT32 containers, and arm64 targets prefer cpusubtype 0/2 over arm64e.x1 (x1 only as sole candidate; other ambiguity stays fail-closed), loading through the verified -l Mach-O --aarch64 thin route. 11 new FAT32 regressions fail on pre-fix code and pass after (plus 143 related tests green); the real-Hopper FAT32 lane (verify:hopper:fat) still needs a run where Hopper is installed.

@akushonkamen commented on GitHub (Oct 14, 2026): Opened PR #3092 with the fix: the existing FAT64 thin-slice preparation now covers FAT32 containers, and arm64 targets prefer cpusubtype 0/2 over `arm64e.x1` (x1 only as sole candidate; other ambiguity stays fail-closed), loading through the verified `-l Mach-O --aarch64` thin route. 11 new FAT32 regressions fail on pre-fix code and pass after (plus 143 related tests green); the real-Hopper FAT32 lane (`verify:hopper:fat`) still needs a run where Hopper is installed.

bsakaguchi commented

2026-10-15 18:30:53 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Resolved on main by cf806ea02.

FAT slice preparation now selects and verifies a deterministic compatible thin image. Current tests exercise slice/profile production; no fresh Hopper analysis of the system binary is claimed.

Rechecked on main b8510ee00 in the current focused run: 17 test files passed, 485 tests passed and one platform-dependent test skipped. Relevant regression coverage: src/hopper/HopperMachOImage.test.ts and src/hopper/HopperAnalysisProfile.test.ts.

@morluto commented on GitHub (Oct 14, 2026): Resolved on main by cf806ea02. FAT slice preparation now selects and verifies a deterministic compatible thin image. Current tests exercise slice/profile production; no fresh Hopper analysis of the system binary is claimed. Rechecked on main b8510ee00 in the current focused run: 17 test files passed, 485 tests passed and one platform-dependent test skipped. Relevant regression coverage: src/hopper/HopperMachOImage.test.ts and src/hopper/HopperAnalysisProfile.test.ts.

bsakaguchi referenced this issue

2026-10-17 21:08:22 +00:00

[PR #1077] [MERGED] docs(website): add a CTF showcase and FAQ #2625

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1077

No description provided.

Rows
Columns

[GH-ISSUE #3072] [Bug] Hopper provider on macOS: FAT Mach-O slice selection loads arm64e.x1, collapsing analysis of system binaries #1077

Summary

Environment

Reproduction

The loaded bytes are the arm64e.x1 slice

Scope — not specific to /bin/echo

Control — the toolchain itself is fine

Workaround (verified)

Suspected cause / suggested fix

[GH-ISSUE #3072] [Bug] Hopper provider on macOS: FAT Mach-O slice selection loads `arm64e.x1`, collapsing analysis of system binaries #1077

The loaded bytes are the `arm64e.x1` slice

Scope — not specific to `/bin/echo`