morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1973] [Bug] inspect_macho mixes slice and container file offsets for universal binaries #656

New issue

Closed

opened 2026-10-15 18:27:01 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:27:01 +00:00

Owner

Originally created by @thefolahan on GitHub (Oct 11, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1973

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

For a universal (FAT) Mach-O, inspect_macho returns file_offset values with two different bases under the same field name, and the result body does not say which base applies:

architectures.items[].file_offset is measured from the start of the input file (the FAT container).
entrypoints.items[].file_offset, segments.items[].file_offset and section file_offset values are copied from otool for the selected slice, so they are measured from the start of that slice.

The selected slice is named only in the Evidence envelope (subject.architecture), not in normalized_result. The envelope subject.format is mach-o although the file is mach-o-universal.

fileOffsetLocations already corrects segment ranges by adding the slice offset, so the Evidence locations are container based while the normalized segment values next to them are slice based. The entrypoint has no location, so its slice based value is the only one reported.

On a universal Swift app (x86_64 slice at 16384, arm64 slice at 1540096), REA selects arm64 and reports:

"entrypoints": { "items": [ { "file_offset": 226996 } ] },
"architectures": { "items": [
  { "name": "x86_64", "file_offset": 16384,   "size": 1514992 },
  { "name": "arm64",  "file_offset": 1540096, "size": 1498000 }
] },
"segments": { "items": [ { "name": "__TEXT", "file_offset": 0, "file_size": 491520 } ] }

Read as an offset into the input file, as the neighbouring architectures values are, 226996 lies inside the x86_64 slice:

$ xxd -s 226996 -l 8 Vigil                  # offset as reported
000376b4: 4710 488b 8578 ffff               # x86_64 code (48 8b = mov)
$ xxd -s $((1540096 + 226996)) -l 8 Vigil   # slice offset + reported value
001af6b4: f44f bea9 fd7b 01a9               # arm64 _main prologue: stp x20, x19 / stp x29, x30
$ nm -arch arm64 Vigil | grep ' _main$'
00000001000376b4 T _main                    # __TEXT vmaddr 0x100000000 + 226996

Reproduction

Build any universal macOS executable, for example swift build -c release --arch arm64 --arch x86_64, or use a universal app from /Applications.
Run rea inspect-macho <executable> --format json on an arm64 host.
Compare normalized_result.entrypoints.items[0].file_offset with otool -arch arm64 -l <executable> | grep entryoff and with the arm64 entry in normalized_result.architectures.

The two values are equal (slice based), while the architecture offsets are container based. A second universal app gave the same result (entry 21652, arm64 slice at 5554176).

Expected behavior

A consumer of the normalized result can tell, without reading the Evidence envelope or the source, which slice was inspected and which base each file_offset uses, and can turn the entrypoint into an offset in the input file. Two possible shapes, open to the maintainers' preference:

Add the selected slice to the result (for example selected_architecture with its container file_offset) and state in the schema that entrypoint, segment and section offsets are slice relative; or
Report container offsets everywhere (adding the slice offset, as fileOffsetLocations already does for segments) and keep the slice relative value in a separately named field.

Either way, the entrypoint could also get an Evidence location like the segments have.

Impact and scope

Any universal Mach-O, which includes most current macOS apps. Thin Mach-O files are not affected, because the slice starts at offset 0. An agent that passes the entrypoint offset to a byte reader, or compares it with architectures or locations, reads the wrong slice. On these targets that means x86_64 instructions are read as the arm64 entry.

REA version

6.4.0 (npm), source checked against main at 1d4c4eb

Runtime and platform

macOS 27.0.1, Apple silicon, Node 24.21.0

Target and artifact context

Two of my own open source SwiftUI apps, built as universal (x86_64 + arm64) and ad hoc signed: Vigil (github.com/thefolahan/vigil) and API Pilot (github.com/thefolahan/api-pilot).

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

Evidence locations from the same run already use container offsets for the arm64 segments, while the normalized __TEXT segment says file_offset: 0:

[{"kind":"file-offset-range","start":16384,"end":1531376},
 {"kind":"file-offset-range","start":1540096,"end":3038096},
 {"kind":"file-offset-range","start":1540096,"end":2031616}, ...]

Typed outcome or recovery expected

No error is expected; the result should be unambiguous about the offset base and the selected slice.

Analysis or suspected cause

normalizeMacho in src/native/NativeMachoInspection.ts copies otool -arch <slice> -l values (entryoff, fileoff, offset) directly into the result. segmentContainerOffset computes the slice offset but applies it only in fileOffsetLocations, and only to segments. inspectMachoSchema in src/domain/native/nativeInspection.ts has no field for the selected architecture and uses the name file_offset for both bases.

Additional context

This changes the meaning or shape of the inspect_macho result, so I am opening an issue before a PR as CONTRIBUTING asks. I am happy to implement whichever shape the maintainers prefer.

Originally created by @thefolahan on GitHub (Oct 11, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1973 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior For a universal (FAT) Mach-O, `inspect_macho` returns `file_offset` values with two different bases under the same field name, and the result body does not say which base applies: - `architectures.items[].file_offset` is measured from the start of the input file (the FAT container). - `entrypoints.items[].file_offset`, `segments.items[].file_offset` and section `file_offset` values are copied from `otool` for the selected slice, so they are measured from the start of that slice. The selected slice is named only in the Evidence envelope (`subject.architecture`), not in `normalized_result`. The envelope `subject.format` is `mach-o` although the file is `mach-o-universal`. `fileOffsetLocations` already corrects segment ranges by adding the slice offset, so the Evidence `locations` are container based while the normalized segment values next to them are slice based. The entrypoint has no location, so its slice based value is the only one reported. On a universal Swift app (x86_64 slice at 16384, arm64 slice at 1540096), REA selects arm64 and reports: ```json "entrypoints": { "items": [ { "file_offset": 226996 } ] }, "architectures": { "items": [ { "name": "x86_64", "file_offset": 16384, "size": 1514992 }, { "name": "arm64", "file_offset": 1540096, "size": 1498000 } ] }, "segments": { "items": [ { "name": "__TEXT", "file_offset": 0, "file_size": 491520 } ] } ``` Read as an offset into the input file, as the neighbouring `architectures` values are, 226996 lies inside the x86_64 slice: ```text $ xxd -s 226996 -l 8 Vigil # offset as reported 000376b4: 4710 488b 8578 ffff # x86_64 code (48 8b = mov) $ xxd -s $((1540096 + 226996)) -l 8 Vigil # slice offset + reported value 001af6b4: f44f bea9 fd7b 01a9 # arm64 _main prologue: stp x20, x19 / stp x29, x30 $ nm -arch arm64 Vigil | grep ' _main$' 00000001000376b4 T _main # __TEXT vmaddr 0x100000000 + 226996 ``` ### Reproduction 1. Build any universal macOS executable, for example `swift build -c release --arch arm64 --arch x86_64`, or use a universal app from `/Applications`. 2. Run `rea inspect-macho <executable> --format json` on an arm64 host. 3. Compare `normalized_result.entrypoints.items[0].file_offset` with `otool -arch arm64 -l <executable> | grep entryoff` and with the arm64 entry in `normalized_result.architectures`. The two values are equal (slice based), while the architecture offsets are container based. A second universal app gave the same result (entry 21652, arm64 slice at 5554176). ### Expected behavior A consumer of the normalized result can tell, without reading the Evidence envelope or the source, which slice was inspected and which base each `file_offset` uses, and can turn the entrypoint into an offset in the input file. Two possible shapes, open to the maintainers' preference: 1. Add the selected slice to the result (for example `selected_architecture` with its container `file_offset`) and state in the schema that entrypoint, segment and section offsets are slice relative; or 2. Report container offsets everywhere (adding the slice offset, as `fileOffsetLocations` already does for segments) and keep the slice relative value in a separately named field. Either way, the entrypoint could also get an Evidence location like the segments have. ### Impact and scope Any universal Mach-O, which includes most current macOS apps. Thin Mach-O files are not affected, because the slice starts at offset 0. An agent that passes the entrypoint offset to a byte reader, or compares it with `architectures` or `locations`, reads the wrong slice. On these targets that means x86_64 instructions are read as the arm64 entry. ### REA version 6.4.0 (npm), source checked against main at 1d4c4eb ### Runtime and platform macOS 27.0.1, Apple silicon, Node 24.21.0 ### Target and artifact context Two of my own open source SwiftUI apps, built as universal (x86_64 + arm64) and ad hoc signed: Vigil (github.com/thefolahan/vigil) and API Pilot (github.com/thefolahan/api-pilot). ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence Evidence `locations` from the same run already use container offsets for the arm64 segments, while the normalized `__TEXT` segment says `file_offset: 0`: ```json [{"kind":"file-offset-range","start":16384,"end":1531376}, {"kind":"file-offset-range","start":1540096,"end":3038096}, {"kind":"file-offset-range","start":1540096,"end":2031616}, ...] ``` ### Typed outcome or recovery expected No error is expected; the result should be unambiguous about the offset base and the selected slice. ### Analysis or suspected cause `normalizeMacho` in `src/native/NativeMachoInspection.ts` copies `otool -arch <slice> -l` values (`entryoff`, `fileoff`, `offset`) directly into the result. `segmentContainerOffset` computes the slice offset but applies it only in `fileOffsetLocations`, and only to segments. `inspectMachoSchema` in `src/domain/native/nativeInspection.ts` has no field for the selected architecture and uses the name `file_offset` for both bases. ### Additional context This changes the meaning or shape of the `inspect_macho` result, so I am opening an issue before a PR as CONTRIBUTING asks. I am happy to implement whichever shape the maintainers prefer.

bsakaguchi

2026-10-15 18:27:01 +00:00

closed this issue
added the
area: native

P1: high

bug
labels

bsakaguchi commented

2026-10-15 18:27:02 +00:00

Author

Owner

@TonyWu2333 commented on GitHub (Oct 12, 2026):

Verified fixed on current main (3bdc873), apparently by 8bd5ba2 ("fix(native): clarify Mach-O offset origins…").

Reproduction on macOS (Apple silicon) with a clang -arch x86_64 -arch arm64 executable (x86_64 slice at 4096, arm64 slice at 16384, arm64 entryoff 816), rea inspect-macho <file> --format json:

normalized_result.selected_architecture is {"name":"arm64","container_file_offset":16384}
the schema now states that architectures[].file_offset is relative to the container and that entrypoint, segment and section offsets are relative to the slice
the entrypoint has an Evidence location {"kind":"file-offset","offset":17200} (16384 + 816); the bytes at that container offset are arm64 code (c0 03 5f d6, ret)
the envelope subject.format is mach-o-universal

Regression coverage: tests/boundary/providers/native/nativeSelectedArchitecture.test.ts. I think this can be closed.

@TonyWu2333 commented on GitHub (Oct 12, 2026): Verified fixed on current main (3bdc873), apparently by 8bd5ba2 ("fix(native): clarify Mach-O offset origins…"). Reproduction on macOS (Apple silicon) with a `clang -arch x86_64 -arch arm64` executable (x86_64 slice at 4096, arm64 slice at 16384, arm64 `entryoff` 816), `rea inspect-macho <file> --format json`: - `normalized_result.selected_architecture` is `{"name":"arm64","container_file_offset":16384}` - the schema now states that `architectures[].file_offset` is relative to the container and that entrypoint, segment and section offsets are relative to the slice - the entrypoint has an Evidence location `{"kind":"file-offset","offset":17200}` (16384 + 816); the bytes at that container offset are arm64 code (`c0 03 5f d6`, `ret`) - the envelope `subject.format` is `mach-o-universal` Regression coverage: `tests/boundary/providers/native/nativeSelectedArchitecture.test.ts`. I think this can be closed.

bsakaguchi commented

2026-10-15 18:27:02 +00:00

Author

Owner

@morluto commented on GitHub (Oct 13, 2026):

Verified fixed on main by commit 8bd5ba2d2 and the existing reporter verification. The normalized result identifies the selected slice and its container offset, schema descriptions distinguish slice-relative and container-relative offsets, and Evidence entrypoint locations use the container basis. Current regression checks passed on macOS arm64 / Node.js 24.18.0: src/native/NativeMachoInspection.test.ts (19 tests) and tests/boundary/providers/native/nativeSelectedArchitecture.test.ts (14 tests). Closing the reported ambiguity as fixed.

@morluto commented on GitHub (Oct 13, 2026): Verified fixed on main by commit 8bd5ba2d2 and the existing reporter verification. The normalized result identifies the selected slice and its container offset, schema descriptions distinguish slice-relative and container-relative offsets, and Evidence entrypoint locations use the container basis. Current regression checks passed on macOS arm64 / Node.js 24.18.0: `src/native/NativeMachoInspection.test.ts` (19 tests) and `tests/boundary/providers/native/nativeSelectedArchitecture.test.ts` (14 tests). Closing the reported ambiguity as fixed.

bsakaguchi referenced this issue

2026-10-15 18:27:31 +00:00

[GH-ISSUE #2139] [Bug] IDA provider analyzes the x86_64 slice of a universal Mach-O while reporting arm64 #700

bsakaguchi referenced this issue

2026-10-15 18:28:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-15 18:31:44 +00:00

[GH-ISSUE #3282] [Feature] Ghidra provider: support universal (FAT) Mach-O by importing the selected slice #1172

bsakaguchi referenced this issue

2026-10-16 21:15:29 +00:00

[PR #713] [MERGED] fix(ghidra): refuse ambiguous universal Mach-O imports #2193