morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4280] inspect_artifact: stock macOS universal binaries report integrity_mismatch (arm64e.x1 slice defeats lipoNameMatches; integrity_policy inert on this path) #1694

New issue

Open

opened 2026-10-16 21:09:10 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:09:10 +00:00

Owner

Originally created by @mkd3434 on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4280

Title

Stock macOS universal binaries fail with integrity_mismatch: arm64e.x1 slice defeats the
lipoNameMatches ABI-variant carve-out (and integrity_policy is inert on this path)

Observed behavior

On macOS 27 / arm64, inspect_artifact and extract_artifact fail on essentially every stock
Apple-signed system binary with an integrity verdict:

$ rea inspect-artifact /bin/echo --format json
code:     artifact_operation_failed
category: integrity_mismatch
message:  "Artifact bytes contradict declared integrity. If expected, rerun inspect_artifact
           with integrity_policy=record-and-continue ... otherwise get a fresh copy"
detail:   "lipo slice metadata disagrees with the Mach-O FAT table: arm64e.x1
           (cputype=CPU_TYPE_ARM64, cpusubtype=unknown arm64 cpusubtype,
            offset=114688, size=51792, alignment=16384)"
exit: 1

The files are pristine and Apple-signed. The failure is not about bytes: the printed cputype,
offset, size and alignment all already agree with the FAT table. The only thing that disagrees
is the slice name. These binaries carry a third fat entry that newer lipo reports as
arm64e.x1 (capabilities PTR_AUTH_VERSION USERSPACE 0).

Root cause

dist/artifacts/MachOSliceArtifactReader.js:278

const lipoNameMatches = (reported, observed) => reported === observed ||
    (observed === "arm64e" && /^arm64e\.[A-Za-z0-9_]+$/u.test(reported));

The ABI-variant carve-out requires observed === "arm64e". Its own comment states the assumption
that fails: "The FAT table names that slice arm64e; its physical identity is still compared field
by field." For this slice the FAT table does not name it arm64e. Raw fat header (big-endian):

[1] cputype=0x0100000c cpusubtype=0x80000002  <- genuine arm64e
[2] cputype=0x0100000c cpusubtype=0x8000000c  <- the arm64e.x1 variant; low bits 12, absent from
                                                  REA's arm64e table, so named "arm64"

With reported = "arm64e.x1" and observed = "arm64", both clauses fail and
MachOSliceArtifactReader.js:63 throws ArtifactReaderFailure("integrity", ...). The carve-out
misses precisely the case it was written for.

Second defect: the advertised recovery is inert here

The error tells the user to rerun with integrity_policy=record-and-continue. The flag is parsed
(bogus values are rejected) but has no effect on this failure:

$ rea inspect-artifact /bin/echo --integrity-policy record-and-continue ; echo $?
1

integrityPolicy is only consumed at dist/artifacts/extraction/ArtifactExtraction.js:28 (threaded
via ArtifactProvider.js:131), downstream of the throw during slice enumeration. There is no
in-product recovery, and the remediation string is misleading at the point of failure.

Reproduction

rea inspect-artifact /bin/echo                                        # exit 1, integrity_mismatch
rea inspect-artifact /usr/bin/grep                                    # same
rea inspect-artifact /bin/echo --integrity-policy record-and-continue # still exit 1

Prevalence on this host (macOS 27.0.0, arm64): /bin 37/37, /usr/bin 668/670 carry an arm64e.x1
fat entry. Negative control — a plain two-arch fat binary with no ABI-variant slice succeeds
(exit 0, enumerates both slices). Workaround outside REA: lipo -thin x86_64 /bin/echo -o /tmp/t/x
then rea inspect /tmp/t/x --provider ghidra (exit 0).

Expected behavior

A slice whose byte ranges agree with the FAT table must not produce an integrity verdict.
Reserve ArtifactReaderFailure("integrity", ...) for byte-range disagreement; a slice that merely
cannot be named is a format/capability limit ("format"/"unsupported"), not tampering.
Either wire integrity_policy into the slice-enumeration path or stop advertising it in this
error's remediation text.

Impact and scope

Affects evidence-integrity reporting: the wording is exactly what an analyst reads as tampering,
and on this platform it fires on ~99.7% of provably-unmodified system binaries. Blocks a whole
target class (inspect_artifact + extract_artifact on stock macOS universal binaries) with no
in-product recovery. Fails closed and loudly — no silent wrong result, no security bypass, logic
recoverable by thinning — hence correctness/UX, not urgent. Not affected: thinned single-arch Mach-O,
plain multi-arch fat binaries, the Ghidra lane on thinned input. This is fat-header slice naming,
unrelated to arm64e pointer-authentication handling during analysis.

REA version / runtime / target

6.4.0 (npm) · Node 24.14.0 · macOS 27.0.0 (Darwin 27.0.0) · arm64 · Ghidra 12.1.4 · JDK 27.
Target: Mach-O universal (FAT), nfat_arch 3, third entry reported by lipo as arm64e.x1.

Verified fix (patch-ready)

The two matcher functions were extracted verbatim from MachOSliceArtifactReader.js:253-279 and
driven with the real /bin/echo arm64e.x1 slice. Minimal one-line change to lipoNameMatches:

-const lipoNameMatches = (reported, observed) => reported === observed ||
-    (observed === "arm64e" && /^arm64e\.[A-Za-z0-9_]+$/u.test(reported));
+const lipoNameMatches = (reported, observed) =>
+    reported === observed ||
+    (/^arm64e(\.[A-Za-z0-9_]+)?$/u.test(reported) &&
+        (observed === "arm64e" || observed === "arm64"));

Rationale: the FAT table names an ABI-variant slice by whichever arm64-family name its
fat_cpu_subtype resolves to — arm64e when the low bits are known (0x…02), arm64 when they are
not (0x8000000C). Accepting either observed name for a reported string of arm64e or
arm64e.<variant> closes the gap without weakening the field-by-field byte checks lipoMatchesSlice
still applies.

Case	current	patched
real `/bin/echo` arm64e.x1 slice	`false` → throws integrity_mismatch	`true` → passes
same slice, tampered `size`	—	`false` — still rejects real corruption
genuine `arm64e` slice [1] (0x…02)	`true`	`true` — no regression

The verification harness is a verbatim extraction of lipoMatchesSlice/lipoNameMatches driven by
the real slice values above; happy to attach it if useful.

Originally created by @mkd3434 on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4280 ## Title Stock macOS universal binaries fail with `integrity_mismatch`: `arm64e.x1` slice defeats the `lipoNameMatches` ABI-variant carve-out (and `integrity_policy` is inert on this path) ## Observed behavior On macOS 27 / arm64, `inspect_artifact` and `extract_artifact` fail on essentially every stock Apple-signed system binary with an **integrity** verdict: ``` $ rea inspect-artifact /bin/echo --format json code: artifact_operation_failed category: integrity_mismatch message: "Artifact bytes contradict declared integrity. If expected, rerun inspect_artifact with integrity_policy=record-and-continue ... otherwise get a fresh copy" detail: "lipo slice metadata disagrees with the Mach-O FAT table: arm64e.x1 (cputype=CPU_TYPE_ARM64, cpusubtype=unknown arm64 cpusubtype, offset=114688, size=51792, alignment=16384)" exit: 1 ``` The files are pristine and Apple-signed. The failure is **not** about bytes: the printed `cputype`, `offset`, `size` and `alignment` all already agree with the FAT table. The only thing that disagrees is the slice **name**. These binaries carry a third fat entry that newer `lipo` reports as `arm64e.x1` (`capabilities PTR_AUTH_VERSION USERSPACE 0`). ### Root cause `dist/artifacts/MachOSliceArtifactReader.js:278` ```js const lipoNameMatches = (reported, observed) => reported === observed || (observed === "arm64e" && /^arm64e\.[A-Za-z0-9_]+$/u.test(reported)); ``` The ABI-variant carve-out requires `observed === "arm64e"`. Its own comment states the assumption that fails: *"The FAT table names that slice `arm64e`; its physical identity is still compared field by field."* For this slice the FAT table does **not** name it `arm64e`. Raw fat header (big-endian): ``` [1] cputype=0x0100000c cpusubtype=0x80000002 <- genuine arm64e [2] cputype=0x0100000c cpusubtype=0x8000000c <- the arm64e.x1 variant; low bits 12, absent from REA's arm64e table, so named "arm64" ``` With `reported = "arm64e.x1"` and `observed = "arm64"`, both clauses fail and `MachOSliceArtifactReader.js:63` throws `ArtifactReaderFailure("integrity", ...)`. The carve-out misses precisely the case it was written for. ### Second defect: the advertised recovery is inert here The error tells the user to rerun with `integrity_policy=record-and-continue`. The flag is parsed (bogus values are rejected) but has **no effect** on this failure: ``` $ rea inspect-artifact /bin/echo --integrity-policy record-and-continue ; echo $? 1 ``` `integrityPolicy` is only consumed at `dist/artifacts/extraction/ArtifactExtraction.js:28` (threaded via `ArtifactProvider.js:131`), downstream of the throw during slice enumeration. There is no in-product recovery, and the remediation string is misleading at the point of failure. ## Reproduction ```sh rea inspect-artifact /bin/echo # exit 1, integrity_mismatch rea inspect-artifact /usr/bin/grep # same rea inspect-artifact /bin/echo --integrity-policy record-and-continue # still exit 1 ``` Prevalence on this host (macOS 27.0.0, arm64): `/bin` 37/37, `/usr/bin` 668/670 carry an `arm64e.x1` fat entry. Negative control — a plain two-arch fat binary with no ABI-variant slice succeeds (exit 0, enumerates both slices). Workaround outside REA: `lipo -thin x86_64 /bin/echo -o /tmp/t/x` then `rea inspect /tmp/t/x --provider ghidra` (exit 0). ## Expected behavior 1. A slice whose byte ranges agree with the FAT table must not produce an **integrity** verdict. 2. Reserve `ArtifactReaderFailure("integrity", ...)` for byte-range disagreement; a slice that merely cannot be *named* is a format/capability limit (`"format"`/`"unsupported"`), not tampering. 3. Either wire `integrity_policy` into the slice-enumeration path or stop advertising it in this error's remediation text. ## Impact and scope Affects evidence-integrity reporting: the wording is exactly what an analyst reads as *tampering*, and on this platform it fires on ~99.7% of provably-unmodified system binaries. Blocks a whole target class (`inspect_artifact` + `extract_artifact` on stock macOS universal binaries) with no in-product recovery. Fails **closed** and loudly — no silent wrong result, no security bypass, logic recoverable by thinning — hence correctness/UX, not urgent. Not affected: thinned single-arch Mach-O, plain multi-arch fat binaries, the Ghidra lane on thinned input. This is fat-header slice *naming*, unrelated to arm64e pointer-authentication handling during analysis. ## REA version / runtime / target `6.4.0` (npm) · Node 24.14.0 · macOS 27.0.0 (Darwin 27.0.0) · arm64 · Ghidra 12.1.4 · JDK 27. Target: Mach-O universal (FAT), `nfat_arch 3`, third entry reported by `lipo` as `arm64e.x1`. --- ## Verified fix (patch-ready) The two matcher functions were extracted verbatim from `MachOSliceArtifactReader.js:253-279` and driven with the real `/bin/echo` `arm64e.x1` slice. Minimal one-line change to `lipoNameMatches`: ```diff -const lipoNameMatches = (reported, observed) => reported === observed || - (observed === "arm64e" && /^arm64e\.[A-Za-z0-9_]+$/u.test(reported)); +const lipoNameMatches = (reported, observed) => + reported === observed || + (/^arm64e(\.[A-Za-z0-9_]+)?$/u.test(reported) && + (observed === "arm64e" || observed === "arm64")); ``` Rationale: the FAT table names an ABI-variant slice by whichever arm64-family name its `fat_cpu_subtype` resolves to — `arm64e` when the low bits are known (0x…02), `arm64` when they are not (`0x8000000C`). Accepting either observed name for a `reported` string of `arm64e` or `arm64e.<variant>` closes the gap without weakening the field-by-field byte checks `lipoMatchesSlice` still applies. | Case | current | patched | |------|---------|---------| | real `/bin/echo` arm64e.x1 slice | `false` → throws integrity_mismatch | **`true` → passes** | | same slice, tampered `size` | — | `false` — still rejects real corruption | | genuine `arm64e` slice [1] (0x…02) | `true` | `true` — no regression | The verification harness is a verbatim extraction of `lipoMatchesSlice`/`lipoNameMatches` driven by the real slice values above; happy to attach it if useful.

bsakaguchi commented

2026-10-16 21:09:11 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 16, 2026):

Cross-referencing with #3072 (Hopper provider on macOS: FAT Mach-O slice selection loads arm64e.x1, collapsing analysis of system binaries). Both issues pinpoint Apple's arm64e.x1 (cpusubtype 12) ABI-variant slice in universal FAT Mach-O system binaries breaking reader slice selection and integrity verification (lipoNameMatches).

Both have been consolidated under Cluster G (Apple Universal FAT Mach-O arm64e.x1 ABI-Variant Slice Incompatibility) in tracker #4330.

@matbrgz commented on GitHub (Oct 16, 2026): Cross-referencing with #3072 (`Hopper provider on macOS: FAT Mach-O slice selection loads arm64e.x1, collapsing analysis of system binaries`). Both issues pinpoint Apple's `arm64e.x1` (cpusubtype 12) ABI-variant slice in universal FAT Mach-O system binaries breaking reader slice selection and integrity verification (`lipoNameMatches`). Both have been consolidated under **Cluster G (Apple Universal FAT Mach-O arm64e.x1 ABI-Variant Slice Incompatibility)** in tracker #4330.

bsakaguchi commented

2026-10-31 00:32:11 +00:00

Author

Owner

@morluto commented on GitHub (Oct 26, 2026):

Thank you for the concrete FAT-table evidence. Main now recognizes subtype 12 as arm64e.x1 (0c49878b9), and the existing Mach-O slice boundary fixture includes the reported 0x8000000c shape, so the originally reported naming mismatch is covered in source. I am leaving this issue open for the separate recovery concern: an unmatched lipo/FAT record still raises an integrity failure during enumeration, while the generic integrity guidance suggests record-and-continue. That path needs an explicit decision or corrected guidance; this source review is not a fresh reproduction on macOS.

@morluto commented on GitHub (Oct 26, 2026): Thank you for the concrete FAT-table evidence. Main now recognizes subtype 12 as `arm64e.x1` (0c49878b9), and the existing Mach-O slice boundary fixture includes the reported `0x8000000c` shape, so the originally reported naming mismatch is covered in source. I am leaving this issue open for the separate recovery concern: an unmatched lipo/FAT record still raises an integrity failure during enumeration, while the generic integrity guidance suggests `record-and-continue`. That path needs an explicit decision or corrected guidance; this source review is not a fresh reproduction on macOS.

bsakaguchi referenced this issue

2026-10-31 00:54:45 +00:00

[PR #1694] [MERGED] perf(evidence): fuse streamed JSON token handling #4760

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1694

No description provided.

Rows
Columns