[PR #618] [CLOSED] fix(native): decode little-endian fat headers as universal binaries #2118

New issue

Closed

opened 2026-10-16 21:14:31 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:14:31 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/618
Author: @aniruddhaadak80
Created: 10/5/2026
Status: ❌ Closed

Base: main ← Head: fix/native-decode-little-endian-fat-headers

📝 Commits (1)

668a7c8 fix(native): decode little-endian fat headers as universal binaries

📊 Changes

2 files changed (+126 additions, -43 deletions)

View changed files

📝 src/native/AppleDispatchMetadata.test.ts (+110 -29)
📝 src/native/AppleDispatchMetadata.ts (+16 -14)

📄 Description

Summary

selectMachoSlice recognized only the two big-endian universal-binary magics, so a fat container whose header was written in little-endian byte order was never treated as a fat container. It fell through to the thin-Mach-O path and reported Only little-endian 64-bit Mach-O metadata is supported — a diagnostic that blames the slice payload when the real constraint is the fat header's byte order.

Both byte orders are produced by real toolchains: lipo and ctool -arch all emit byte-swapped fat headers on hosts whose native fat-header write order differs from the reader's. Such a file is a valid universal binary, not a malformed or thin one. This also contradicted the target resolver, which already accepts all four fat magics and reports format: "mach-o" — so binaryTarget.ts classified the file as supported and this consumer then rejected it.

Fixes #592

Problem and expected behavior

The smallest trigger is a FAT_CIGAM (0xbebafeca) or FAT_CIGAM_64 (0xbfbafeca) container wrapping an ordinary little-endian 64-bit Mach-O slice. The violated invariant: a malformed input must stay distinct from an unsupported target and from a real format limitation, and a transformation that loses information must preserve the reported value rather than guess a canonical identity.

Observed before this change, for both byte-swapped magics:

TypeError: Only little-endian 64-bit Mach-O metadata is supported

while parseExecutableHeader admitted the same bytes as format=mach-o arch=arm64.

Expected: selectMachoSlice selects the fat-header byte order from the magic and uses it consistently for every subsequent field read.

Magic	Byte order	Stride
`0xcafebabe` / `FAT_MAGIC`	big-endian	20
`0xcafebabf` / `FAT_MAGIC_64`	big-endian	32
`0xbebafeca` / `FAT_CIGAM`	little-endian	20
`0xbfbafeca` / `FAT_CIGAM_64`	little-endian	32

The selected slice is then decoded as little-endian 64-bit Mach-O regardless of the fat header's byte order, because the slice payload's own magic is validated separately at src/native/AppleDispatchMetadata.ts:36. That validation is unchanged, so a byte-swapped header wrapping a big-endian or 32-bit slice still reports the original payload diagnostic.

Change and scope

selectMachoSlice derives a little/fat64 pair from the magic across all four fat magics and routes nfat_arch, cputype, offset, and size through read/read64 helpers. This matches the byte-order selection already used by the sibling fat consumers at src/domain/binaryTarget.ts:72-76 and src/artifacts/ArtifactGraphConstruction.ts:310.

The four malformed-container diagnostics now name the byte order, so the message identifies the producer representation that failed: Malformed little-endian FAT architecture table, little-endian FAT slice exceeds file, Ambiguous little-endian FAT architecture slice, Requested little-endian FAT architecture is absent.

Not included: no change to the thin-Mach-O path, the slice-payload magic check, the architecture mapping, or the record cap. No new exports and no CLI/MCP contract change.

Contract and boundary impact

Semantic owner and earliest changed stage: src/native/AppleDispatchMetadata.ts — the fat-container slice selection step of the Apple dispatch metadata native boundary, unchanged by upstream PR #590 which added big-endian FAT_MAGIC_64.
CLI and MCP/tool-catalog contract: unchanged. inspect_apple_dispatch_metadata keeps the same input schema, output schema, and tool-catalog entry; only the diagnostic text on two malformed byte-swapped inputs changes.
Provider, bridge, target-format, or platform compatibility: broader within the existing Apple native boundary. binaryTarget.ts already classified these files as mach-o; the consumer now agrees instead of rejecting them. No new provider, bridge, or tool.
Evidence, artifact, provenance, or reconstruction contract: unchanged. Decoded file_offset values for byte-swapped containers are byte-identical to the big-endian fixtures, so artifact identity and provenance stamping are unaffected.
Process execution, authorization, cleanup, or containment impact: none.
Generated metadata (docs/product-catalog.json), package, or installation impact: none. No contract signature changed, so no regeneration is required.

Evidence and regression coverage

Tests added or updated: src/native/AppleDispatchMetadata.test.ts. The existing it.each FAT decode case became a 4-case fat64 × littleEndian matrix; a little-endian validation block adds the truncated-table, slice-exceeds-file, and payload-rejection cases; the four pre-existing big-endian assertions were updated for the now-present byte-order qualifier.
Base reproduction or other evidence: a temporary harness in the same vitest project confirmed the wrong message on FAT_CIGAM and FAT_CIGAM_64 and correct file_offset=768 on the big-endian pair before the change, and file_offset=768 on all four after. The same run confirmed parseExecutableHeader admitted the byte-swapped files, establishing the resolver/consumer disagreement.
User-visible CLI/MCP output: a byte-swapped universal binary previously failed with Only little-endian 64-bit Mach-O metadata is supported; it now decodes. Malformed byte-swapped containers now name their byte order.
Remaining proof gaps: fixtures are synthetic. No real lipo-produced FAT_CIGAM binary was decoded — this host is Windows, so npm run verify:apple-dispatch could not run.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npx vitest run --project adapters src/native/AppleDispatchMetadata.test.ts → 18 passed
npm run check:fast → 0 errors, 98 warnings (unchanged from base; no new warnings added)
git stash push -- src/native/AppleDispatchMetadata.ts then re-running the test file → 10 failed / 8 passed, with the three little-endian rejection cases failing on Only little-endian 64-bit Mach-O metadata is supported — the regression is driven by the new tests, not merely asserted.
npm test → 48 files failed / 304 passed with the change versus 54 failed / 299 passed at base. Failures are Hopper, macOS, hdiutil, and browser suites that require hosts unavailable here and fail identically at base; this Windows host is not a green lane for the full suite.

Compatibility, safety, and release

Breaking changes or migration steps: none. The only observable change is diagnostic text on malformed byte-swapped fat headers and successful decode of previously rejected byte-swapped containers.
Real Hopper/Ghidra, browser, or OS coverage: none performed. scripts/verify-apple-dispatch.mjs is intentionally untouched — adding a lipo-produced universal fixture there could not be exercised on this Windows host, and shipping an unexercised real-toolchain claim would assert a support guarantee this PR cannot back. That fixture remains a follow-up for a macOS lane.
Package or release metadata impact: none. CHANGELOG.md is release-please generated from PR titles and is not edited by hand.
Security, privacy, process, or containment review: no new I/O, no new inputs, no process or permission surface. Slice bounds checks are unchanged and still run for both byte orders.

Review checklist

The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/618 **Author:** [@aniruddhaadak80](https://github.com/aniruddhaadak80) **Created:** 10/5/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/native-decode-little-endian-fat-headers` --- ### 📝 Commits (1) - [`668a7c8`](https://github.com/morluto/rea/commit/668a7c8004983f53c2507301a02c30b18edcfe39) fix(native): decode little-endian fat headers as universal binaries ### 📊 Changes **2 files changed** (+126 additions, -43 deletions) <details> <summary>View changed files</summary> 📝 `src/native/AppleDispatchMetadata.test.ts` (+110 -29) 📝 `src/native/AppleDispatchMetadata.ts` (+16 -14) </details> ### 📄 Description ## Summary `selectMachoSlice` recognized only the two big-endian universal-binary magics, so a fat container whose header was written in little-endian byte order was never treated as a fat container. It fell through to the thin-Mach-O path and reported `Only little-endian 64-bit Mach-O metadata is supported` — a diagnostic that blames the slice payload when the real constraint is the fat header's byte order. Both byte orders are produced by real toolchains: `lipo` and `ctool -arch all` emit byte-swapped fat headers on hosts whose native fat-header write order differs from the reader's. Such a file is a valid universal binary, not a malformed or thin one. This also contradicted the target resolver, which already accepts all four fat magics and reports `format: "mach-o"` — so `binaryTarget.ts` classified the file as supported and this consumer then rejected it. Fixes #592 ## Problem and expected behavior The smallest trigger is a `FAT_CIGAM` (`0xbebafeca`) or `FAT_CIGAM_64` (`0xbfbafeca`) container wrapping an ordinary little-endian 64-bit Mach-O slice. The violated invariant: a malformed input must stay distinct from an unsupported target and from a real format limitation, and a transformation that loses information must preserve the reported value rather than guess a canonical identity. Observed before this change, for both byte-swapped magics: ``` TypeError: Only little-endian 64-bit Mach-O metadata is supported ``` while `parseExecutableHeader` admitted the same bytes as `format=mach-o arch=arm64`. Expected: `selectMachoSlice` selects the fat-header byte order from the magic and uses it consistently for every subsequent field read. | Magic | Byte order | Stride | | --- | --- | --- | | `0xcafebabe` / `FAT_MAGIC` | big-endian | 20 | | `0xcafebabf` / `FAT_MAGIC_64` | big-endian | 32 | | `0xbebafeca` / `FAT_CIGAM` | little-endian | 20 | | `0xbfbafeca` / `FAT_CIGAM_64` | little-endian | 32 | The selected slice is then decoded as little-endian 64-bit Mach-O regardless of the fat header's byte order, because the slice payload's own magic is validated separately at `src/native/AppleDispatchMetadata.ts:36`. That validation is unchanged, so a byte-swapped header wrapping a big-endian or 32-bit slice still reports the original payload diagnostic. ## Change and scope `selectMachoSlice` derives a `little`/`fat64` pair from the magic across all four fat magics and routes `nfat_arch`, `cputype`, `offset`, and `size` through `read`/`read64` helpers. This matches the byte-order selection already used by the sibling fat consumers at `src/domain/binaryTarget.ts:72-76` and `src/artifacts/ArtifactGraphConstruction.ts:310`. The four malformed-container diagnostics now name the byte order, so the message identifies the producer representation that failed: `Malformed little-endian FAT architecture table`, `little-endian FAT slice exceeds file`, `Ambiguous little-endian FAT architecture slice`, `Requested little-endian FAT architecture is absent`. Not included: no change to the thin-Mach-O path, the slice-payload magic check, the architecture mapping, or the record cap. No new exports and no CLI/MCP contract change. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/native/AppleDispatchMetadata.ts` — the fat-container slice selection step of the Apple dispatch metadata native boundary, unchanged by upstream PR #590 which added big-endian `FAT_MAGIC_64`. - CLI and MCP/tool-catalog contract: unchanged. `inspect_apple_dispatch_metadata` keeps the same input schema, output schema, and tool-catalog entry; only the diagnostic text on two malformed byte-swapped inputs changes. - Provider, bridge, target-format, or platform compatibility: broader within the existing Apple native boundary. `binaryTarget.ts` already classified these files as `mach-o`; the consumer now agrees instead of rejecting them. No new provider, bridge, or tool. - Evidence, artifact, provenance, or reconstruction contract: unchanged. Decoded `file_offset` values for byte-swapped containers are byte-identical to the big-endian fixtures, so artifact identity and provenance stamping are unaffected. - Process execution, authorization, cleanup, or containment impact: none. - Generated metadata (`docs/product-catalog.json`), package, or installation impact: none. No contract signature changed, so no regeneration is required. ## Evidence and regression coverage - Tests added or updated: `src/native/AppleDispatchMetadata.test.ts`. The existing `it.each` FAT decode case became a 4-case `fat64 × littleEndian` matrix; a little-endian validation block adds the truncated-table, slice-exceeds-file, and payload-rejection cases; the four pre-existing big-endian assertions were updated for the now-present byte-order qualifier. - Base reproduction or other evidence: a temporary harness in the same vitest project confirmed the wrong message on `FAT_CIGAM` and `FAT_CIGAM_64` and correct `file_offset=768` on the big-endian pair before the change, and `file_offset=768` on all four after. The same run confirmed `parseExecutableHeader` admitted the byte-swapped files, establishing the resolver/consumer disagreement. - User-visible CLI/MCP output: a byte-swapped universal binary previously failed with `Only little-endian 64-bit Mach-O metadata is supported`; it now decodes. Malformed byte-swapped containers now name their byte order. - Remaining proof gaps: fixtures are synthetic. No real `lipo`-produced `FAT_CIGAM` binary was decoded — this host is Windows, so `npm run verify:apple-dispatch` could not run. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npx vitest run --project adapters src/native/AppleDispatchMetadata.test.ts` → 18 passed - `npm run check:fast` → 0 errors, 98 warnings (unchanged from base; no new warnings added) - `git stash push -- src/native/AppleDispatchMetadata.ts` then re-running the test file → 10 failed / 8 passed, with the three little-endian rejection cases failing on `Only little-endian 64-bit Mach-O metadata is supported` — the regression is driven by the new tests, not merely asserted. - `npm test` → 48 files failed / 304 passed with the change versus 54 failed / 299 passed at base. Failures are Hopper, macOS, `hdiutil`, and browser suites that require hosts unavailable here and fail identically at base; this Windows host is not a green lane for the full suite. ## Compatibility, safety, and release - Breaking changes or migration steps: none. The only observable change is diagnostic text on malformed byte-swapped fat headers and successful decode of previously rejected byte-swapped containers. - Real Hopper/Ghidra, browser, or OS coverage: none performed. `scripts/verify-apple-dispatch.mjs` is intentionally untouched — adding a `lipo`-produced universal fixture there could not be exercised on this Windows host, and shipping an unexercised real-toolchain claim would assert a support guarantee this PR cannot back. That fixture remains a follow-up for a macOS lane. - Package or release metadata impact: none. `CHANGELOG.md` is release-please generated from PR titles and is not edited by hand. - Security, privacy, process, or containment review: no new I/O, no new inputs, no process or permission surface. Slice bounds checks are unchanged and still run for both byte orders. ## Review checklist - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:14:31 +00:00