[GH-ISSUE #4102] [Feature] Let inspect_go_binary / inspect_rust_binary select a slice of a universal Mach-O without lipo #1619

New issue

Closed

opened 2026-10-16 21:08:04 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-16 21:08:04 +00:00

Owner

Originally created by @Rinkia on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4102

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues and pull requests for a related proposal (lipo, universal Mach-O slice, fat Mach-O, "select a thin slice"). Related but different: #3282 (Ghidra provider importing a selected slice), #1502/#3072/#3389 (slice selection in other providers), PR #2458 (kept the Go thin-slice failure for confirmed universal containers).
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Native analysis or decompilation

Current limitation

For a universal (fat) Mach-O, inspect_go_binary returns unsupported_target:

Unsupported target for inspect_go_binary at …/gh: Universal/fat Mach-O containers require selecting a thin slice
remediation: Select a target supported by this operation or choose an operation supporting the reported target format.

docs/go-binaries.md and docs/rust-binaries.md say the same thing ("Select a thin slice first"). On a Linux host, though, REA offers no way to select one:

Neither inspect-go-binary nor inspect-rust-binary takes an architecture or slice argument.
extract-artifact on the same file fails because it relies on lipo, which a Linux host (including the REA Docker image) doesn't have: artifact_operation_failed / unavailable, detail "lipo could not enumerate universal Mach-O slices". The message doesn't say that lipo is missing, and the remediation is generic ("Check artifact support and required tools").

REA already parses the FAT table structurally. readFatContainerHeader and classifyFatContainerIdentity (src/domain/apple/machoContainer.ts) are what GoBinaryImage.ts:215-235 uses to confirm the container before refusing it. MachOSliceArtifactReader can also derive slice ranges from the structural Mach-O facts ("Slice range was derived from structural Mach-O facts because lipo omitted it"), but only after lipo has run.

For comparison, go version -m reads the same universal file directly (output below).

Use cases

Many macOS releases ship only as universal binaries (.pkg installers, Homebrew bottles for some tools, app bundles). The GitHub CLI's macOS_universal.pkg is one example; the thin macOS_amd64/macOS_arm64 zips happen to exist for gh, but many projects don't publish them.
Supply-chain and inventory checks on Linux CI or in Docker, where lipo isn't available.

Desired outcome and success criteria

One of these, in order of preference:

inspect_go_binary / inspect_rust_binary accept a universal Mach-O plus an optional architecture selector (e.g. architecture: "arm64"). They read the selected slice using the existing structural FAT parser and record the slice's offset, size and CPU in the result. Without a selector, they either report every slice or fail with a message listing the available architectures.
Or: extract_artifact extracts slices from the structural FAT table when lipo isn't installed, keeping lipo as cross-check evidence when present.

In both cases, when lipo is the missing piece, the error should say so.

Expected interaction with targets

Static reads of the FAT header and the selected slice's bytes. No execution.

Target and scope details (optional)

GitHub CLI v2.102.0 gh_2.102.0_macOS_universal.pkg (sha256 from the release's gh_2.102.0_checksums.txt, verified; release provenance verified with gh attestation verify: SLSA v1, cli/cli deployment.yml @ fc4b137cdef0a6bd28fd461b7cf9c84a5812a8cd = tag v2.102.0). REA can't open the .pkg (extract_artifact: "Artifact format has no extraction reader: pkg"), so usr/local/bin/gh was extracted with bsdtar: a Mach-O universal binary with x86_64 and arm64 slices, sha256 d46fd1b3f6dff765288856d614f1fe46fb7f8940bf6522a11f3ec6ec966f3882.

Verification available so far

Executed on rea-agents@6.5.0 (Linux Docker image). The refusal in src/go/GoBinaryImage.ts and the docs wording are unchanged on main @ 55cef2566 (source-derived; the Rust reader shares the Go image reader's container handling).

Verification details (optional)

$ rea inspect-go-binary …/universal/gh
unsupported_target: Universal/fat Mach-O containers require selecting a thin slice

$ rea extract-artifact …/universal/gh
artifact_operation_failed (unavailable): lipo could not enumerate universal Mach-O slices
$ command -v lipo llvm-lipo   # inside the rea image
(none)

$ go version -m …/universal/gh   # golang:1.27
…/universal/gh: go1.27.1
	path	github.com/cli/cli/v2/cmd/gh
	mod	github.com/cli/cli/v2	v2.102.0
	…

For context, inspect_go_binary matched go version -m exactly on the other 7 single-architecture gh v2.102.0 builds I tested: linux 386/amd64/arm64/armv6, macOS arm64, and windows 386/arm64, covering ELF, PE and thin Mach-O at 32 and 64 bits. Go version, every dependency path/version/sum and all build settings matched. It also matched on gh v1.0.0 (Go 1.15.2, pointer encoding) for linux, macOS and windows amd64, including a => replacement. So the slice gap is the only problem I found.

Alternatives considered

Installing llvm-lipo or cctools in the image. That adds a toolchain dependency for something REA already parses itself.
Asking users to slice the file with external tools. That leaves the Evidence without the parent container's identity.

Possible direction

Reuse readFatContainerHeader to select fat_arch by CPU type or subtype, then pass bytes.subarray(offset, offset + size) to readGoMachoImage (and the Rust equivalent). Record the parent container's sha256 and the slice range in the result so the provenance stays traceable.

References

src/go/GoBinaryImage.ts (fat refusal), src/rust/RustBinaryImage.ts
src/domain/apple/machoContainer.ts (readFatContainerHeader, classifyFatContainerIdentity)
src/artifacts/MachOSliceArtifactReader.ts (lipo-backed slice extraction)
docs/go-binaries.md ("Supported boundary"), docs/rust-binaries.md

Originally created by @Rinkia on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4102 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal (lipo, universal Mach-O slice, fat Mach-O, "select a thin slice"). Related but different: #3282 (Ghidra provider importing a selected slice), #1502/#3072/#3389 (slice selection in other providers), PR #2458 (kept the Go thin-slice failure for confirmed universal containers). - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Native analysis or decompilation ### Current limitation For a universal (fat) Mach-O, `inspect_go_binary` returns `unsupported_target`: ```text Unsupported target for inspect_go_binary at …/gh: Universal/fat Mach-O containers require selecting a thin slice remediation: Select a target supported by this operation or choose an operation supporting the reported target format. ``` `docs/go-binaries.md` and `docs/rust-binaries.md` say the same thing ("Select a thin slice first"). On a Linux host, though, REA offers no way to select one: - Neither `inspect-go-binary` nor `inspect-rust-binary` takes an architecture or slice argument. - `extract-artifact` on the same file fails because it relies on `lipo`, which a Linux host (including the REA Docker image) doesn't have: `artifact_operation_failed` / `unavailable`, detail "lipo could not enumerate universal Mach-O slices". The message doesn't say that `lipo` is missing, and the remediation is generic ("Check artifact support and required tools"). REA already parses the FAT table structurally. `readFatContainerHeader` and `classifyFatContainerIdentity` (`src/domain/apple/machoContainer.ts`) are what `GoBinaryImage.ts:215-235` uses to confirm the container before refusing it. `MachOSliceArtifactReader` can also derive slice ranges from the structural Mach-O facts ("Slice range was derived from structural Mach-O facts because lipo omitted it"), but only after `lipo` has run. For comparison, `go version -m` reads the same universal file directly (output below). ### Use cases - Many macOS releases ship only as universal binaries (`.pkg` installers, Homebrew bottles for some tools, app bundles). The GitHub CLI's `macOS_universal.pkg` is one example; the thin `macOS_amd64`/`macOS_arm64` zips happen to exist for gh, but many projects don't publish them. - Supply-chain and inventory checks on Linux CI or in Docker, where `lipo` isn't available. ### Desired outcome and success criteria One of these, in order of preference: 1. `inspect_go_binary` / `inspect_rust_binary` accept a universal Mach-O plus an optional architecture selector (e.g. `architecture: "arm64"`). They read the selected slice using the existing structural FAT parser and record the slice's offset, size and CPU in the result. Without a selector, they either report every slice or fail with a message listing the available architectures. 2. Or: `extract_artifact` extracts slices from the structural FAT table when `lipo` isn't installed, keeping `lipo` as cross-check evidence when present. In both cases, when `lipo` is the missing piece, the error should say so. ### Expected interaction with targets Static reads of the FAT header and the selected slice's bytes. No execution. ### Target and scope details (optional) GitHub CLI v2.102.0 `gh_2.102.0_macOS_universal.pkg` (sha256 from the release's `gh_2.102.0_checksums.txt`, verified; release provenance verified with `gh attestation verify`: SLSA v1, `cli/cli` `deployment.yml` @ `fc4b137cdef0a6bd28fd461b7cf9c84a5812a8cd` = tag v2.102.0). REA can't open the `.pkg` (`extract_artifact`: "Artifact format has no extraction reader: pkg"), so `usr/local/bin/gh` was extracted with `bsdtar`: a Mach-O universal binary with x86_64 and arm64 slices, sha256 `d46fd1b3f6dff765288856d614f1fe46fb7f8940bf6522a11f3ec6ec966f3882`. ### Verification available so far Executed on `rea-agents@6.5.0` (Linux Docker image). The refusal in `src/go/GoBinaryImage.ts` and the docs wording are unchanged on `main` @ `55cef2566` (source-derived; the Rust reader shares the Go image reader's container handling). ### Verification details (optional) ```text $ rea inspect-go-binary …/universal/gh unsupported_target: Universal/fat Mach-O containers require selecting a thin slice $ rea extract-artifact …/universal/gh artifact_operation_failed (unavailable): lipo could not enumerate universal Mach-O slices $ command -v lipo llvm-lipo # inside the rea image (none) $ go version -m …/universal/gh # golang:1.27 …/universal/gh: go1.27.1 path github.com/cli/cli/v2/cmd/gh mod github.com/cli/cli/v2 v2.102.0 … ``` For context, `inspect_go_binary` matched `go version -m` exactly on the other 7 single-architecture gh v2.102.0 builds I tested: linux 386/amd64/arm64/armv6, macOS arm64, and windows 386/arm64, covering ELF, PE and thin Mach-O at 32 and 64 bits. Go version, every dependency path/version/sum and all build settings matched. It also matched on gh v1.0.0 (Go 1.15.2, pointer encoding) for linux, macOS and windows amd64, including a `=>` replacement. So the slice gap is the only problem I found. ### Alternatives considered - Installing `llvm-lipo` or `cctools` in the image. That adds a toolchain dependency for something REA already parses itself. - Asking users to slice the file with external tools. That leaves the Evidence without the parent container's identity. ### Possible direction Reuse `readFatContainerHeader` to select `fat_arch` by CPU type or subtype, then pass `bytes.subarray(offset, offset + size)` to `readGoMachoImage` (and the Rust equivalent). Record the parent container's sha256 and the slice range in the result so the provenance stays traceable. ### References - `src/go/GoBinaryImage.ts` (fat refusal), `src/rust/RustBinaryImage.ts` - `src/domain/apple/machoContainer.ts` (`readFatContainerHeader`, `classifyFatContainerIdentity`) - `src/artifacts/MachOSliceArtifactReader.ts` (lipo-backed slice extraction) - `docs/go-binaries.md` ("Supported boundary"), `docs/rust-binaries.md`

bsakaguchi commented

2026-10-16 21:08:05 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 16, 2026):

Cross-referencing with #3282 ([Feature] Ghidra provider: support universal (FAT) Mach-O by importing the selected slice). Both propose first-class slice selection from universal FAT Mach-O binaries across inspection tools and providers without requiring external lipo invocation.

Consolidated under Cluster O (Universal Mach-O Direct Slice Selection Across Providers) in tracker #4330.

@matbrgz commented on GitHub (Oct 16, 2026): Cross-referencing with #3282 (`[Feature] Ghidra provider: support universal (FAT) Mach-O by importing the selected slice`). Both propose first-class slice selection from universal FAT Mach-O binaries across inspection tools and providers without requiring external `lipo` invocation. Consolidated under **Cluster O (Universal Mach-O Direct Slice Selection Across Providers)** in tracker #4330.

bsakaguchi closed this issue

2026-10-31 00:31:50 +00:00

bsakaguchi referenced this issue

2026-10-31 00:54:15 +00:00

[PR #1629] [MERGED] fix(ghidra): admit carved MIPS ELF with incomplete section tables #4711