morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #5501] [MERGED] fix(native): admit universal Mach-O files without a host slice #6942

New issue

Closed

opened 2026-11-01 21:44:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:44:13 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5501
Author: @mhamzasakhi
Created: 10/20/2026
Status: ✅ Merged
Merged: 10/22/2026
Merged by: @morluto

Base: main ← Head: fix/list-architectures-foreign-universal

📝 Commits (6)

8b5fe34 fix(native): admit universal Mach-O files without a host slice
9cae7db fix(native): select the first valid foreign FAT slice
d6d225b fix(native): select foreign FAT slices from every declaration
c89ee59 fix(native): admit only unambiguous foreign FAT families
d019e9b Merge main into fix/list-architectures-foreign-universal
0409a44 fix(ghidra): decide FAT decompiler compatibility from validated families

📊 Changes

6 files changed (+289 additions, -76 deletions)

View changed files

📝 src/application/BinaryTargetResolver.ts (+3 -2)
📝 src/domain/binaryTarget.test.ts (+51 -21)
📝 src/domain/binaryTarget.ts (+96 -29)
📝 src/ghidra/GhidraInstallation.test.ts (+26 -0)
📝 src/ghidra/GhidraInstallation.ts (+23 -24)
➕ tests/boundary/cli/foreignUniversalMachoCli.test.ts (+90 -0)

📄 Description

Summary

Fixes #5404. A valid universal (FAT) Mach-O whose slices are all foreign to
the host was rejected at target admission with target_unavailable: "FAT
binary has no host-compatible architecture". This happened before any
operation ran, so list-architectures could not inventory, for example, an
x86_64-only container on an arm64 Mac. A thin foreign image (such as a thin
arm64 Mach-O on an x64 host) was already admitted.

Problem and expected behavior

parseFatMachO in src/domain/binaryTarget.ts selected only the host's slice
and returned an error when the table had none.

Expected: a foreign-only container is admitted like its thin equivalent, and
list-architectures returns its slices with container-relative coordinates
and a mach-o-universal subject.

Change and scope

FAT slice selection still prefers the host's slice, and a malformed host slice
is still an error. When the table declares no host slice, it selects the
first declared supported slice, in table order, that can hold its Mach-O
header, among families declared exactly once. A foreign family declared more
than once, such as arm64 and arm64e, has no provider-neutral choice: IDA uses
the recorded slice, while Hopper applies its own subtype preference. Such
containers stay rejected, as on main. available_architectures still lists
each family once.
Table validation and slice selection have separate owners in
binaryTarget.ts. validatedFatDeclarations checks the table and every slice
identity once, including main's thin-header CPU validation from #5542.
Selection builds on it, and so does the new readFatArchitectureFamilies,
which returns the validated families without selecting a slice. A table declaring no supported CPU type is still rejected, with
"FAT binary declares no supported Mach-O architecture".
Removed the downstream workaround in GhidraInstallation.ts. It re-parsed
the decompiler header under every other host architecture to recover the
architecture table that the rejection hid. When selection fails, the probe
now reads the validated families and reports incompatible if they exclude the
host. That covers ambiguous foreign families such as arm64 and arm64e. A
malformed host slice stays unknown. The existing
"Ghidra FAT native decompiler admission" cases pass unchanged.
Updated the selection comments in binaryTarget.ts and
BinaryTargetResolver.ts.
Not included: no caller-selected slice option. Containers that include a host
slice select exactly as before.

Contribution scope: an ordinary static format-admission fix within existing
boundaries; a maintainer scope decision is not applicable.

Contract and boundary impact

Semantic owner and earliest changed stage: parseExecutableHeader FAT slice selection (target admission).
Contribution scope fit and linked maintainer decision: in scope; not applicable.
CLI and MCP/tool-catalog contract: no schema change. Foreign-only FAT targets that previously failed with target_unavailable are now admitted.
Provider, bridge, target-format, or platform compatibility: deep-analysis providers receive the selected foreign slice as universalSlice. This is the same slice range they already analyze for a FAT that has a host slice.
Evidence, artifact, provenance, or reconstruction contract: the subject stays mach-o-universal. architecture reports the selected slice, and available_architectures lists the full table.
Target selection, authority, upstream behavior, and cleanup details: no new execution or mutation. Foreign thin images were already admitted.
Generated metadata, package, or installation impact: none.
Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated:
- tests/boundary/cli/foreignUniversalMachoCli.test.ts (new, public CLI):
  - On macOS: compiles tests/conformance/c/fixture.c for the non-host architecture with xcrun clang, wraps it with lipo -create, and requires list-architectures to exit 0 with exactly that one slice, a positive container offset, exhaustive coverage and a mach-o-universal subject.
  - On other hosts: requires a foreign-only FAT to pass admission and reach the macOS-only lipo refusal (capability_unavailable, native-macos) rather than target_unavailable.
- src/domain/binaryTarget.test.ts: the "rejects FAT files without a host-compatible slice" case is replaced by first-valid-slice selection:
  - a one-slice and a two-slice foreign table
  - a foreign family declared twice, valid or with an undersized first declaration, is rejected; a unique foreign family beside it is selected (review follow-ups)
  - an unsupported-CPU-only table, added to the rejection table
- src/ghidra/GhidraInstallation.test.ts: a foreign-only decompiler whose first slice is undersized stays unavailable, as does one declaring a foreign family twice, whether or not the first declaration is undersized (review follow-ups).
- The FAT selection unit tests moved into their own describe block.
Base reproduction (executed): with only src/domain/binaryTarget.ts reverted, the new non-macOS CLI case fails, receiving code: "target_unavailable".
End-to-end on Linux x64 with Ghidra 12.1.4 and JDK 21 (executed):
- Input: the C fixture cross-compiled to a real arm64 macOS executable and wrapped in a one-slice FAT container (slice offset 16384).
- Command: rea decompile <universal> _main --provider ghidra --json.
- On main: target_unavailable, "FAT binary has no host-compatible x64 architecture".
- With this change: subject mach-o-universal, architecture arm64, and pseudo-code from the arm64 slice:
```
bool entry(void)
{
  int iVar1;
  iVar1 = _rea_entry();
  return iVar1 != 0xc;
}
```
Remaining proof gaps:
- The macOS CLI case, including real lipo output, was not run; it is skipped on Linux.
- Hopper and IDA were not run with a foreign-only container.
- The Linux end-to-end container was assembled from a zig cc arm64 build, not by lipo.
Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Run on Ubuntu 24.04 x64 with Node 24.18.0 and npm 11.16.0:

After merging main, npm run test:focused -- with these 9 files plus src/hopper/HopperMachOImage.test.ts, src/artifacts/apple/machoContainer.test.ts and src/artifacts/inventory/ArtifactGraphConstruction.test.ts: 278 passed, 6 skipped (macOS only), 0 failed. Each review follow-up case fails against the parser of the commit before it.
- src/domain/binaryTarget.test.ts
- src/ghidra/GhidraInstallation.test.ts
- src/ida/IdaWorkspaceThinning.test.ts
- tests/composition/idaProvider.test.ts
- tests/composition/directAnalysisFactories.test.ts
- tests/boundary/rust/rustBinaryImage.test.ts
- tests/boundary/cli/foreignUniversalMachoCli.test.ts
- tests/boundary/cli/nativeMachoEchoedPathCli.test.ts
- tests/boundary/providers/native/nativeProvider.test.ts
npm run test:focused -- tests/boundary/cli/foreignUniversalMachoCli.test.ts: 1 passed, 1 skipped. With the fix reverted: 1 failed (target_unavailable).
npm run check:fast: passed.
rea decompile with Ghidra on the arm64-only universal executable: shown above.

Compatibility, safety, and release

Breaking changes or migration steps: none. Previously refused inputs are now admitted, and inputs with a host slice are unchanged.
Real Hopper/Ghidra, browser, or OS coverage: Ghidra 12.1.4 on Linux x64 (above). macOS, Hopper and IDA not run.
Package or release metadata impact: none.
Security, privacy, process, or containment review: no new execution, network access, or mutation. Foreign thin images were already admitted with the same providers.

Author readiness

Human owner: @mhamzasakhi

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5501 **Author:** [@mhamzasakhi](https://github.com/mhamzasakhi) **Created:** 10/20/2026 **Status:** ✅ Merged **Merged:** 10/22/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/list-architectures-foreign-universal` --- ### 📝 Commits (6) - [`8b5fe34`](https://github.com/morluto/rea/commit/8b5fe34987e204b0c2cd437d30fc60d211f4bfc9) fix(native): admit universal Mach-O files without a host slice - [`9cae7db`](https://github.com/morluto/rea/commit/9cae7db44de473eadc841a19183276880581adb3) fix(native): select the first valid foreign FAT slice - [`d6d225b`](https://github.com/morluto/rea/commit/d6d225b7ddd9a092d41c0547a1425bd4418af67f) fix(native): select foreign FAT slices from every declaration - [`c89ee59`](https://github.com/morluto/rea/commit/c89ee5949fb00a556a5a7df893e107d26ff6e1b4) fix(native): admit only unambiguous foreign FAT families - [`d019e9b`](https://github.com/morluto/rea/commit/d019e9b16ec0e0060c3cae3d0b3408d1297a54ff) Merge main into fix/list-architectures-foreign-universal - [`0409a44`](https://github.com/morluto/rea/commit/0409a44cfa834c5c2c4aa82525292989135d34d1) fix(ghidra): decide FAT decompiler compatibility from validated families ### 📊 Changes **6 files changed** (+289 additions, -76 deletions) <details> <summary>View changed files</summary> 📝 `src/application/BinaryTargetResolver.ts` (+3 -2) 📝 `src/domain/binaryTarget.test.ts` (+51 -21) 📝 `src/domain/binaryTarget.ts` (+96 -29) 📝 `src/ghidra/GhidraInstallation.test.ts` (+26 -0) 📝 `src/ghidra/GhidraInstallation.ts` (+23 -24) ➕ `tests/boundary/cli/foreignUniversalMachoCli.test.ts` (+90 -0) </details> ### 📄 Description ## Summary Fixes #5404. A valid universal (FAT) Mach-O whose slices are all foreign to the host was rejected at target admission with `target_unavailable`: "FAT binary has no host-compatible <arch> architecture". This happened before any operation ran, so `list-architectures` could not inventory, for example, an x86_64-only container on an arm64 Mac. A thin foreign image (such as a thin arm64 Mach-O on an x64 host) was already admitted. ## Problem and expected behavior `parseFatMachO` in `src/domain/binaryTarget.ts` selected only the host's slice and returned an error when the table had none. Expected: a foreign-only container is admitted like its thin equivalent, and `list-architectures` returns its slices with container-relative coordinates and a `mach-o-universal` subject. ## Change and scope - FAT slice selection still prefers the host's slice, and a malformed host slice is still an error. When the table declares no host slice, it selects the first declared supported slice, in table order, that can hold its Mach-O header, among families declared exactly once. A foreign family declared more than once, such as arm64 and arm64e, has no provider-neutral choice: IDA uses the recorded slice, while Hopper applies its own subtype preference. Such containers stay rejected, as on `main`. `available_architectures` still lists each family once. - Table validation and slice selection have separate owners in `binaryTarget.ts`. `validatedFatDeclarations` checks the table and every slice identity once, including main's thin-header CPU validation from #5542. Selection builds on it, and so does the new `readFatArchitectureFamilies`, which returns the validated families without selecting a slice. A table declaring no supported CPU type is still rejected, with "FAT binary declares no supported Mach-O architecture". - Removed the downstream workaround in `GhidraInstallation.ts`. It re-parsed the decompiler header under every other host architecture to recover the architecture table that the rejection hid. When selection fails, the probe now reads the validated families and reports `incompatible` if they exclude the host. That covers ambiguous foreign families such as arm64 and arm64e. A malformed host slice stays `unknown`. The existing "Ghidra FAT native decompiler admission" cases pass unchanged. - Updated the selection comments in `binaryTarget.ts` and `BinaryTargetResolver.ts`. - Not included: no caller-selected slice option. Containers that include a host slice select exactly as before. Contribution scope: an ordinary static format-admission fix within existing boundaries; a maintainer scope decision is not applicable. ## Contract and boundary impact - Semantic owner and earliest changed stage: `parseExecutableHeader` FAT slice selection (target admission). - Contribution scope fit and linked maintainer decision: in scope; not applicable. - CLI and MCP/tool-catalog contract: no schema change. Foreign-only FAT targets that previously failed with `target_unavailable` are now admitted. - Provider, bridge, target-format, or platform compatibility: deep-analysis providers receive the selected foreign slice as `universalSlice`. This is the same slice range they already analyze for a FAT that has a host slice. - Evidence, artifact, provenance, or reconstruction contract: the subject stays `mach-o-universal`. `architecture` reports the selected slice, and `available_architectures` lists the full table. - Target selection, authority, upstream behavior, and cleanup details: no new execution or mutation. Foreign thin images were already admitted. - Generated metadata, package, or installation impact: none. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - `tests/boundary/cli/foreignUniversalMachoCli.test.ts` (new, public CLI): - On macOS: compiles `tests/conformance/c/fixture.c` for the non-host architecture with `xcrun clang`, wraps it with `lipo -create`, and requires `list-architectures` to exit 0 with exactly that one slice, a positive container offset, exhaustive coverage and a `mach-o-universal` subject. - On other hosts: requires a foreign-only FAT to pass admission and reach the macOS-only `lipo` refusal (`capability_unavailable`, `native-macos`) rather than `target_unavailable`. - `src/domain/binaryTarget.test.ts`: the "rejects FAT files without a host-compatible slice" case is replaced by first-valid-slice selection: - a one-slice and a two-slice foreign table - a foreign family declared twice, valid or with an undersized first declaration, is rejected; a unique foreign family beside it is selected (review follow-ups) - an unsupported-CPU-only table, added to the rejection table - `src/ghidra/GhidraInstallation.test.ts`: a foreign-only decompiler whose first slice is undersized stays `unavailable`, as does one declaring a foreign family twice, whether or not the first declaration is undersized (review follow-ups). - The FAT selection unit tests moved into their own `describe` block. - Base reproduction (executed): with only `src/domain/binaryTarget.ts` reverted, the new non-macOS CLI case fails, receiving `code: "target_unavailable"`. - End-to-end on Linux x64 with Ghidra 12.1.4 and JDK 21 (executed): - Input: the C fixture cross-compiled to a real arm64 macOS executable and wrapped in a one-slice FAT container (slice offset 16384). - Command: `rea decompile <universal> _main --provider ghidra --json`. - On `main`: `target_unavailable`, "FAT binary has no host-compatible x64 architecture". - With this change: subject `mach-o-universal`, architecture `arm64`, and pseudo-code from the arm64 slice: ```c bool entry(void) { int iVar1; iVar1 = _rea_entry(); return iVar1 != 0xc; } ``` - Remaining proof gaps: - The macOS CLI case, including real `lipo` output, was not run; it is skipped on Linux. - Hopper and IDA were not run with a foreign-only container. - The Linux end-to-end container was assembled from a `zig cc` arm64 build, not by `lipo`. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Run on Ubuntu 24.04 x64 with Node 24.18.0 and npm 11.16.0: - After merging main, `npm run test:focused --` with these 9 files plus `src/hopper/HopperMachOImage.test.ts`, `src/artifacts/apple/machoContainer.test.ts` and `src/artifacts/inventory/ArtifactGraphConstruction.test.ts`: 278 passed, 6 skipped (macOS only), 0 failed. Each review follow-up case fails against the parser of the commit before it. - `src/domain/binaryTarget.test.ts` - `src/ghidra/GhidraInstallation.test.ts` - `src/ida/IdaWorkspaceThinning.test.ts` - `tests/composition/idaProvider.test.ts` - `tests/composition/directAnalysisFactories.test.ts` - `tests/boundary/rust/rustBinaryImage.test.ts` - `tests/boundary/cli/foreignUniversalMachoCli.test.ts` - `tests/boundary/cli/nativeMachoEchoedPathCli.test.ts` - `tests/boundary/providers/native/nativeProvider.test.ts` - `npm run test:focused -- tests/boundary/cli/foreignUniversalMachoCli.test.ts`: 1 passed, 1 skipped. With the fix reverted: 1 failed (`target_unavailable`). - `npm run check:fast`: passed. - `rea decompile` with Ghidra on the arm64-only universal executable: shown above. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Previously refused inputs are now admitted, and inputs with a host slice are unchanged. - Real Hopper/Ghidra, browser, or OS coverage: Ghidra 12.1.4 on Linux x64 (above). macOS, Hopper and IDA not run. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new execution, network access, or mutation. Foreign thin images were already admitted with the same providers. ## Author readiness Human owner: @mhamzasakhi - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:44:13 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:48:01 +00:00

[PR #6942] fix(artifacts): retain completed Rust, BEAM, and pyc facts through finalization #7463

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6942

No description provided.

Rows
Columns