morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #7605] [Bug] inspect-macho aborts on lipo-generated FAT64 input when optional vtool rejects the container #4535

New issue

Closed

opened 2026-10-31 00:50:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:50:34 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7605

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

On macOS arm64, node scripts/rea.mjs inspect-macho "$WORK/fat64" --json exits 1 with execution_failure for a freshly linked arm64 executable wrapped by Apple's lipo -create -fat64. The captured failure identifies vtool, exit 1, with stderr ending in file is not mach-o. REA returns no inspection Evidence and recommends running rea doctor.

list-architectures succeeds on that file and identifies an arm64 Mach-O universal container. All five required native inspection commands succeed directly on the same file. Inspection also succeeds for the identical payload as a thin executable, inside a FAT32 container, and after extraction from FAT64 with lipo -thin arm64. The failure is specific to the tested container/tool interaction.

Reproduction

Start at source commit 65f1c447344a423fa2785847dbcea1e9a9e2454a, with its locked dependencies and built runtime already available. Use Node.js v24.22.1 and the Apple command-line development tools.
From the checkout root, run this script. It creates its own disposable fixtures, isolates HOME/XDG/temp paths for subprocesses, and deletes the fixtures when finished. No executable under inspection is run.

python3 -I - <<'PYREPRO'
import json, os, pathlib, subprocess, tempfile

with tempfile.TemporaryDirectory(prefix="rea-fat64-") as base:
    w = pathlib.Path(base)
    env = {"PATH": os.environ["PATH"]}
    for key, folder in [("HOME", "home"), ("XDG_CONFIG_HOME", "config"),
                        ("XDG_DATA_HOME", "data"), ("XDG_CACHE_HOME", "cache"),
                        ("XDG_STATE_HOME", "state"), ("XDG_RUNTIME_DIR", "runtime"),
                        ("TMPDIR", "temp")]:
        (w / folder).mkdir()
        env[key] = str(w / folder)
    def call(args):
        return subprocess.run(args, env=env, text=True, capture_output=True)
    def inspect(name):
        p = call(["node", "scripts/rea.mjs", "inspect-macho", str(w / name), "--json"])
        j = json.loads(p.stdout)
        print(name, "exit=" + str(p.returncode), j.get("code", "success"))
        if p.returncode:
            print(json.dumps(j["details"]["diagnostics"]))
        return p, j
    (w / "main.c").write_text("int main(void) { return 0; }\n")
    for args in [["clang", "-arch", "arm64", str(w / "main.c"), "-o", str(w / "thin")],
                 ["lipo", "-create", "-fat64", str(w / "thin"), "-output", str(w / "fat64")],
                 ["lipo", "-create", str(w / "thin"), "-output", str(w / "fat32")]]:
        p = call(args)
        assert p.returncode == 0, p.stderr
    p = call(["lipo", "-detailed_info", str(w / "fat64")])
    assert p.returncode == 0 and "0xcafebabf" in p.stdout
    print("lipo: FAT64 arm64 container")
    for tool, flags in [("otool", ["-h", "-l"]), ("nm", ["-gjU", "--no-dyldinfo"]),
                        ("dyld_info", ["-imports"]), ("dyld_info", ["-exports"]),
                        ("dwarfdump", ["--uuid"])]:
        p = call([tool, *flags, str(w / "fat64")])
        print(tool, *flags, "exit=" + str(p.returncode))
        assert p.returncode == 0, p.stderr
    p = call(["node", "scripts/rea.mjs", "list-architectures", str(w / "fat64"), "--json"])
    assert p.returncode == 0, p.stdout
    j = json.loads(p.stdout)
    print("list-architectures:", j["subject"]["format"], j["normalized_result"]["architectures"]["items"][0]["name"])
    for name in ["thin", "fat32"]:
        assert inspect(name)[0].returncode == 0
    for _ in range(2):
        p, j = inspect("fat64")
        assert p.returncode == 1 and j["code"] == "execution_failure"
        assert j["details"]["diagnostics"]["tool"] == "vtool"
    p = call(["lipo", "-thin", "arm64", str(w / "fat64"), "-output", str(w / "extracted")])
    assert p.returncode == 0, p.stderr
    assert (w / "extracted").read_bytes() == (w / "thin").read_bytes()
    assert inspect("extracted")[0].returncode == 0
PYREPRO

The final script was executed once and reproduced the FAT64 inspection failure on both attempts. Earlier independent runs using a manually constructed FAT64 wrapper and a separate lipo-generated wrapper gave the same failure. The two wrappers were byte-identical.

Expected behavior

Return the selected Mach-O image's available metadata for this valid universal container, preserving the container identity and selected-slice offset. If the optional build-metadata reader cannot inspect this container, retain the facts from the required commands and state the reader limitation explicitly.

The native inspection contract advertises Mach-O slices, load commands, build metadata, symbols and provenance. The static-inspection guide describes FAT containers, expressly including one-slice containers. The implementation already derives normalized build metadata from otool and tolerates unavailable vtool or unsupported vtool variant selection. A failure confined to that optional reader should not discard the entire otherwise readable image inspection.

Impact and scope

P2: this bounded input class loses its complete normalized inspection result and receives environment-repair guidance even though the required readers succeed. The tested input is a one-slice plain-arm64 FAT64 executable created by the native linker and lipo. Multi-slice FAT64, arm64e payloads, other toolchains and real-world frequency were not measured.

A verified workaround is lipo -thin arm64 "$WORK/fat64" -output "$WORK/extracted", followed by inspecting the extracted file. The extraction produced bytes identical to the passing original thin executable. This preserves the selected payload's metadata but changes artifact identity and reports offsets relative to the thin copy; retain the original container's architecture inventory separately.

REA version

Exact node scripts/rea.mjs --version output: 6.8.0.
Source commit: 65f1c447344a423fa2785847dbcea1e9a9e2454a.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1, build 26A434; arm64; Apple clang 21.0.0 (clang-2100.3.34.2). Real Apple lipo, otool, nm, dyld_info, dwarfdump and vtool were used. REA reports no uniform stable version flag for vtool; the passing thin control recorded its executable SHA-256 as 2750e40544ec380d7ad549f8adcee65f08b8df0a3d41d7ca584d0d9c326cd9c0. No GUI or deep-analysis provider was launched.

Target and artifact context

A synthetic executable compiled from int main(void) { return 0; }, placed in a one-entry FAT64 container by lipo -create -fat64. The independent fixture's header was 0xcafebabf, with an arm64 slice at offset 16384, size 16840, alignment 16384. FAT32 and thin controls contain the same executable bytes. No user binary or attachment is required.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The final reproduction printed the following summary, with byte-count-only diagnostics omitted here:

lipo: FAT64 arm64 container
otool -h -l exit=0
nm -gjU --no-dyldinfo exit=0
dyld_info -imports exit=0
dyld_info -exports exit=0
dwarfdump --uuid exit=0
list-architectures: mach-o-universal arm64
thin exit=0 success
fat32 exit=0 success
fat64 exit=1 execution_failure
{"tool":"vtool","reason":"nonzero-exit","exit_code":1,"signal":null}
fat64 exit=1 execution_failure
{"tool":"vtool","reason":"nonzero-exit","exit_code":1,"signal":null}
extracted exit=0 success

The failing public CLI output includes details.operation: "inspect_macho", details.provider_id: "native-macos", captured_output.truncated: false, empty stdout, and vtool stderr ending in fat64 file is not mach-o. A separate direct vtool -show-build call on FAT64 also exited 1 with that refusal; the FAT32 control exited 0. Successful thin/FAT32 output includes the limitation vtool output is retained as provenance but only otool build metadata is normalized.

Expected-source locators at this baseline: src/contracts/native/nativeToolContracts.ts:101-104 advertises the operation; docs/native-investigation.md:536-546 describes inspected FAT container identity, including one-slice FAT; src/domain/apple/machoContainer.ts:113-116,159-165 recognizes FAT64 magic. The inspection guide does not explicitly name FAT64. The explicit FAT32/FAT64 discussion in docs/installation.md:911-926 concerns Hopper loading, which was not used as an inspect-macho support guarantee here.

Failure-source locators: src/native/NativeMachoInspection.ts:69-79,132-167 separates required commands from optional vtool but only tolerates a missing capability or skips unsupported variant selection. Lines 374-379 identify vtool output as provenance only. All other optional-command errors return before normalization. src/native/NativeMacOSProvider.ts:1230-1249 exposes this through the native inspection operation. This source reading explains the actual CLI result; it is separate from the executed controls.

Typed outcome or recovery expected

A successful inspection Evidence record identifying the FAT64 input as mach-o-universal, the selected arm64 slice at its original container offset, and normalized facts from the successful required commands. Record the unsupported vtool container reader as an explicit limitation, or otherwise arrange for that helper to read the selected image without losing the source-container identity. Required-tool, I/O, timeout and cancellation failures must retain their meaningful failure semantics. These are proposed restoration checks; no fix was implemented.

Analysis or suspected cause

Source-derived explanation: the optional vtool call receives the universal container directly. On the tested toolchain, that reader rejects FAT64 even though the other required readers accept it. The optional-command branch propagates this nonzero exit as a provider failure before it normalizes the successful required captures. The existing missing-tool fallback and unsupported-variant skip do not cover this input-format refusal. No claim is made about vtool behavior on other releases.

Additional context

Open and closed issues, issue comments, and all PR states were searched for FAT64, vtool, fat_arch_64, the native refusal, and universal/optional inspection combinations. No report with this input and restoration condition was found.

Related closed #187 was fixed by merged #207: it handles missing vtool and explicitly leaves other failures fatal. This report uses a present tool that works on the thin/FAT32 controls, so it does not claim a regression of the missing-tool fix. Merged #3500 already skips unsupported optional vtool selection for arm64e.x1; its exact-architecture closure does not cover this plain-arm64 FAT64 reader refusal.

Closed #674 concerns absent architecture arguments, and closed #6154 concerns corrupt load-command admission. Open #7391 concerns section-name text parsing, and open #7556 concerns otool filename operand handling. Those cases require different restoration checks. This fixture is a lipo-produced executable with successful required readers and a vtool-only container refusal.

No earlier known-good REA release was established. MCP transport and additional FAT64 layouts were not replayed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A valid, admitted one-slice FAT64 executable reproducibly loses its entire static-inspection result because an optional reader refuses the container. Required readers and thin/FAT32 controls succeed. Wider prevalence and severe downstream harm were not established.
Workaround status: available
Workaround basis: Real lipo extraction and inspection of the extracted arm64 payload succeeded; extracted bytes matched the passing thin executable exactly. The workaround changes artifact identity and container coordinates, requiring the original architecture inventory to be retained separately.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7605 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior On macOS arm64, `node scripts/rea.mjs inspect-macho "$WORK/fat64" --json` exits 1 with `execution_failure` for a freshly linked arm64 executable wrapped by Apple's `lipo -create -fat64`. The captured failure identifies `vtool`, exit 1, with stderr ending in `file is not mach-o`. REA returns no inspection Evidence and recommends running `rea doctor`. `list-architectures` succeeds on that file and identifies an arm64 Mach-O universal container. All five required native inspection commands succeed directly on the same file. Inspection also succeeds for the identical payload as a thin executable, inside a FAT32 container, and after extraction from FAT64 with `lipo -thin arm64`. The failure is specific to the tested container/tool interaction. ### Reproduction 1. Start at source commit `65f1c447344a423fa2785847dbcea1e9a9e2454a`, with its locked dependencies and built runtime already available. Use Node.js v24.22.1 and the Apple command-line development tools. 2. From the checkout root, run this script. It creates its own disposable fixtures, isolates HOME/XDG/temp paths for subprocesses, and deletes the fixtures when finished. No executable under inspection is run. ```sh python3 -I - <<'PYREPRO' import json, os, pathlib, subprocess, tempfile with tempfile.TemporaryDirectory(prefix="rea-fat64-") as base: w = pathlib.Path(base) env = {"PATH": os.environ["PATH"]} for key, folder in [("HOME", "home"), ("XDG_CONFIG_HOME", "config"), ("XDG_DATA_HOME", "data"), ("XDG_CACHE_HOME", "cache"), ("XDG_STATE_HOME", "state"), ("XDG_RUNTIME_DIR", "runtime"), ("TMPDIR", "temp")]: (w / folder).mkdir() env[key] = str(w / folder) def call(args): return subprocess.run(args, env=env, text=True, capture_output=True) def inspect(name): p = call(["node", "scripts/rea.mjs", "inspect-macho", str(w / name), "--json"]) j = json.loads(p.stdout) print(name, "exit=" + str(p.returncode), j.get("code", "success")) if p.returncode: print(json.dumps(j["details"]["diagnostics"])) return p, j (w / "main.c").write_text("int main(void) { return 0; }\n") for args in [["clang", "-arch", "arm64", str(w / "main.c"), "-o", str(w / "thin")], ["lipo", "-create", "-fat64", str(w / "thin"), "-output", str(w / "fat64")], ["lipo", "-create", str(w / "thin"), "-output", str(w / "fat32")]]: p = call(args) assert p.returncode == 0, p.stderr p = call(["lipo", "-detailed_info", str(w / "fat64")]) assert p.returncode == 0 and "0xcafebabf" in p.stdout print("lipo: FAT64 arm64 container") for tool, flags in [("otool", ["-h", "-l"]), ("nm", ["-gjU", "--no-dyldinfo"]), ("dyld_info", ["-imports"]), ("dyld_info", ["-exports"]), ("dwarfdump", ["--uuid"])]: p = call([tool, *flags, str(w / "fat64")]) print(tool, *flags, "exit=" + str(p.returncode)) assert p.returncode == 0, p.stderr p = call(["node", "scripts/rea.mjs", "list-architectures", str(w / "fat64"), "--json"]) assert p.returncode == 0, p.stdout j = json.loads(p.stdout) print("list-architectures:", j["subject"]["format"], j["normalized_result"]["architectures"]["items"][0]["name"]) for name in ["thin", "fat32"]: assert inspect(name)[0].returncode == 0 for _ in range(2): p, j = inspect("fat64") assert p.returncode == 1 and j["code"] == "execution_failure" assert j["details"]["diagnostics"]["tool"] == "vtool" p = call(["lipo", "-thin", "arm64", str(w / "fat64"), "-output", str(w / "extracted")]) assert p.returncode == 0, p.stderr assert (w / "extracted").read_bytes() == (w / "thin").read_bytes() assert inspect("extracted")[0].returncode == 0 PYREPRO ``` 3. The final script was executed once and reproduced the FAT64 inspection failure on both attempts. Earlier independent runs using a manually constructed FAT64 wrapper and a separate lipo-generated wrapper gave the same failure. The two wrappers were byte-identical. ### Expected behavior Return the selected Mach-O image's available metadata for this valid universal container, preserving the container identity and selected-slice offset. If the optional build-metadata reader cannot inspect this container, retain the facts from the required commands and state the reader limitation explicitly. The native inspection contract advertises Mach-O slices, load commands, build metadata, symbols and provenance. The static-inspection guide describes FAT containers, expressly including one-slice containers. The implementation already derives normalized build metadata from otool and tolerates unavailable vtool or unsupported vtool variant selection. A failure confined to that optional reader should not discard the entire otherwise readable image inspection. ### Impact and scope P2: this bounded input class loses its complete normalized inspection result and receives environment-repair guidance even though the required readers succeed. The tested input is a one-slice plain-arm64 FAT64 executable created by the native linker and lipo. Multi-slice FAT64, arm64e payloads, other toolchains and real-world frequency were not measured. A verified workaround is `lipo -thin arm64 "$WORK/fat64" -output "$WORK/extracted"`, followed by inspecting the extracted file. The extraction produced bytes identical to the passing original thin executable. This preserves the selected payload's metadata but changes artifact identity and reports offsets relative to the thin copy; retain the original container's architecture inventory separately. ### REA version Exact `node scripts/rea.mjs --version` output: `6.8.0`. Source commit: `65f1c447344a423fa2785847dbcea1e9a9e2454a`. ### Runtime and platform Node.js v24.22.1; macOS 27.0.1, build 26A434; arm64; Apple clang 21.0.0 (clang-2100.3.34.2). Real Apple lipo, otool, nm, dyld_info, dwarfdump and vtool were used. REA reports no uniform stable version flag for vtool; the passing thin control recorded its executable SHA-256 as `2750e40544ec380d7ad549f8adcee65f08b8df0a3d41d7ca584d0d9c326cd9c0`. No GUI or deep-analysis provider was launched. ### Target and artifact context A synthetic executable compiled from `int main(void) { return 0; }`, placed in a one-entry FAT64 container by `lipo -create -fat64`. The independent fixture's header was `0xcafebabf`, with an arm64 slice at offset 16384, size 16840, alignment 16384. FAT32 and thin controls contain the same executable bytes. No user binary or attachment is required. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence The final reproduction printed the following summary, with byte-count-only diagnostics omitted here: ```text lipo: FAT64 arm64 container otool -h -l exit=0 nm -gjU --no-dyldinfo exit=0 dyld_info -imports exit=0 dyld_info -exports exit=0 dwarfdump --uuid exit=0 list-architectures: mach-o-universal arm64 thin exit=0 success fat32 exit=0 success fat64 exit=1 execution_failure {"tool":"vtool","reason":"nonzero-exit","exit_code":1,"signal":null} fat64 exit=1 execution_failure {"tool":"vtool","reason":"nonzero-exit","exit_code":1,"signal":null} extracted exit=0 success ``` The failing public CLI output includes `details.operation: "inspect_macho"`, `details.provider_id: "native-macos"`, `captured_output.truncated: false`, empty stdout, and vtool stderr ending in `fat64 file is not mach-o`. A separate direct `vtool -show-build` call on FAT64 also exited 1 with that refusal; the FAT32 control exited 0. Successful thin/FAT32 output includes the limitation `vtool output is retained as provenance but only otool build metadata is normalized.` Expected-source locators at this baseline: `src/contracts/native/nativeToolContracts.ts:101-104` advertises the operation; `docs/native-investigation.md:536-546` describes inspected FAT container identity, including one-slice FAT; `src/domain/apple/machoContainer.ts:113-116,159-165` recognizes FAT64 magic. The inspection guide does not explicitly name FAT64. The explicit FAT32/FAT64 discussion in `docs/installation.md:911-926` concerns Hopper loading, which was not used as an inspect-macho support guarantee here. Failure-source locators: `src/native/NativeMachoInspection.ts:69-79,132-167` separates required commands from optional vtool but only tolerates a missing capability or skips unsupported variant selection. Lines 374-379 identify vtool output as provenance only. All other optional-command errors return before normalization. `src/native/NativeMacOSProvider.ts:1230-1249` exposes this through the native inspection operation. This source reading explains the actual CLI result; it is separate from the executed controls. ### Typed outcome or recovery expected A successful inspection Evidence record identifying the FAT64 input as `mach-o-universal`, the selected arm64 slice at its original container offset, and normalized facts from the successful required commands. Record the unsupported vtool container reader as an explicit limitation, or otherwise arrange for that helper to read the selected image without losing the source-container identity. Required-tool, I/O, timeout and cancellation failures must retain their meaningful failure semantics. These are proposed restoration checks; no fix was implemented. ### Analysis or suspected cause Source-derived explanation: the optional vtool call receives the universal container directly. On the tested toolchain, that reader rejects FAT64 even though the other required readers accept it. The optional-command branch propagates this nonzero exit as a provider failure before it normalizes the successful required captures. The existing missing-tool fallback and unsupported-variant skip do not cover this input-format refusal. No claim is made about vtool behavior on other releases. ### Additional context Open and closed issues, issue comments, and all PR states were searched for FAT64, vtool, fat_arch_64, the native refusal, and universal/optional inspection combinations. No report with this input and restoration condition was found. Related closed [#187](https://github.com/morluto/rea/issues/187) was fixed by merged [#207](https://github.com/morluto/rea/pull/207): it handles missing vtool and explicitly leaves other failures fatal. This report uses a present tool that works on the thin/FAT32 controls, so it does not claim a regression of the missing-tool fix. Merged [#3500](https://github.com/morluto/rea/pull/3500) already skips unsupported optional vtool selection for arm64e.x1; its exact-architecture closure does not cover this plain-arm64 FAT64 reader refusal. Closed [#674](https://github.com/morluto/rea/issues/674) concerns absent architecture arguments, and closed [#6154](https://github.com/morluto/rea/issues/6154) concerns corrupt load-command admission. Open [#7391](https://github.com/morluto/rea/issues/7391) concerns section-name text parsing, and open [#7556](https://github.com/morluto/rea/issues/7556) concerns otool filename operand handling. Those cases require different restoration checks. This fixture is a lipo-produced executable with successful required readers and a vtool-only container refusal. No earlier known-good REA release was established. MCP transport and additional FAT64 layouts were not replayed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A valid, admitted one-slice FAT64 executable reproducibly loses its entire static-inspection result because an optional reader refuses the container. Required readers and thin/FAT32 controls succeed. Wider prevalence and severe downstream harm were not established. - Workaround status: available - Workaround basis: Real lipo extraction and inspection of the extracted arm64 payload succeeded; extracted bytes matched the passing thin executable exactly. The workaround changes artifact identity and container coordinates, requiring the original architecture inventory to be retained separately.

bsakaguchi closed this issue

2026-10-31 00:50:34 +00:00

bsakaguchi referenced this issue

2026-10-31 01:13:20 +00:00

[PR #4535] [CLOSED] fix(rizin): disable color by default for one-shot commands #6292