morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1168] [Bug] `inspect-artifact` reports valid arm64e universal binaries as corrupt #412

New issue

Closed

opened 2026-10-15 18:24:51 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:24:51 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 9, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1168

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

inspect-artifact fails with integrity_mismatch ("Artifact is invalid or has changed. Get a fresh copy and try again.") on valid, unmodified universal Mach-O files whose arm64e slice lipo labels with a variant suffix: arm64e.x1 on macOS system binaries such as /bin/ls and /usr/lib/libgmalloc.dylib, and arm64e.v1 on a universal binary freshly built with the current Xcode toolchain. codesign --verify accepts the two system files (the self-built fixture is unsigned), lipo -detailed_info agrees with the raw FAT table for all three, and list-architectures, inspect-signature, and inspect-macho succeed on the system files. The false integrity claim tells the user their file is damaged and blocks artifact inspection of the whole file.

Both commands exit 1. For /bin/ls:

{"code":"artifact_operation_failed","category":"integrity_mismatch","message":"Artifact is invalid or has changed. Get a fresh copy and try again.","details":{"operation":"inspect_artifact","reason":"integrity","detail":"lipo slice metadata disagrees with the Mach-O FAT table: arm64e.x1 (cputype=CPU_TYPE_ARM64, cpusubtype=unknown arm64 cpusubtype, offset=163840, size=88672, alignment=16384)"}}

For the fixture the detail names arm64e.v1 (cputype=CPU_TYPE_ARM64, cpusubtype=CPU_SUBTYPE_ARM64E, offset=16384, ...). lipo -detailed_info reports the same offsets, sizes, and alignments as the raw FAT table, and codesign --verify /bin/ls exits 0. --integrity-policy record-and-continue gives the same error. Controls that succeed: a thinned lipo /bin/ls -thin arm64e copy, and an x86_64 + plain arm64 universal fixture. Reproduced twice per input on the npm release and again on the main build.

Reproduction

System binary, no build needed:

xcrun lipo -detailed_info /bin/ls
codesign --verify /bin/ls && echo signature-ok
npx -y rea-agents@6.1.0 inspect-artifact /bin/ls --json; echo "exit=$?"

Self-built fixture:

cat > demo.c <<'C'
#include <stdio.h>
int main(void) { puts("demo"); return 0; }
C
xcrun clang -O0 -arch x86_64 demo.c -o thin-x86_64
xcrun clang -O0 -arch arm64e demo.c -o thin-arm64e
xcrun lipo -create thin-x86_64 thin-arm64e -output fat-arm64e
xcrun lipo -detailed_info fat-arm64e     # reports the second slice as arm64e.v1
npx -y rea-agents@6.1.0 inspect-artifact ./fat-arm64e --json; echo "exit=$?"

Expected behavior

inspect-artifact help: "Inspect one artifact and return all available observations and next probes"; the tool contract returns "the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline". For a valid, unchanged universal file the result should include every slice, or, if a variant cannot be interpreted, an explicit unsupported-format limitation. An integrity_mismatch should be reserved for files that actually disagree with their own metadata.

Impact and scope

Artifact inspection fails with a false corruption claim for valid universal binaries that ship with macOS and for newly built arm64e universal binaries; the artifact workflow is degraded rather than blocked because the per-slice native metadata commands still return architecture, signature, and Mach-O data for the same files, and app-bundle inventories that did not hit such a slice completed.

Workaround: Inspecting a lipo -thin copy of each slice and the native metadata commands are partial alternatives; neither produces the single artifact graph of the original universal file, and --integrity-policy record-and-continue fails the same way.

REA version

rea-agents@6.1.0 from npm, and main at 089bbf58f3 built locally with npm ci && npm run build:cached.

Runtime and platform

Node v24.21.0, macOS 27.0.1 (26A434), arm64; Xcode 27.0 (27A266a), Command Line Tools 27.0.0.0.1788430756, Apple clang 21.0.0.

Target and artifact context

Universal Mach-O with arm64e.x1 / arm64e.v1 slices (/bin/ls, self-built x86_64 + arm64e)

Interface

CLI

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

Expected source: inspect-artifact --help text and the tool contract in src/contracts/artifactToolContracts.ts.
Failure source: the error documents quoted under Observed behavior.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Typed outcome or recovery expected

Failing now: inspect-artifact /bin/ls --json and the fat-arm64e fixture exit 1 with integrity_mismatch. Restored: both exit 0 and the result covers every slice of the file (or records an explicit limitation for a slice it cannot interpret) without an integrity contradiction. Control: a universal file whose FAT table really disagrees with a slice's offset or size still fails with integrity_mismatch.

Analysis or suspected cause

Source-derived, not verified by a patch: src/artifacts/MachOSliceArtifactReader.ts (lipoMatchesSlice) compares the lipo slice name with the architecture name derived from the FAT entry, and src/artifacts/apple/MachoLoadCommandReader.ts (architectureName) yields arm64e or arm64, never the arm64e.v1 / arm64e.x1 labels lipo prints. Other macOS or Xcode versions and other subtype labels were not tested.

Additional context

Found while exercising typical REA workflows on macOS with rea-agents@6.1.0; each case was reproduced on the npm release and on a local build of main, and re-checked on main at 6a1e558db3 just before filing.

Originally created by @coygeek on GitHub (Oct 9, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1168 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `inspect-artifact` fails with `integrity_mismatch` ("Artifact is invalid or has changed. Get a fresh copy and try again.") on valid, unmodified universal Mach-O files whose arm64e slice `lipo` labels with a variant suffix: `arm64e.x1` on macOS system binaries such as `/bin/ls` and `/usr/lib/libgmalloc.dylib`, and `arm64e.v1` on a universal binary freshly built with the current Xcode toolchain. `codesign --verify` accepts the two system files (the self-built fixture is unsigned), `lipo -detailed_info` agrees with the raw FAT table for all three, and `list-architectures`, `inspect-signature`, and `inspect-macho` succeed on the system files. The false integrity claim tells the user their file is damaged and blocks artifact inspection of the whole file. Both commands exit 1. For `/bin/ls`: ```json {"code":"artifact_operation_failed","category":"integrity_mismatch","message":"Artifact is invalid or has changed. Get a fresh copy and try again.","details":{"operation":"inspect_artifact","reason":"integrity","detail":"lipo slice metadata disagrees with the Mach-O FAT table: arm64e.x1 (cputype=CPU_TYPE_ARM64, cpusubtype=unknown arm64 cpusubtype, offset=163840, size=88672, alignment=16384)"}} ``` For the fixture the detail names `arm64e.v1 (cputype=CPU_TYPE_ARM64, cpusubtype=CPU_SUBTYPE_ARM64E, offset=16384, ...)`. `lipo -detailed_info` reports the same offsets, sizes, and alignments as the raw FAT table, and `codesign --verify /bin/ls` exits 0. `--integrity-policy record-and-continue` gives the same error. Controls that succeed: a thinned `lipo /bin/ls -thin arm64e` copy, and an x86_64 + plain arm64 universal fixture. Reproduced twice per input on the npm release and again on the `main` build. ### Reproduction System binary, no build needed: ```sh xcrun lipo -detailed_info /bin/ls codesign --verify /bin/ls && echo signature-ok npx -y rea-agents@6.1.0 inspect-artifact /bin/ls --json; echo "exit=$?" ``` Self-built fixture: ```sh cat > demo.c <<'C' #include <stdio.h> int main(void) { puts("demo"); return 0; } C xcrun clang -O0 -arch x86_64 demo.c -o thin-x86_64 xcrun clang -O0 -arch arm64e demo.c -o thin-arm64e xcrun lipo -create thin-x86_64 thin-arm64e -output fat-arm64e xcrun lipo -detailed_info fat-arm64e # reports the second slice as arm64e.v1 npx -y rea-agents@6.1.0 inspect-artifact ./fat-arm64e --json; echo "exit=$?" ``` ### Expected behavior `inspect-artifact` help: "Inspect one artifact and return all available observations and next probes"; the tool contract returns "the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline". For a valid, unchanged universal file the result should include every slice, or, if a variant cannot be interpreted, an explicit unsupported-format limitation. An `integrity_mismatch` should be reserved for files that actually disagree with their own metadata. ### Impact and scope Artifact inspection fails with a false corruption claim for valid universal binaries that ship with macOS and for newly built arm64e universal binaries; the artifact workflow is degraded rather than blocked because the per-slice native metadata commands still return architecture, signature, and Mach-O data for the same files, and app-bundle inventories that did not hit such a slice completed. Workaround: Inspecting a `lipo -thin` copy of each slice and the native metadata commands are partial alternatives; neither produces the single artifact graph of the original universal file, and `--integrity-policy record-and-continue` fails the same way. ### REA version `rea-agents@6.1.0` from npm, and `main` at 089bbf58f3561e3f465c544bbcf8c307287b1256 built locally with `npm ci && npm run build:cached`. ### Runtime and platform Node v24.21.0, macOS 27.0.1 (26A434), arm64; Xcode 27.0 (27A266a), Command Line Tools 27.0.0.0.1788430756, Apple clang 21.0.0. ### Target and artifact context Universal Mach-O with `arm64e.x1` / `arm64e.v1` slices (`/bin/ls`, self-built x86_64 + arm64e) ### Interface CLI ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence - Expected source: `inspect-artifact --help` text and the tool contract in `src/contracts/artifactToolContracts.ts`. - Failure source: the error documents quoted under Observed behavior. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete ### Typed outcome or recovery expected Failing now: `inspect-artifact /bin/ls --json` and the `fat-arm64e` fixture exit 1 with `integrity_mismatch`. Restored: both exit 0 and the result covers every slice of the file (or records an explicit limitation for a slice it cannot interpret) without an integrity contradiction. Control: a universal file whose FAT table really disagrees with a slice's offset or size still fails with `integrity_mismatch`. ### Analysis or suspected cause Source-derived, not verified by a patch: `src/artifacts/MachOSliceArtifactReader.ts` (`lipoMatchesSlice`) compares the `lipo` slice name with the architecture name derived from the FAT entry, and `src/artifacts/apple/MachoLoadCommandReader.ts` (`architectureName`) yields `arm64e` or `arm64`, never the `arm64e.v1` / `arm64e.x1` labels `lipo` prints. Other macOS or Xcode versions and other subtype labels were not tested. ### Additional context Found while exercising typical REA workflows on macOS with `rea-agents@6.1.0`; each case was reproduced on the npm release and on a local build of `main`, and re-checked on `main` at 6a1e558db3ec2b69561e91ea817f1c2065dfc676 just before filing.

bsakaguchi

2026-10-15 18:24:51 +00:00

closed this issue
added the
area: native

bug
labels

bsakaguchi commented

2026-10-15 18:24:52 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 9, 2026):

Reproduced on main with the self-built fixture: lipo labels the slice arm64e.v1 (PTR_AUTH_VERSION USERSPACE 1) and inspect-artifact reports the false integrity mismatch. (On this Mac's Command Line Tools, /bin/ls is labeled plain arm64e, so that case reproduces only with newer toolchains.) I'm preparing a fix.

@kaoru0822-kitauji commented on GitHub (Oct 9, 2026): Reproduced on `main` with the self-built fixture: `lipo` labels the slice `arm64e.v1` (`PTR_AUTH_VERSION USERSPACE 1`) and `inspect-artifact` reports the false integrity mismatch. (On this Mac's Command Line Tools, `/bin/ls` is labeled plain `arm64e`, so that case reproduces only with newer toolchains.) I'm preparing a fix.

bsakaguchi commented

2026-10-15 18:24:53 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 9, 2026):

fix(artifacts): arm64e.x1 system binaries still fail artifact integrity checks

Summary

The arm64e.x1 case from the existing report still fails on the tested main snapshot. A valid signed universal system binary is rejected as corrupt, while its thinned arm64e slice and the native metadata commands can be inspected. This is follow-up evidence for the existing issue, rather than a new issue identity.

Runtime or environment

REA npm release 6.1.0 and main source revision f0da05f67fff1411cf2f7a48a752ee7b348c71ff, built with npm ci and npm run build:cached; Node.js 26.10.0, macOS 27.0.1, arm64, with Xcode Command Line Tools. Target: the public system binary /bin/ls whose FAT inventory includes a slice labeled arm64e.x1. Interface: CLI. The main-build commands below require a checkout built at that revision.

Steps to reproduce

On the stated runtime, run the following read-only commands against the system binary. Set rea_checkout to the built checkout's absolute directory.

xcrun lipo -detailed_info /bin/ls
codesign --verify /bin/ls
node "$rea_checkout/scripts/rea.mjs" inspect-artifact /bin/ls --json

Run the thinned-copy control in a disposable directory.

fixture_dir="$(mktemp -d)"
xcrun lipo /bin/ls -thin arm64e -output "$fixture_dir/ls-arm64e"
node "$rea_checkout/scripts/rea.mjs" inspect-artifact "$fixture_dir/ls-arm64e" --json

The universal command was also executed with the pinned release runner npx -y rea-agents@6.1.0. The copy leaves the system binary unchanged.

Expected behavior

The existing artifact-inspection contract returns available observations for a valid artifact. A valid FAT table and native-tool representation of its subtype should not produce an integrity contradiction merely because the tool's slice label has a variant suffix. Any unsupported interpretation should remain an explicit limitation rather than a false corruption claim.

Actual behavior

The main universal-file command exits 1 with the following meaningful fields. The release returns the same failure. Signature verification exits 0. The main thinned-copy control exits 0. Architecture inventory, signature inspection, and Mach-O inspection of the original binary also exit 0 on the release.

{
  "code": "artifact_operation_failed",
  "category": "integrity_mismatch",
  "message": "Artifact is invalid or has changed. Get a fresh copy and try again.",
  "details": {
    "operation": "inspect_artifact",
    "reason": "integrity",
    "detail": "lipo slice metadata disagrees with the Mach-O FAT table: arm64e.x1 (cputype=CPU_TYPE_ARM64, cpusubtype=unknown arm64 cpusubtype, offset=163840, size=88672, alignment=16384)"
  }
}

Evidence

Expected source: The existing report morluto/rea#1168, including its valid-system-binary case and restoration criterion; src/contracts/artifactToolContracts.ts defines complete artifact inspection.
Failure source: The executed universal-file command and exact failure fields under Actual behavior, checked against the successful signature verification and thinned-copy control.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Restoration check

Re-run the universal-file command on the stated system binary. It should return available slice observations without a false integrity mismatch, or accurately identify an unsupported interpretation. Keep the successful thin-copy control. A genuinely inconsistent FAT table must still produce a meaningful integrity failure.

Additional context

The existing issue is closed and its discussion confirms a self-built arm64e.v1 reproduction. This follow-up establishes the remaining arm64e.x1 case only. It does not claim the self-built case is still broken or that every macOS toolchain uses the same label.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Artifact inspection of a valid universal system binary fails and incorrectly advises replacing corrupt input. The native metadata tools and thinned-copy inspection remain usable; no data loss was observed.
Workaround status: available
Workaround basis: The native metadata commands and inspection of an authored thin copy succeeded. These provide partial inspection alternatives but do not restore the single artifact graph of the original universal file.

@coygeek commented on GitHub (Oct 9, 2026): fix(artifacts): arm64e.x1 system binaries still fail artifact integrity checks ## Summary The `arm64e.x1` case from the existing report still fails on the tested main snapshot. A valid signed universal system binary is rejected as corrupt, while its thinned arm64e slice and the native metadata commands can be inspected. This is follow-up evidence for the existing issue, rather than a new issue identity. ## Runtime or environment REA npm release `6.1.0` and main source revision `f0da05f67fff1411cf2f7a48a752ee7b348c71ff`, built with `npm ci` and `npm run build:cached`; Node.js `26.10.0`, macOS `27.0.1`, arm64, with Xcode Command Line Tools. Target: the public system binary `/bin/ls` whose FAT inventory includes a slice labeled `arm64e.x1`. Interface: CLI. The main-build commands below require a checkout built at that revision. ## Steps to reproduce 1. On the stated runtime, run the following read-only commands against the system binary. Set `rea_checkout` to the built checkout's absolute directory. ```sh xcrun lipo -detailed_info /bin/ls codesign --verify /bin/ls node "$rea_checkout/scripts/rea.mjs" inspect-artifact /bin/ls --json ``` 2. Run the thinned-copy control in a disposable directory. ```sh fixture_dir="$(mktemp -d)" xcrun lipo /bin/ls -thin arm64e -output "$fixture_dir/ls-arm64e" node "$rea_checkout/scripts/rea.mjs" inspect-artifact "$fixture_dir/ls-arm64e" --json ``` The universal command was also executed with the pinned release runner `npx -y rea-agents@6.1.0`. The copy leaves the system binary unchanged. ## Expected behavior The existing artifact-inspection contract returns available observations for a valid artifact. A valid FAT table and native-tool representation of its subtype should not produce an integrity contradiction merely because the tool's slice label has a variant suffix. Any unsupported interpretation should remain an explicit limitation rather than a false corruption claim. ## Actual behavior The main universal-file command exits 1 with the following meaningful fields. The release returns the same failure. Signature verification exits 0. The main thinned-copy control exits 0. Architecture inventory, signature inspection, and Mach-O inspection of the original binary also exit 0 on the release. ```json { "code": "artifact_operation_failed", "category": "integrity_mismatch", "message": "Artifact is invalid or has changed. Get a fresh copy and try again.", "details": { "operation": "inspect_artifact", "reason": "integrity", "detail": "lipo slice metadata disagrees with the Mach-O FAT table: arm64e.x1 (cputype=CPU_TYPE_ARM64, cpusubtype=unknown arm64 cpusubtype, offset=163840, size=88672, alignment=16384)" } } ``` ## Evidence - Expected source: The existing report [morluto/rea#1168](https://github.com/morluto/rea/issues/1168), including its valid-system-binary case and restoration criterion; `src/contracts/artifactToolContracts.ts` defines complete artifact inspection. - Failure source: The executed universal-file command and exact failure fields under Actual behavior, checked against the successful signature verification and thinned-copy control. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete ## Restoration check Re-run the universal-file command on the stated system binary. It should return available slice observations without a false integrity mismatch, or accurately identify an unsupported interpretation. Keep the successful thin-copy control. A genuinely inconsistent FAT table must still produce a meaningful integrity failure. ## Additional context The existing issue is closed and its discussion confirms a self-built `arm64e.v1` reproduction. This follow-up establishes the remaining `arm64e.x1` case only. It does not claim the self-built case is still broken or that every macOS toolchain uses the same label. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Artifact inspection of a valid universal system binary fails and incorrectly advises replacing corrupt input. The native metadata tools and thinned-copy inspection remain usable; no data loss was observed. - Workaround status: available - Workaround basis: The native metadata commands and inspection of an authored thin copy succeeded. These provide partial inspection alternatives but do not restore the single artifact graph of the original universal file.

bsakaguchi referenced this issue

2026-10-16 21:12:53 +00:00

[PR #413] [MERGED] build: streamline local and CI validation #1961

bsakaguchi referenced this issue

2026-10-16 21:12:54 +00:00

[PR #412] [MERGED] build: separate cached local and PR checks #1962

bsakaguchi referenced this issue

2026-10-16 21:12:56 +00:00

[PR #417] [MERGED] build: shorten local validation loops #1967