morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8935] [Bug] inspect-macho adds an undecoded-payload limitation to fully decoded load commands #7790

New issue

Open

opened 2026-11-04 20:25:30 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-04 20:25:30 +00:00

Owner

Originally created by @coygeek on GitHub (Nov 4, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8935

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

On the tested source revision, inspect-macho succeeds on a real Clang-generated arm64 object and returns a 16-byte LC_LINKER_OPTION with count: 1 and string #1: "-lz". Those fields account for its entire payload, yet load_commands.limitations says some payload fields are not decoded, and the command gains command_sha256 fallback metadata. A plain object without this option has no such limitation.

The latest published package, rea-agents@6.8.0, does not reproduce this behavior on the same objects. The affected source revision also declares version 6.8.0 but includes the later binary-reader implementation. This report concerns that post-release source implementation.

Reproduction

Use a built checkout at commit c3647ade1e557c4179a8b825a37e0c2193cd2526 on macOS arm64 with Node 24.22.1 and Xcode Command Line Tools. For a fresh checkout, install its locked dependencies with npm ci and build with npm run build:cached; these are preparation commands. The following fixture-generation and CLI script was executed from the already-built checkout with exit 0:

set -eu
work=$(mktemp -d)
mkdir -p "$work/home" "$work/tmp"
cat > "$work/options.c" <<'C'
__asm__(".linker_option \"-lz\"");
int sample(void) { return 7; }
C
xcrun clang -c "$work/options.c" -o "$work/options.o"
env -i HOME="$work/home" TMPDIR="$work/tmp" PATH="$PATH" REA_LOG_LEVEL=silent node scripts/rea.mjs inspect-macho "$work/options.o" --json > "$work/result.json"
node --input-type=module - "$work/result.json" <<'JS'
import { readFileSync } from 'node:fs';
const { normalized_result: { load_commands } } = JSON.parse(readFileSync(process.argv[2], 'utf8'));
console.log(JSON.stringify({
  command: load_commands.items.find(row => row.kind === 'LC_LINKER_OPTION'),
  exhaustive: load_commands.exhaustive,
  limitations: load_commands.limitations,
}, null, 2));
JS

The exactly sized compiler-generated option reproduced in three CLI executions. Supplemental synthetic single-command Mach-O fixtures for a 40-byte LC_NOTE and a 40-byte LC_FILESET_ENTRY also returned the same false limitation through the real CLI. No target was executed.

Expected behavior

A command whose entire payload is decoded should not cause the fallback diagnostic for undecoded payload fields. Continue reporting genuine gaps, including trailing bytes outside the modeled fields.

Expected source: the fullyDecoded contract defines completeness by whether every byte after the header belongs to a decoded field. The projection reserves the diagnostic and digest fallback for incomplete or unknown commands. The existing extent regression distinguishes exact modeled extents from unmodeled trailing bytes.

For this option, the payload contains the four-byte count and the four-byte NUL-terminated string -lz. There are no trailing bytes.

Impact and scope

P3: false coverage wording and redundant fallback metadata. The observed result retains the option value, correct command count, successful exit and load_commands.exhaustive: true. No lost facts or downstream failure was observed. Artifact frequency and performance effects are unmeasured.

Controls succeeded: a plain compiler object and an exact LC_UUID had no load-command limitation. A compiler-generated -lzz option had cmdsize 24 with trailing padding; its unmodeled-byte limitation is legitimate. The issue is exact modeled extents, not every command of these kinds. otool -l independently shows the correct option and cmdsize.

REA version

Source c3647ade1e557c4179a8b825a37e0c2193cd2526; manifest/CLI version 6.8.0. The latest published rea-agents@6.8.0 was installed separately and did not reproduce this false limitation. Main source at reviewed revision 9ef70696dd3eda7d6be73c169672a0ab14b027bb has identical affected reader, projection and inspection code; this comparison was source inspection only.

Runtime and platform

Node.js v24.22.1; macOS 27.2 build 26B123; arm64; Apple clang 21.0.0 (clang-2100.4.1.1). No Hopper or Ghidra is required.

Target and artifact context

Real compiler-produced thin 64-bit little-endian arm64 Mach-O relocatable object with .linker_option "-lz". Supplemental exact note/fileset fixtures were synthetic and only establish behavior for those fixture bytes.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Actual output from the reproduction script:

{
  "command": {
    "index": 4,
    "kind": "LC_LINKER_OPTION",
    "fields": {
      "cmd": "LC_LINKER_OPTION",
      "cmdsize": 16,
      "count": 1,
      "string #1": "-lz",
      "command_code": 45,
      "command_offset": 392,
      "command_sha256": "ea5afe249de1960961762845bf766f4e33576c8c78022f4d1a028be167b9e154"
    }
  },
  "exhaustive": true,
  "limitations": [
    "Some load-command payload fields are not decoded; command codes, positions and sizes remain observed, but the payload bytes are not retained. A whole-command SHA-256 is included when it fit the read allowance."
  ]
}

Independent xcrun otool -l output for the compiler-generated object included:

      cmd LC_LINKER_OPTION
  cmdsize 16
    count 1
  string #1 -lz

Existing focused checks passed: npm run test:focused -- src/native/MachoBinaryProjection.test.ts tests/boundary/cli/nativeMachoLinkerOptionsCli.test.ts — 2 files, 8 tests. Their passing assertions do not check absence of this false limitation for exact linker-option commands.

Typed outcome or recovery expected

Keep the decoded command fields and exhaustive command inventory. For this exact command, omit the undecoded-payload limitation and its fallback digest fields. Retain fallback provenance for commands with genuinely unmodeled bytes or fields.

Analysis or suspected cause

Source-derived: the reader's fullyDecoded expression enumerates exact fixed-word, UUID, entrypoint, legacy version, segment and build-version cases. It omits the successful linker-option, note and fileset decoders. The projection first emits their decoded fields, then takes the fallback branch because row.fullyDecoded remains false. Inspection normalization returns that diagnostic in the load-command collection.

Git history places this reader/projection introduction in 640e19d79d13c8aa767909cdbac64556f6879f46. The introducing revision itself was not executed; the runtime observations above use the specified later source baseline.

Additional context

No exact open duplicate was found in open and closed issue/PR searches for command names, completeness terminology and the distinctive fallback output. Closed #4008, resolved by #4048, concerned dropped earlier linker-option strings; this case retains its string and adds a false diagnostic. Merged #7952 introduced the binary owner and intentionally distinguishes command enumeration from field completeness. Closed #5649 concerns genuinely unread lazy dependency payloads, which is a different completeness condition.

Originally created by @coygeek on GitHub (Nov 4, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8935 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior On the tested source revision, `inspect-macho` succeeds on a real Clang-generated arm64 object and returns a 16-byte `LC_LINKER_OPTION` with `count: 1` and `string #1: "-lz"`. Those fields account for its entire payload, yet `load_commands.limitations` says some payload fields are not decoded, and the command gains `command_sha256` fallback metadata. A plain object without this option has no such limitation. The latest published package, `rea-agents@6.8.0`, does not reproduce this behavior on the same objects. The affected source revision also declares version 6.8.0 but includes the later binary-reader implementation. This report concerns that post-release source implementation. ### Reproduction Use a built checkout at commit `c3647ade1e557c4179a8b825a37e0c2193cd2526` on macOS arm64 with Node 24.22.1 and Xcode Command Line Tools. For a fresh checkout, install its locked dependencies with `npm ci` and build with `npm run build:cached`; these are preparation commands. The following fixture-generation and CLI script was executed from the already-built checkout with exit 0: ```sh set -eu work=$(mktemp -d) mkdir -p "$work/home" "$work/tmp" cat > "$work/options.c" <<'C' __asm__(".linker_option \"-lz\""); int sample(void) { return 7; } C xcrun clang -c "$work/options.c" -o "$work/options.o" env -i HOME="$work/home" TMPDIR="$work/tmp" PATH="$PATH" REA_LOG_LEVEL=silent node scripts/rea.mjs inspect-macho "$work/options.o" --json > "$work/result.json" node --input-type=module - "$work/result.json" <<'JS' import { readFileSync } from 'node:fs'; const { normalized_result: { load_commands } } = JSON.parse(readFileSync(process.argv[2], 'utf8')); console.log(JSON.stringify({ command: load_commands.items.find(row => row.kind === 'LC_LINKER_OPTION'), exhaustive: load_commands.exhaustive, limitations: load_commands.limitations, }, null, 2)); JS ``` The exactly sized compiler-generated option reproduced in three CLI executions. Supplemental synthetic single-command Mach-O fixtures for a 40-byte `LC_NOTE` and a 40-byte `LC_FILESET_ENTRY` also returned the same false limitation through the real CLI. No target was executed. ### Expected behavior A command whose entire payload is decoded should not cause the fallback diagnostic for undecoded payload fields. Continue reporting genuine gaps, including trailing bytes outside the modeled fields. Expected source: [the `fullyDecoded` contract](https://github.com/morluto/rea/blob/c3647ade1e557c4179a8b825a37e0c2193cd2526/src/native/MachoCommandNames.ts#L150) defines completeness by whether every byte after the header belongs to a decoded field. [The projection](https://github.com/morluto/rea/blob/c3647ade1e557c4179a8b825a37e0c2193cd2526/src/native/MachoBinaryProjection.ts#L546) reserves the diagnostic and digest fallback for incomplete or unknown commands. [The existing extent regression](https://github.com/morluto/rea/blob/c3647ade1e557c4179a8b825a37e0c2193cd2526/src/native/MachoBinaryProjection.test.ts#L369) distinguishes exact modeled extents from unmodeled trailing bytes. For this option, the payload contains the four-byte count and the four-byte NUL-terminated string `-lz`. There are no trailing bytes. ### Impact and scope P3: false coverage wording and redundant fallback metadata. The observed result retains the option value, correct command count, successful exit and `load_commands.exhaustive: true`. No lost facts or downstream failure was observed. Artifact frequency and performance effects are unmeasured. Controls succeeded: a plain compiler object and an exact `LC_UUID` had no load-command limitation. A compiler-generated `-lzz` option had cmdsize 24 with trailing padding; its unmodeled-byte limitation is legitimate. The issue is exact modeled extents, not every command of these kinds. `otool -l` independently shows the correct option and cmdsize. ### REA version Source `c3647ade1e557c4179a8b825a37e0c2193cd2526`; manifest/CLI version 6.8.0. The latest published `rea-agents@6.8.0` was installed separately and did not reproduce this false limitation. Main source at reviewed revision `9ef70696dd3eda7d6be73c169672a0ab14b027bb` has identical affected reader, projection and inspection code; this comparison was source inspection only. ### Runtime and platform Node.js v24.22.1; macOS 27.2 build 26B123; arm64; Apple clang 21.0.0 (clang-2100.4.1.1). No Hopper or Ghidra is required. ### Target and artifact context Real compiler-produced thin 64-bit little-endian arm64 Mach-O relocatable object with `.linker_option "-lz"`. Supplemental exact note/fileset fixtures were synthetic and only establish behavior for those fixture bytes. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Actual output from the reproduction script: ```json { "command": { "index": 4, "kind": "LC_LINKER_OPTION", "fields": { "cmd": "LC_LINKER_OPTION", "cmdsize": 16, "count": 1, "string #1": "-lz", "command_code": 45, "command_offset": 392, "command_sha256": "ea5afe249de1960961762845bf766f4e33576c8c78022f4d1a028be167b9e154" } }, "exhaustive": true, "limitations": [ "Some load-command payload fields are not decoded; command codes, positions and sizes remain observed, but the payload bytes are not retained. A whole-command SHA-256 is included when it fit the read allowance." ] } ``` Independent `xcrun otool -l` output for the compiler-generated object included: ```text cmd LC_LINKER_OPTION cmdsize 16 count 1 string #1 -lz ``` Existing focused checks passed: `npm run test:focused -- src/native/MachoBinaryProjection.test.ts tests/boundary/cli/nativeMachoLinkerOptionsCli.test.ts` — 2 files, 8 tests. Their passing assertions do not check absence of this false limitation for exact linker-option commands. ### Typed outcome or recovery expected Keep the decoded command fields and exhaustive command inventory. For this exact command, omit the undecoded-payload limitation and its fallback digest fields. Retain fallback provenance for commands with genuinely unmodeled bytes or fields. ### Analysis or suspected cause Source-derived: [the reader's `fullyDecoded` expression](https://github.com/morluto/rea/blob/c3647ade1e557c4179a8b825a37e0c2193cd2526/src/native/MachoCommandNames.ts#L725) enumerates exact fixed-word, UUID, entrypoint, legacy version, segment and build-version cases. It omits the successful linker-option, note and fileset decoders. [The projection](https://github.com/morluto/rea/blob/c3647ade1e557c4179a8b825a37e0c2193cd2526/src/native/MachoBinaryProjection.ts#L518) first emits their decoded fields, then takes the fallback branch because `row.fullyDecoded` remains false. [Inspection normalization](https://github.com/morluto/rea/blob/c3647ade1e557c4179a8b825a37e0c2193cd2526/src/native/NativeMachoInspection.ts#L527) returns that diagnostic in the load-command collection. Git history places this reader/projection introduction in [640e19d79d13c8aa767909cdbac64556f6879f46](https://github.com/morluto/rea/commit/640e19d79d13c8aa767909cdbac64556f6879f46). The introducing revision itself was not executed; the runtime observations above use the specified later source baseline. ### Additional context No exact open duplicate was found in open and closed issue/PR searches for command names, completeness terminology and the distinctive fallback output. Closed [#4008](https://github.com/morluto/rea/issues/4008), resolved by [#4048](https://github.com/morluto/rea/pull/4048), concerned dropped earlier linker-option strings; this case retains its string and adds a false diagnostic. Merged [#7952](https://github.com/morluto/rea/pull/7952) introduced the binary owner and intentionally distinguishes command enumeration from field completeness. Closed [#5649](https://github.com/morluto/rea/issues/5649) concerns genuinely unread lazy dependency payloads, which is a different completeness condition.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7790

No description provided.

Rows
Columns