morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7696] [Bug] Dispatch metadata rejects a walkable 4097-command Mach-O table as malformed #4618

New issue

Closed

opened 2026-10-31 00:51:26 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:51:26 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 28, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7696

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

node scripts/rea.mjs inspect-native-dispatch-metadata "$WORK/4097" --format json exits 1 with execution_failure, the diagnostic Malformed Mach-O command bounds, and recovery advising rea doctor. The input is a little-endian arm64 Mach-O header with a fully present, consistently sized table of 4097 16-byte LC_SOURCE_VERSION commands. Its total size is 65,584 bytes.

The corresponding 4096-command control returns successful dispatch-metadata Evidence and an empty class list. For both inputs, trace-dylib-resolution exits 0 with parse_status: parsed and complete coverage, and inspect-artifact exits 0 with root_format: mach-o. These comparisons establish a count-dependent difference between static readers; they do not establish runtime loadability.

Reproduction

Use the built baseline and its locked dependencies with Node.js 24.22.1 first on PATH. Set VERIFY_SCRATCH to an existing disposable directory outside the checkout.
Run the following from the checkout root. It creates and removes the fixtures, isolates product state before the first REA invocation, and prevents package-runner downloads. The assertions describe the current failure.

python3 -I - <<'PY'
import json, os, pathlib, shutil, struct, subprocess, tempfile

node = shutil.which('node')
assert node is not None
with tempfile.TemporaryDirectory(prefix='rea-command-count-', dir=os.environ['VERIFY_SCRATCH']) as work:
    root = pathlib.Path(work)
    (root / 'bin').mkdir()
    for name in ('npx', 'npm', 'bun'):
        stub = root / 'bin' / name
        stub.write_text('#!/bin/sh\necho "$0 $*" >> "$HOME/package-runner-attempts"\nexit 97\n')
        stub.chmod(0o755)
    env = {'PATH': str(root / 'bin') + ':' + str(pathlib.Path(node).parent) + ':/usr/bin:/bin:/usr/sbin:/sbin', 'NO_COLOR': '1'}
    for key in ('HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME', 'XDG_STATE_HOME', 'DSH_HOME', 'DSH_AGENTS_HOME', 'TMPDIR'):
        folder = root / key.lower()
        folder.mkdir()
        env[key] = str(folder)
    for args in ([node, '--version'], [node, 'scripts/rea.mjs', '--version']):
        p = subprocess.run(args, env=env, capture_output=True, text=True, timeout=60)
        assert p.returncode == 0 and p.stderr == '', (p.returncode, p.stderr)
        print(p.stdout.strip())
    for count in (4096, 4097):
        commands = b''.join(struct.pack('<IIQ', 0x2a, 16, i) for i in range(count))
        header = struct.pack('<IiiIIIII', 0xfeedfacf, 0x0100000c, 0, 2, count, len(commands), 0x200085, 0)
        target = root / str(count)
        target.write_bytes(header + commands)
        assert len(commands) == count * 16
        for command in ('inspect-native-dispatch-metadata', 'trace-dylib-resolution', 'inspect-artifact'):
            args = [node, 'scripts/rea.mjs', command, str(target), '--format', 'json']
            p = subprocess.run(args, env=env, capture_output=True, text=True, timeout=60)
            data = json.loads(p.stdout)
            row = {'ncmds': count, 'sizeofcmds': len(commands), 'file_bytes': target.stat().st_size, 'command': command, 'exit': p.returncode}
            if p.returncode:
                row.update(code=data['code'], reason=data['details']['diagnostics']['reason'], remediation=data['remediation']['action'])
                assert command == 'inspect-native-dispatch-metadata' and count == 4097
                assert data['code'] == 'execution_failure'
                assert data['details']['diagnostics']['reason'] == 'Malformed Mach-O command bounds'
            else:
                result = data['normalized_result']
                if command == 'inspect-native-dispatch-metadata':
                    row['input_admission'] = result['input_admission']
                    row['objc_classes'] = result['result']['objc_classes']
                    assert count == 4096
                elif command == 'trace-dylib-resolution':
                    row['coverage'] = result['coverage']['status']
                    row['parse_status'] = result['images'][0]['parse_status']
                    assert row['coverage'] == 'complete' and row['parse_status'] == 'parsed'
                else:
                    row['root_format'] = result['subject']['root_format']
                    assert row['root_format'] == 'mach-o'
            assert p.stderr == '', p.stderr
            print(json.dumps(row))
    assert not (pathlib.Path(env['HOME']) / 'package-runner-attempts').exists()
    print('No package-runner fixture was called.')
PY

Compare the six printed rows. I ran the full matrix once in an initial independent probe and once with this recipe: the 4097 dispatch call failed both times, the 4096 dispatch call succeeded both times, and both comparison commands succeeded for both inputs in each pass. Every inspection had empty stderr; this recipe recorded no package-runner attempt.

Expected behavior

A count of 4097 should not alone turn an otherwise walkable supported command table into a malformed-input diagnosis. Decode it under the documented input budget, or, if a count-specific defensive limit is intentional, document its rationale and return a typed limitation naming that bound and applicable recovery.

The native investigation guide describes little-endian 64-bit metadata reading, CPU-family matching, a default 64 MiB file budget and a selectable maximum of 1 GiB. No command-count cap is stated there. The repository's limit invariant requires real format, protocol, authority or resource constraints. The shared framing contract checks command size, alignment and exact table agreement. These expectations preserve structural checks and resource protection; they do not require accepting unbounded files. Exact sources are in Evidence below.

Impact and scope

P2: the supported public static-inspection command produces no metadata result for the larger table and directs recovery toward environment diagnostics instead of identifying a reader limit. The smaller table succeeds under the same runtime and default budget.

Observed scope is two synthetic thin arm64 fixtures on macOS through the built CLI. They contain no Objective-C/Swift records, so this verification does not demonstrate loss of existing metadata. Frequency in real binaries, MCP behavior, other architectures and full executable loadability remain untested. No workaround that preserves the requested dispatch result was verified; the successful comparison commands answer different questions.

REA version

Exact node scripts/rea.mjs --version output: 6.8.0.
Source baseline: 65f1c447344a423fa2785847dbcea1e9a9e2454a.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1, build 26A434; arm64. These runs use the static native reader and artifact workflows. No Hopper, Ghidra, IDA or Kuna analysis command was run.

Target and artifact context

Each inert fixture has a 32-byte little-endian 64-bit arm64 header and only individually well-formed LC_SOURCE_VERSION records (cmd 0x2a, cmdsize 16, uint64 version). The control declares ncmds=4096, sizeofcmds=65536, total size 65568; the failing case declares ncmds=4097, sizeofcmds=65552, total size 65584. Both tables are exactly present and aligned. These minimal static-format fixtures have no segments, entry point or metadata payload and were never executed.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Selected output from the executed recipe, omitting path fields:

ncmds  command                            exit  result
4096   inspect-native-dispatch-metadata   0     input_admission.maximum_bytes=67108864; retained_bytes=65568; objc_classes=[]
4097   inspect-native-dispatch-metadata   1     execution_failure; Malformed Mach-O command bounds
4096   trace-dylib-resolution            0     parse_status=parsed; coverage=complete
4097   trace-dylib-resolution            0     parse_status=parsed; coverage=complete
4096   inspect-artifact                  0     root_format=mach-o
4097   inspect-artifact                  0     root_format=mach-o

The failing result has retryable: false, details.provider_id: native-macos, and details.operation: inspect_native_dispatch_metadata. Its remediation is:

Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.

Contract and failure locators at the baseline:

docs/native-investigation.md:430-435: shared command framing and the separate 4096-byte probe window. It says: "This validation does not add a global command-table byte cap." That statement concerns bytes, not an explicit unlimited-count guarantee.
docs/native-investigation.md:729-749: supported reader and file budgets. Its default is "64 MiB / 67108864 bytes by default"; the maximum selectable budget is 1 GiB. No command-count ceiling is stated in this section, and a search of the documentation and bundled workflow skill found none for this reader.
AGENTS.md:84: limits must follow real format, protocol, authority or resource constraints.
src/domain/apple/machoLoadCommands.ts:84-137: shared count/alignment/extent walk and exact completion check.
src/native/AppleMachoSelection.ts:201-204: the separate 4096 count guard and malformed-bounds error.
src/native/AppleDispatchMetadata.ts:442-507: generic provider-error translation for this failure.
src/application/native/NativeDispatchMetadataInspection.ts:73-75 and src/application/CapabilityFallback.ts:5-10: this execution error does not enter capability-only symbol fallback.

Apple's Mach-O declarations use a 32-bit command count and define the source-version command as 16 bytes. The reviewed dyld-1330 command walker checks command extents without a 4096-count ceiling. This is source evidence about framing, not an executed loader test.

Typed outcome or recovery expected

Successful static inspection of the larger table, or a justified, documented resource/capability refusal that explicitly identifies the command-count bound. Repeating environment diagnostics should not be presented as the remedy for this fixed input property. No fix or restored-behavior run was performed.

Analysis or suspected cause

Source-derived: the metadata layout reader combines a count ceiling with its out-of-file table check before the shared cursor runs. For the failing fixture, the declared table ends exactly at EOF; only the count operand triggers the refusal. A generic error translator then projects that refusal as a provider execution failure. The cap may be defensive in origin, but no documented count rule or format constraint explaining 4096 was found.

Additional context

Read-only duplicate searches covered open and closed issues, issue comments, and open/closed/merged PRs, using command-count, 4096/4097, dispatch, load-command, malformed-bounds, admission and limit terms. No matching count-only refusal was found.

Related reports have different closure conditions: #7652 preserves refusal of malformed section sizes while correcting error attribution; #7651 preserves an unsupported-width explanation across failed symbol fallback; #6154 and closed, unmerged PR #6155 reject unwalkable framing at admission. This table already has valid framing, and changing only its error category leaves the count-dependent acceptance difference. #5556 and PR #5588 concern recovery for input-byte budgets, not this count ceiling. No known-good revision or regression of those fixes is asserted.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Two public CLI matrices show a bounded static-inspection failure at 4097 commands despite a walkable table below the documented input budget, with generic environment-repair guidance. Real-world frequency and loss of existing metadata were not measured.
Workaround status: unverified
Workaround basis: Dylib tracing and artifact inspection succeed but do not provide the requested dispatch result. No preserving alternative for the affected dispatch input was exercised.

Originally created by @coygeek on GitHub (Oct 28, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7696 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior `node scripts/rea.mjs inspect-native-dispatch-metadata "$WORK/4097" --format json` exits 1 with `execution_failure`, the diagnostic `Malformed Mach-O command bounds`, and recovery advising `rea doctor`. The input is a little-endian arm64 Mach-O header with a fully present, consistently sized table of 4097 16-byte `LC_SOURCE_VERSION` commands. Its total size is 65,584 bytes. The corresponding 4096-command control returns successful dispatch-metadata Evidence and an empty class list. For both inputs, `trace-dylib-resolution` exits 0 with `parse_status: parsed` and complete coverage, and `inspect-artifact` exits 0 with `root_format: mach-o`. These comparisons establish a count-dependent difference between static readers; they do not establish runtime loadability. ### Reproduction 1. Use the built baseline and its locked dependencies with Node.js 24.22.1 first on PATH. Set `VERIFY_SCRATCH` to an existing disposable directory outside the checkout. 2. Run the following from the checkout root. It creates and removes the fixtures, isolates product state before the first REA invocation, and prevents package-runner downloads. The assertions describe the current failure. ```sh python3 -I - <<'PY' import json, os, pathlib, shutil, struct, subprocess, tempfile node = shutil.which('node') assert node is not None with tempfile.TemporaryDirectory(prefix='rea-command-count-', dir=os.environ['VERIFY_SCRATCH']) as work: root = pathlib.Path(work) (root / 'bin').mkdir() for name in ('npx', 'npm', 'bun'): stub = root / 'bin' / name stub.write_text('#!/bin/sh\necho "$0 $*" >> "$HOME/package-runner-attempts"\nexit 97\n') stub.chmod(0o755) env = {'PATH': str(root / 'bin') + ':' + str(pathlib.Path(node).parent) + ':/usr/bin:/bin:/usr/sbin:/sbin', 'NO_COLOR': '1'} for key in ('HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME', 'XDG_STATE_HOME', 'DSH_HOME', 'DSH_AGENTS_HOME', 'TMPDIR'): folder = root / key.lower() folder.mkdir() env[key] = str(folder) for args in ([node, '--version'], [node, 'scripts/rea.mjs', '--version']): p = subprocess.run(args, env=env, capture_output=True, text=True, timeout=60) assert p.returncode == 0 and p.stderr == '', (p.returncode, p.stderr) print(p.stdout.strip()) for count in (4096, 4097): commands = b''.join(struct.pack('<IIQ', 0x2a, 16, i) for i in range(count)) header = struct.pack('<IiiIIIII', 0xfeedfacf, 0x0100000c, 0, 2, count, len(commands), 0x200085, 0) target = root / str(count) target.write_bytes(header + commands) assert len(commands) == count * 16 for command in ('inspect-native-dispatch-metadata', 'trace-dylib-resolution', 'inspect-artifact'): args = [node, 'scripts/rea.mjs', command, str(target), '--format', 'json'] p = subprocess.run(args, env=env, capture_output=True, text=True, timeout=60) data = json.loads(p.stdout) row = {'ncmds': count, 'sizeofcmds': len(commands), 'file_bytes': target.stat().st_size, 'command': command, 'exit': p.returncode} if p.returncode: row.update(code=data['code'], reason=data['details']['diagnostics']['reason'], remediation=data['remediation']['action']) assert command == 'inspect-native-dispatch-metadata' and count == 4097 assert data['code'] == 'execution_failure' assert data['details']['diagnostics']['reason'] == 'Malformed Mach-O command bounds' else: result = data['normalized_result'] if command == 'inspect-native-dispatch-metadata': row['input_admission'] = result['input_admission'] row['objc_classes'] = result['result']['objc_classes'] assert count == 4096 elif command == 'trace-dylib-resolution': row['coverage'] = result['coverage']['status'] row['parse_status'] = result['images'][0]['parse_status'] assert row['coverage'] == 'complete' and row['parse_status'] == 'parsed' else: row['root_format'] = result['subject']['root_format'] assert row['root_format'] == 'mach-o' assert p.stderr == '', p.stderr print(json.dumps(row)) assert not (pathlib.Path(env['HOME']) / 'package-runner-attempts').exists() print('No package-runner fixture was called.') PY ``` 3. Compare the six printed rows. I ran the full matrix once in an initial independent probe and once with this recipe: the 4097 dispatch call failed both times, the 4096 dispatch call succeeded both times, and both comparison commands succeeded for both inputs in each pass. Every inspection had empty stderr; this recipe recorded no package-runner attempt. ### Expected behavior A count of 4097 should not alone turn an otherwise walkable supported command table into a malformed-input diagnosis. Decode it under the documented input budget, or, if a count-specific defensive limit is intentional, document its rationale and return a typed limitation naming that bound and applicable recovery. The native investigation guide describes little-endian 64-bit metadata reading, CPU-family matching, a default 64 MiB file budget and a selectable maximum of 1 GiB. No command-count cap is stated there. The repository's limit invariant requires real format, protocol, authority or resource constraints. The shared framing contract checks command size, alignment and exact table agreement. These expectations preserve structural checks and resource protection; they do not require accepting unbounded files. Exact sources are in Evidence below. ### Impact and scope P2: the supported public static-inspection command produces no metadata result for the larger table and directs recovery toward environment diagnostics instead of identifying a reader limit. The smaller table succeeds under the same runtime and default budget. Observed scope is two synthetic thin arm64 fixtures on macOS through the built CLI. They contain no Objective-C/Swift records, so this verification does not demonstrate loss of existing metadata. Frequency in real binaries, MCP behavior, other architectures and full executable loadability remain untested. No workaround that preserves the requested dispatch result was verified; the successful comparison commands answer different questions. ### REA version Exact `node scripts/rea.mjs --version` output: `6.8.0`. Source baseline: `65f1c447344a423fa2785847dbcea1e9a9e2454a`. ### Runtime and platform Node.js `v24.22.1`; macOS `27.0.1`, build `26A434`; `arm64`. These runs use the static native reader and artifact workflows. No Hopper, Ghidra, IDA or Kuna analysis command was run. ### Target and artifact context Each inert fixture has a 32-byte little-endian 64-bit arm64 header and only individually well-formed `LC_SOURCE_VERSION` records (`cmd` 0x2a, `cmdsize` 16, uint64 version). The control declares `ncmds=4096`, `sizeofcmds=65536`, total size 65568; the failing case declares `ncmds=4097`, `sizeofcmds=65552`, total size 65584. Both tables are exactly present and aligned. These minimal static-format fixtures have no segments, entry point or metadata payload and were never executed. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Selected output from the executed recipe, omitting path fields: ```text ncmds command exit result 4096 inspect-native-dispatch-metadata 0 input_admission.maximum_bytes=67108864; retained_bytes=65568; objc_classes=[] 4097 inspect-native-dispatch-metadata 1 execution_failure; Malformed Mach-O command bounds 4096 trace-dylib-resolution 0 parse_status=parsed; coverage=complete 4097 trace-dylib-resolution 0 parse_status=parsed; coverage=complete 4096 inspect-artifact 0 root_format=mach-o 4097 inspect-artifact 0 root_format=mach-o ``` The failing result has `retryable: false`, `details.provider_id: native-macos`, and `details.operation: inspect_native_dispatch_metadata`. Its remediation is: ```text Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed. ``` Contract and failure locators at the baseline: - `docs/native-investigation.md:430-435`: shared command framing and the separate 4096-byte probe window. It says: "This validation does not add a global command-table byte cap." That statement concerns bytes, not an explicit unlimited-count guarantee. - `docs/native-investigation.md:729-749`: supported reader and file budgets. Its default is "64 MiB / 67108864 bytes by default"; the maximum selectable budget is 1 GiB. No command-count ceiling is stated in this section, and a search of the documentation and bundled workflow skill found none for this reader. - `AGENTS.md:84`: limits must follow real format, protocol, authority or resource constraints. - `src/domain/apple/machoLoadCommands.ts:84-137`: shared count/alignment/extent walk and exact completion check. - `src/native/AppleMachoSelection.ts:201-204`: the separate 4096 count guard and malformed-bounds error. - `src/native/AppleDispatchMetadata.ts:442-507`: generic provider-error translation for this failure. - `src/application/native/NativeDispatchMetadataInspection.ts:73-75` and `src/application/CapabilityFallback.ts:5-10`: this execution error does not enter capability-only symbol fallback. Apple's [Mach-O declarations](https://github.com/apple-oss-distributions/xnu/blob/main/EXTERNAL_HEADERS/mach-o/loader.h#L66-L74) use a 32-bit command count and [define the source-version command](https://github.com/apple-oss-distributions/xnu/blob/main/EXTERNAL_HEADERS/mach-o/loader.h#L1439-L1447) as 16 bytes. The reviewed [dyld-1330 command walker](https://github.com/apple-oss-distributions/dyld/blob/dyld-1330/mach_o/Header.cpp#L976-L1017) checks command extents without a 4096-count ceiling. This is source evidence about framing, not an executed loader test. ### Typed outcome or recovery expected Successful static inspection of the larger table, or a justified, documented resource/capability refusal that explicitly identifies the command-count bound. Repeating environment diagnostics should not be presented as the remedy for this fixed input property. No fix or restored-behavior run was performed. ### Analysis or suspected cause Source-derived: the metadata layout reader combines a count ceiling with its out-of-file table check before the shared cursor runs. For the failing fixture, the declared table ends exactly at EOF; only the count operand triggers the refusal. A generic error translator then projects that refusal as a provider execution failure. The cap may be defensive in origin, but no documented count rule or format constraint explaining 4096 was found. ### Additional context Read-only duplicate searches covered open and closed issues, issue comments, and open/closed/merged PRs, using command-count, 4096/4097, dispatch, load-command, malformed-bounds, admission and limit terms. No matching count-only refusal was found. Related reports have different closure conditions: [#7652](https://github.com/morluto/rea/issues/7652) preserves refusal of malformed section sizes while correcting error attribution; [#7651](https://github.com/morluto/rea/issues/7651) preserves an unsupported-width explanation across failed symbol fallback; [#6154](https://github.com/morluto/rea/issues/6154) and closed, unmerged [PR #6155](https://github.com/morluto/rea/pull/6155) reject unwalkable framing at admission. This table already has valid framing, and changing only its error category leaves the count-dependent acceptance difference. [#5556](https://github.com/morluto/rea/issues/5556) and [PR #5588](https://github.com/morluto/rea/pull/5588) concern recovery for input-byte budgets, not this count ceiling. No known-good revision or regression of those fixes is asserted. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Two public CLI matrices show a bounded static-inspection failure at 4097 commands despite a walkable table below the documented input budget, with generic environment-repair guidance. Real-world frequency and loss of existing metadata were not measured. - Workaround status: unverified - Workaround basis: Dylib tracing and artifact inspection succeed but do not provide the requested dispatch result. No preserving alternative for the affected dispatch input was exercised.

bsakaguchi closed this issue

2026-10-31 00:51:26 +00:00

bsakaguchi referenced this issue

2026-10-31 01:13:39 +00:00

[PR #4618] feat(clients): register Kiro during setup #6335

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425