morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5716] [Bug] inspect-binary-layout classifies an overflowing ELF program-header offset as execution_failure #3465

New issue

Closed

opened 2026-10-31 00:40:45 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:40:45 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5716

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-binary-layout rejects a malformed ELF64 whose program-header table offset exceeds the signed file-offset range, but reports execution_failure and recommends rea doctor. The same provider correctly reports a truncated control as invalid_request with a path issue whose reason is invalid_format. Successful inspection of the mutated bytes is not expected.

Reproduction

From a built source checkout on Linux x64, set REA_PWNTOOLS_PYTHON to an absolute Python executable containing pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2, then run:

work_dir="$(mktemp -d)"
mkdir -p "$work_dir"/{home,cfg,data,cache,state,temp}
printf 'int main(void){return 0;}\n' > "$work_dir/fixture.c"
gcc -O1 "$work_dir/fixture.c" -o "$work_dir/fixture"
python3 - "$work_dir/fixture" "$work_dir/mutated" "$work_dir/truncated" <<'PYTHON'
from pathlib import Path
import sys
content = bytearray(Path(sys.argv[1]).read_bytes())
content[0x27] = 0x80
Path(sys.argv[2]).write_bytes(content)
Path(sys.argv[3]).write_bytes(content[:64])
PYTHON
for selected in fixture mutated truncated; do
  env -i PATH="$PATH" HOME="$work_dir/home" \
    XDG_CONFIG_HOME="$work_dir/cfg" XDG_DATA_HOME="$work_dir/data" \
    XDG_CACHE_HOME="$work_dir/cache" XDG_STATE_HOME="$work_dir/state" \
    TMPDIR="$work_dir/temp" REA_LOG_LEVEL=silent \
    REA_PWNTOOLS_PYTHON="$REA_PWNTOOLS_PYTHON" \
    node scripts/rea.mjs inspect-binary-layout "$work_dir/$selected" --json \
    > "$work_dir/$selected.json"
  printf '%s exit=%s\n' "$selected" "$?"
done
cat "$work_dir/mutated.json" "$work_dir/truncated.json"

An independent verifier executed this procedure at the baseline below, including a second pass through the baseline checkout's CLI mounted read-only in the Linux container. Both passes reproduced the classification. The valid control exits 0; the mutated and truncated inputs each exit 1. This submission rechecked source on main and inspected the retained runtime outputs; it did not replay the CLI on main.

Expected behavior

Reject the malformed file through the existing typed input-error path: invalid_request, with an issue for path whose reason is invalid_format and meaningful malformed-input diagnostics. Preserve successful inspection of the valid control and keep genuine provider failures separately classified.

The source contract is the format-failure mapping in PwntoolsDecoder.ts:415-424. docs/binary-diagnostics.md documents typed decoder failures; the layout contract also requires malformed objects to return no partial success.

Impact and scope

P2: the machine-readable failure category and recovery advice are wrong for this bounded malformed-input case. The input remains rejected, and no valid-file result loss, data loss or security boundary crossing was observed. No command option restores the expected category for the same bytes; external tools can diagnose the file but do not correct REA's typed result. Other architectures, formats and MCP transport were not tested.

REA version

rea-agents 6.7.0, source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Verified at baseline; main source-checked only at 84c89d1107cda017509bc3cd851573d073f9906d. The cited parser, adapter, failure mapping, contract and documentation files are unchanged between those commits. The latest published release remains 6.7.0.

Runtime and platform

Reproduction: Linux x64 container, Node.js v24.22.0, GCC 12.2.0; unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. Submission/source-check host: Node.js v26.12.0, macOS 27.0.1, arm64; the command was not run natively on macOS. Hopper and Ghidra are not involved.

Target and artifact context

Self-built ELF64 x86-64 little-endian PIE, 15,840 bytes and 13 program headers. Changing byte 0x27 to 0x80 changes e_phoff from 0x40 to 0x8000000000000040 while preserving the remaining bytes. The truncated control contains the first 64 bytes of that mutation. The inspected fixtures were not executed as programs.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Retained baseline CLI output for the mutation contains this subset:

{
  "code": "execution_failure",
  "category": "execution_failure",
  "retryable": false,
  "message": "Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",
  "details": {
    "provider_id": "pwntools-elf",
    "operation": "inspect_binary_layout",
    "diagnostics": {
      "phase": "decoder",
      "failure_kind": "decoder",
      "reason": "OverflowError: Python int too large to convert to C ssize_t",
      "stdout": "",
      "stderr": ""
    }
  }
}

The truncated control returns invalid_request, category invalid_input, with issues: [{"path":["path"],"reason":"invalid_format","message":"Reading section 29 at offset 15776 past EOF 64"}]. It fails on the section table before reaching the overflowing program-header offset. The valid control returns complete layout Evidence.

readelf -lW diagnoses Reading 728 bytes extends past end of file for program headers but exits 0; objdump -h rejects the mutation as an unrecognized format and exits 1. Direct construction with the configured pwntools Python raises the same OverflowError; its traceback reaches pyelftools struct_parse calling stream.seek for the invalid offset. Evidence provenance: observed by the independent verifier; retained JSON, exit records and traceback inspected during submission.

Typed outcome or recovery expected

Keep exit 1 for the mutation, with invalid_request / invalid_format and useful input diagnostics instead of provider-repair guidance. Preserve valid-control success and truncated-control format rejection. Do not relabel every provider failure as invalid input.

Analysis or suspected cause

Source-derived reasoning: layout_impl.py:170 constructs the pwntools image before program-header range validation. The structural exception handler at line 305 omits OverflowError. The generic decoder runtime path then emits reason: decoder, which PwntoolsDecoder.ts:442-451 maps to a provider error rather than the format input error. This agrees with the retained runtime result.

The exception arises while bundled integration code calls the unchanged, caller-supplied pwntools/pyelftools dependencies. The observed defect is REA's typed classification; the traceback alone does not establish a separate dependency defect.

Additional context

Prepared with AI assistance and independently verified. No fix or successful restoration was tested, and no earlier known-good version was established. Related #2191 concerns malformed PE section headers in the managed provider; #3055 concerns unavailable pwntools configuration. Neither covers this ELF decoder trigger or its format-error restoration condition.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5716 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior `inspect-binary-layout` rejects a malformed ELF64 whose program-header table offset exceeds the signed file-offset range, but reports `execution_failure` and recommends `rea doctor`. The same provider correctly reports a truncated control as `invalid_request` with a `path` issue whose reason is `invalid_format`. Successful inspection of the mutated bytes is not expected. ### Reproduction From a built source checkout on Linux x64, set `REA_PWNTOOLS_PYTHON` to an absolute Python executable containing pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2, then run: ```sh work_dir="$(mktemp -d)" mkdir -p "$work_dir"/{home,cfg,data,cache,state,temp} printf 'int main(void){return 0;}\n' > "$work_dir/fixture.c" gcc -O1 "$work_dir/fixture.c" -o "$work_dir/fixture" python3 - "$work_dir/fixture" "$work_dir/mutated" "$work_dir/truncated" <<'PYTHON' from pathlib import Path import sys content = bytearray(Path(sys.argv[1]).read_bytes()) content[0x27] = 0x80 Path(sys.argv[2]).write_bytes(content) Path(sys.argv[3]).write_bytes(content[:64]) PYTHON for selected in fixture mutated truncated; do env -i PATH="$PATH" HOME="$work_dir/home" \ XDG_CONFIG_HOME="$work_dir/cfg" XDG_DATA_HOME="$work_dir/data" \ XDG_CACHE_HOME="$work_dir/cache" XDG_STATE_HOME="$work_dir/state" \ TMPDIR="$work_dir/temp" REA_LOG_LEVEL=silent \ REA_PWNTOOLS_PYTHON="$REA_PWNTOOLS_PYTHON" \ node scripts/rea.mjs inspect-binary-layout "$work_dir/$selected" --json \ > "$work_dir/$selected.json" printf '%s exit=%s\n' "$selected" "$?" done cat "$work_dir/mutated.json" "$work_dir/truncated.json" ``` An independent verifier executed this procedure at the baseline below, including a second pass through the baseline checkout's CLI mounted read-only in the Linux container. Both passes reproduced the classification. The valid control exits 0; the mutated and truncated inputs each exit 1. This submission rechecked source on main and inspected the retained runtime outputs; it did not replay the CLI on main. ### Expected behavior Reject the malformed file through the existing typed input-error path: `invalid_request`, with an issue for `path` whose reason is `invalid_format` and meaningful malformed-input diagnostics. Preserve successful inspection of the valid control and keep genuine provider failures separately classified. The source contract is the format-failure mapping in [`PwntoolsDecoder.ts:415-424`](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/pwntools/PwntoolsDecoder.ts#L415-L424). [`docs/binary-diagnostics.md`](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/binary-diagnostics.md) documents typed decoder failures; the layout contract also requires malformed objects to return no partial success. ### Impact and scope P2: the machine-readable failure category and recovery advice are wrong for this bounded malformed-input case. The input remains rejected, and no valid-file result loss, data loss or security boundary crossing was observed. No command option restores the expected category for the same bytes; external tools can diagnose the file but do not correct REA's typed result. Other architectures, formats and MCP transport were not tested. ### REA version `rea-agents` 6.7.0, source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Verified at baseline; main source-checked only at `84c89d1107cda017509bc3cd851573d073f9906d`. The cited parser, adapter, failure mapping, contract and documentation files are unchanged between those commits. The latest published release remains 6.7.0. ### Runtime and platform Reproduction: Linux x64 container, Node.js v24.22.0, GCC 12.2.0; unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. Submission/source-check host: Node.js v26.12.0, macOS 27.0.1, arm64; the command was not run natively on macOS. Hopper and Ghidra are not involved. ### Target and artifact context Self-built ELF64 x86-64 little-endian PIE, 15,840 bytes and 13 program headers. Changing byte `0x27` to `0x80` changes `e_phoff` from `0x40` to `0x8000000000000040` while preserving the remaining bytes. The truncated control contains the first 64 bytes of that mutation. The inspected fixtures were not executed as programs. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Retained baseline CLI output for the mutation contains this subset: ```json { "code": "execution_failure", "category": "execution_failure", "retryable": false, "message": "Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.", "details": { "provider_id": "pwntools-elf", "operation": "inspect_binary_layout", "diagnostics": { "phase": "decoder", "failure_kind": "decoder", "reason": "OverflowError: Python int too large to convert to C ssize_t", "stdout": "", "stderr": "" } } } ``` The truncated control returns `invalid_request`, category `invalid_input`, with `issues: [{"path":["path"],"reason":"invalid_format","message":"Reading section 29 at offset 15776 past EOF 64"}]`. It fails on the section table before reaching the overflowing program-header offset. The valid control returns complete layout Evidence. `readelf -lW` diagnoses `Reading 728 bytes extends past end of file for program headers` but exits 0; `objdump -h` rejects the mutation as an unrecognized format and exits 1. Direct construction with the configured pwntools Python raises the same `OverflowError`; its traceback reaches pyelftools `struct_parse` calling `stream.seek` for the invalid offset. Evidence provenance: observed by the independent verifier; retained JSON, exit records and traceback inspected during submission. ### Typed outcome or recovery expected Keep exit 1 for the mutation, with `invalid_request` / `invalid_format` and useful input diagnostics instead of provider-repair guidance. Preserve valid-control success and truncated-control format rejection. Do not relabel every provider failure as invalid input. ### Analysis or suspected cause Source-derived reasoning: [`layout_impl.py:170`](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/bridge/pwntools/layout_impl.py#L170) constructs the pwntools image before program-header range validation. The structural exception handler at line 305 omits `OverflowError`. The generic decoder runtime path then emits `reason: decoder`, which `PwntoolsDecoder.ts:442-451` maps to a provider error rather than the format input error. This agrees with the retained runtime result. The exception arises while bundled integration code calls the unchanged, caller-supplied pwntools/pyelftools dependencies. The observed defect is REA's typed classification; the traceback alone does not establish a separate dependency defect. ### Additional context Prepared with AI assistance and independently verified. No fix or successful restoration was tested, and no earlier known-good version was established. Related #2191 concerns malformed PE section headers in the managed provider; #3055 concerns unavailable pwntools configuration. Neither covers this ELF decoder trigger or its format-error restoration condition.

bsakaguchi closed this issue

2026-10-31 00:40:46 +00:00

bsakaguchi commented

2026-10-31 00:40:47 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 21, 2026):

Cross-referencing with canonical issue #5715 (inspect-binary-layout fails on a valid ELF containing IKCFG_ST) and sibling ELF binary layout reports #5717, #5718.

Consolidated under Cluster CC (ELF Binary Layout Inspection & Section Allocation Bounds) in tracker #4330.

@matbrgz commented on GitHub (Oct 21, 2026): Cross-referencing with canonical issue #5715 (`inspect-binary-layout fails on a valid ELF containing IKCFG_ST`) and sibling ELF binary layout reports #5717, #5718. Consolidated under **Cluster CC (ELF Binary Layout Inspection & Section Allocation Bounds)** in tracker #4330.

bsakaguchi commented

2026-10-31 00:40:47 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Verified resolved on main at d87e73f2db.

I replayed the exact overflowing program-header-offset fixture, a first-64-byte truncated control and the valid control through the public CLI and production Python decoder on Linux x64. Both invalid inputs now exit 1 with invalid_request / invalid_input and a caller path issue whose reason is invalid_format; the decoder reports reason: "format". The valid control succeeds. The failures contain no doctor advice or partial success, and all input bytes and digests remain unchanged. This verifies the reported classification repair without executing a target.

The checks used pinned Node 24.22.1 and pwntools 4.15.0. Relevant implementation: layout_impl.py, layout_impl.py.

Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Verified resolved on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. I replayed the exact overflowing program-header-offset fixture, a first-64-byte truncated control and the valid control through the public CLI and production Python decoder on Linux x64. Both invalid inputs now exit 1 with `invalid_request / invalid_input` and a caller `path` issue whose reason is `invalid_format`; the decoder reports `reason: "format"`. The valid control succeeds. The failures contain no doctor advice or partial success, and all input bytes and digests remain unchanged. This verifies the reported classification repair without executing a target. The checks used pinned Node 24.22.1 and pwntools 4.15.0. Relevant implementation: [layout_impl.py](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/bridge/pwntools/layout_impl.py#L132), [layout_impl.py](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/bridge/pwntools/layout_impl.py#L323). Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 00:50:50 +00:00

[GH-ISSUE #7641] [Bug] Get started's all-supported-clients section still reflects npm 6.4.0 #4564

bsakaguchi referenced this issue

2026-10-31 01:09:12 +00:00

[PR #3465] [MERGED] docs(installation): list every client that first shipped in 6.5.0 #5756