[PR #5755] fix(native): keep ELF layout independent of pwntools kernel probes #7040

Open
opened 2026-11-01 21:44:55 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5755
Author: @August-Z
Created: 10/21/2026
Status: 🔄 Open

Base: main ← Head: augustz/fix-elf-layout-constructor-probes


📝 Commits (1)

  • 65d7abb fix(native): keep ELF layout independent of pwntools kernel probes

📊 Changes

4 files changed (+160 additions, -8 deletions)

View changed files

📝 bridge/pwntools/layout_impl.py (+28 -0)
📝 docs/binary-diagnostics.md (+4 -0)
📝 scripts/verify-binary-layout.mjs (+124 -7)
📝 third_party/pwntools/README.md (+4 -1)

📄 Description

Summary

Fixes #5715
Fixes #5716
Fixes #5717
Fixes #5718

Before: inspect-binary-layout / inspect_binary_layout failed on some valid ELF files because the pwntools constructor's Linux kernel-image probe ran on every input. A data string IKCFG_ST raised StopIteration. Large declared BSS was zero-filled into memory and hit the address-space limit. An undefined linux_banner symbol made the probe read an unmapped address. Separately, an e_phoff beyond the signed 64-bit seek range raised OverflowError, which was reported as a decoder failure instead of malformed input.

After: these valid files return their ordinary layout, and an out-of-range program-header table is reported as a format failure.

Problem and expected behavior

  • Smallest triggers: a static "IKCFG_ST" string; char zero_fill[3221225473UL] built with -mcmodel=large; an object with an undefined linux_banner reference; any ELF with bit 63 of e_phoff set.
  • Violated invariant: ordinary bytes or symbol names in a selected file must not change layout results, and malformed input must fail as invalid_input/format rather than as a decoder failure.

Change and scope

  • bridge/pwntools/layout_impl.py: the instance-local LayoutELF subclass returns no search() matches while it is being constructed and makes _populate_kernel_version a no-op. Only the constructor's IKCFG probe calls search(), and the kernel config, version and build it fills are used only by checksec() output, which the layout never reports. search() behaves normally after construction. No upstream file or global method changes.
  • Before upstream parsing, the adapter checks that the whole program-header table lies inside the snapshot. For PN_XNUM it reads the count from section zero's raw header, so section zero's name is not resolved without the adapter's checks.
  • Not included: kernel config/version/build reporting (never part of the layout contract).

Contribution scope: ordinary fix within existing boundaries; maintainer scope decision not applicable.

Contract and boundary impact

  • Semantic owner and earliest changed stage: pwntools layout adapter (bridge/pwntools/layout_impl.py), during ELF construction.
  • Contribution scope fit and linked maintainer decision: not applicable (bug fix).
  • CLI and MCP/tool-catalog contract: unchanged. Affected inputs now succeed, or fail as invalid_input instead of a decoder failure.
  • Provider, bridge, target-format, or platform compatibility: same pinned pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2.
  • Evidence, artifact, provenance, or reconstruction contract: unchanged; ordinary fixtures give byte-identical replies on base and head.
  • Target selection, authority, upstream behavior, and cleanup details: no new execution or file access. The upstream kernel probe is skipped, as documented in third_party/pwntools/README.md.
  • Generated metadata, package, or installation impact: none.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated, in scripts/verify-binary-layout.mjs (real-provider lane, CLI and MCP):
    • kernel-config-marker, large-zero-fill and kernel-banner-reference succeed, with sha256 and file bytes unchanged.
    • program-header-overflow returns invalid_input.
    • Under a 96 MiB address space, the zero-tail fixture with +128 MiB p_memsz now succeeds and reports the enlarged size. A new 128 MiB snapshot fixture keeps the resource_constraint/memory path covered, independent of the interpreter's baseline memory use.
  • Base reproduction, executed on base d0067ba34a39:
    • marker: decoder: StopIteration
    • large BSS: resource limit
    • undefined linux_banner: PwnlibException: Address 0x7 is not contained
    • e_phoff bit 63: decoder: OverflowError
    • All four succeed or fail as format on head.
  • User-visible CLI/MCP output: ordinary layout output for the valid files; for the bad offset, "ELF program-header table of 13 entries of 56 bytes at file offset 9223372036854775872 lies outside the selected snapshot."
  • Remaining proof gaps:
    • Only Linux x64 ELF64 is covered, as for the existing lane.
    • Pre-existing and unchanged: an unmapped PT_INTERP vaddr still surfaces as a decoder failure (PwnlibException).

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run check: passed (typecheck, lint, format, knip, metadata, architecture guards)
  • npm run check:fast: passed
  • npm run docs:check: passed
  • npm run test:python: passed (88 tests)
  • npm run test:focused -- src/native/pwntools/PwntoolsFailures.test.ts src/native/pwntools/PwntoolsInputLimit.test.ts: passed (27 tests)
  • npm run verify:binary:layout with REA_PWNTOOLS_PYTHON (pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2) and REA_VERIFY_STRACE_COMMAND=/usr/bin/strace on Linux x64: passed (167 public cases; exec syscalls verified absent)

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: Linux x64 with real pwntools 4.15.0; Hopper/Ghidra not involved.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new process launches; malformed program-header tables are rejected before upstream parsing.

Author readiness

Human owner: @August-Z

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5755 **Author:** [@August-Z](https://github.com/August-Z) **Created:** 10/21/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `augustz/fix-elf-layout-constructor-probes` --- ### 📝 Commits (1) - [`65d7abb`](https://github.com/morluto/rea/commit/65d7abbdcf3d28bb63f2a13689b1a4aa794bed5e) fix(native): keep ELF layout independent of pwntools kernel probes ### 📊 Changes **4 files changed** (+160 additions, -8 deletions) <details> <summary>View changed files</summary> 📝 `bridge/pwntools/layout_impl.py` (+28 -0) 📝 `docs/binary-diagnostics.md` (+4 -0) 📝 `scripts/verify-binary-layout.mjs` (+124 -7) 📝 `third_party/pwntools/README.md` (+4 -1) </details> ### 📄 Description ## Summary Fixes #5715 Fixes #5716 Fixes #5717 Fixes #5718 Before: `inspect-binary-layout` / `inspect_binary_layout` failed on some valid ELF files because the pwntools constructor's Linux kernel-image probe ran on every input. A data string `IKCFG_ST` raised `StopIteration`. Large declared BSS was zero-filled into memory and hit the address-space limit. An undefined `linux_banner` symbol made the probe read an unmapped address. Separately, an `e_phoff` beyond the signed 64-bit seek range raised `OverflowError`, which was reported as a decoder failure instead of malformed input. After: these valid files return their ordinary layout, and an out-of-range program-header table is reported as a `format` failure. ## Problem and expected behavior - Smallest triggers: a static `"IKCFG_ST"` string; `char zero_fill[3221225473UL]` built with `-mcmodel=large`; an object with an undefined `linux_banner` reference; any ELF with bit 63 of `e_phoff` set. - Violated invariant: ordinary bytes or symbol names in a selected file must not change layout results, and malformed input must fail as `invalid_input`/format rather than as a decoder failure. ## Change and scope - `bridge/pwntools/layout_impl.py`: the instance-local `LayoutELF` subclass returns no `search()` matches while it is being constructed and makes `_populate_kernel_version` a no-op. Only the constructor's IKCFG probe calls `search()`, and the kernel config, version and build it fills are used only by `checksec()` output, which the layout never reports. `search()` behaves normally after construction. No upstream file or global method changes. - Before upstream parsing, the adapter checks that the whole program-header table lies inside the snapshot. For `PN_XNUM` it reads the count from section zero's raw header, so section zero's name is not resolved without the adapter's checks. - Not included: kernel config/version/build reporting (never part of the layout contract). Contribution scope: ordinary fix within existing boundaries; maintainer scope decision not applicable. ## Contract and boundary impact - Semantic owner and earliest changed stage: pwntools layout adapter (`bridge/pwntools/layout_impl.py`), during ELF construction. - Contribution scope fit and linked maintainer decision: not applicable (bug fix). - CLI and MCP/tool-catalog contract: unchanged. Affected inputs now succeed, or fail as `invalid_input` instead of a decoder failure. - Provider, bridge, target-format, or platform compatibility: same pinned pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2. - Evidence, artifact, provenance, or reconstruction contract: unchanged; ordinary fixtures give byte-identical replies on base and head. - Target selection, authority, upstream behavior, and cleanup details: no new execution or file access. The upstream kernel probe is skipped, as documented in `third_party/pwntools/README.md`. - Generated metadata, package, or installation impact: none. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated, in `scripts/verify-binary-layout.mjs` (real-provider lane, CLI and MCP): - `kernel-config-marker`, `large-zero-fill` and `kernel-banner-reference` succeed, with sha256 and file bytes unchanged. - `program-header-overflow` returns `invalid_input`. - Under a 96 MiB address space, the zero-tail fixture with +128 MiB `p_memsz` now succeeds and reports the enlarged size. A new 128 MiB snapshot fixture keeps the `resource_constraint`/memory path covered, independent of the interpreter's baseline memory use. - Base reproduction, executed on base `d0067ba34a39`: - marker: `decoder: StopIteration` - large BSS: resource limit - undefined `linux_banner`: `PwnlibException: Address 0x7 is not contained` - `e_phoff` bit 63: `decoder: OverflowError` - All four succeed or fail as format on head. - User-visible CLI/MCP output: ordinary layout output for the valid files; for the bad offset, "ELF program-header table of 13 entries of 56 bytes at file offset 9223372036854775872 lies outside the selected snapshot." - Remaining proof gaps: - Only Linux x64 ELF64 is covered, as for the existing lane. - Pre-existing and unchanged: an unmapped PT_INTERP vaddr still surfaces as a decoder failure (`PwnlibException`). For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run check`: passed (typecheck, lint, format, knip, metadata, architecture guards) - `npm run check:fast`: passed - `npm run docs:check`: passed - `npm run test:python`: passed (88 tests) - `npm run test:focused -- src/native/pwntools/PwntoolsFailures.test.ts src/native/pwntools/PwntoolsInputLimit.test.ts`: passed (27 tests) - `npm run verify:binary:layout` with `REA_PWNTOOLS_PYTHON` (pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2) and `REA_VERIFY_STRACE_COMMAND=/usr/bin/strace` on Linux x64: passed (167 public cases; exec syscalls verified absent) ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: Linux x64 with real pwntools 4.15.0; Hopper/Ghidra not involved. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new process launches; malformed program-header tables are rejected before upstream parsing. ## Author readiness Human owner: @August-Z - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7040
No description provided.