[PR #7052] [MERGED] feat(binary-layout): admit ELF32 i386, ARM and MIPS layout profiles #7607

Closed
opened 2026-11-03 13:39:04 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7052
Author: @salmanalmutairi1
Created: 10/25/2026
Status: ✅ Merged
Merged: 10/25/2026
Merged by: @morluto

Base: main ← Head: feat/elf32-binary-layout


📝 Commits (4)

  • ebaf7a4 feat(binary-layout): admit ELF32 i386, ARM and MIPS layout profiles
  • b8426c9 ci(binary-layout): install target libc headers for ELF32 fixtures
  • 1443a20 fix(binary-layout): size the minimum symbol entry by ELF class
  • 27dc69f test(binary-layout): compare ELF32 symbols and relocations with readelf

📊 Changes

10 files changed (+291 additions, -25 deletions)

View changed files

📝 .github/workflows/real-binary-layout.yml (+6 -0)
📝 bridge/pwntools/layout_impl.py (+18 -10)
📝 docs/binary-diagnostics.md (+6 -3)
📝 docs/testing.md (+6 -2)
📝 scripts/verify-binary-layout.mjs (+175 -2)
📝 src/application/binaryDiagnostics/BinaryLayoutService.test.ts (+22 -0)
📝 src/application/binaryDiagnostics/BinaryLayoutService.ts (+7 -1)
📝 src/contracts/native/binaryDiagnosticsToolContracts.ts (+1 -1)
📝 src/domain/native/binaryLayout.test.ts (+30 -0)
📝 src/domain/native/binaryLayout.ts (+20 -6)

📄 Description

Summary

Fixes #6934.

inspect_binary_layout refused every ELF32 object with unsupported_target, so ARM32 and MIPS32 router/IoT firmware got no layout facts. It now admits EXEC/DYN/REL objects in these profiles: ELF64 x86-64 (unchanged), ELF32 i386, ELF32 little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes and byte orders still return unsupported_target, and the message names the admitted profiles.

Problem and expected behavior

bridge/pwntools/layout_impl.py accepted only EI_CLASS == ELFCLASS64, EI_DATA == ELFDATA2LSB and e_machine == EM_X86_64, and binaryLayoutSchema pinned architecture to EM_X86_64 / 64 / little. The rest of the adapter already goes through class- and byte-order-aware pyelftools structures (image.structs.Elf_Sym, Elf_Dyn, header sizes), so 32-bit and big-endian layouts need no other parsing change.

Expected: an ELF32 ARM or MIPS executable returns the same layout report as an x86-64 one, with architecture naming its real machine, class and byte order.

Change and scope

  • Bridge. PROFILES maps each admitted (e_machine, class bits, byte order) to the architecture pwntools must also report. The identification fields are read in the declared byte order, ELF32 headers are bounds-checked at 52 bytes, and pwntools' own arch/bits/endian must match the admitted profile after loading.
  • Schema. architecture is a union of the exact admitted combinations, so a report such as EM_386 / 64 or EM_ARM / big still fails validation.
  • Evidence. The subject architecture previously always said x86_64. It now follows the reported machine (x86_64, x86, arm, mips).
  • Verification. verify:binary:layout cross-compiles i386, ARM, MIPS and MIPSel executables and relocatable objects. They are never executed. The verifier compares section names, addresses, offsets and sizes with GNU readelf through both CLI and MCP. It also compares every symbol value/size and every relocation offset, numeric type and symbol table/index reference, retaining table/entry identities and checking the oracle row counts. The real-binary-layout workflow installs the four cross toolchains.

Not included: AArch64, ELF32 big-endian ARM, MIPS64/n32, x32, or any machine outside the list above. GOT/PLT maps and mitigation indicators still come from the unchanged upstream heuristics, and their completeness stays unknown, as before.

Follow the contribution scope. This widens target-format admission of an existing read-only offline operation. It adds no execution, injection or mutation authority. Whether widening architecture needs a maintainer scope decision is asked on #6934 and is still pending.

Contract and boundary impact

  • Semantic owner and earliest changed stage: PwntoolsLayoutProvider's Python adapter (layout_impl.py) owns target admission. binaryLayoutSchema owns the output contract, and BinaryLayoutService composes Evidence. No superseded path remains.
  • Contribution scope fit and linked maintainer decision: existing read-only capability. Question posted on #6934, no decision recorded yet.
  • CLI and MCP/tool-catalog contract: input unchanged. The output architecture field widens from one literal combination to five admitted combinations, and the tool description states the new profile.
  • Provider, bridge, target-format, or platform compatibility: same pinned pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2 on Linux x64. Existing ELF64 x86-64 reports are unchanged.
  • Evidence, artifact, provenance, or reconstruction contract: the Evidence subject architecture now reflects the target instead of a constant x86_64.
  • Target selection, authority, upstream behavior, and cleanup details: unchanged. The target is never launched, and Unicorn PLT emulation is the existing upstream behavior.
  • Generated metadata, package, or installation impact: none committed. The catalog is regenerated by CI.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated:
    • src/domain/native/binaryLayout.test.ts: admits the four new combinations and rejects mismatched class/machine/byte-order combinations. It also accepts 16-byte Elf32_Sym entries for ELF32 relocations while still refusing undersized ELF32 entries and 16-byte entries in ELF64 reports.
    • src/application/binaryDiagnostics/BinaryLayoutService.test.ts: the Evidence subject names the reported machine.
    • scripts/verify-binary-layout.mjs: real ELF32 cases cross-checked against readelf. The existing AArch64 and core unsupported_target cases are kept.
  • Base reproduction or other evidence: on the base revision, layout_impl.py:119-120 refuses any object with EI_CLASS != 2 before parsing, as quoted in #6934.
  • User-visible CLI/MCP output: the real lane asserts, through both inspect-binary-layout --json and MCP, that each cross-compiled fixture reports architecture {EM_386, 32, little}, {EM_ARM, 32, little}, {EM_MIPS, 32, big} or {EM_MIPS, 32, little} and the expected image_type, and that every section's name, address, offset and size equals GNU readelf -SW. A verbatim report excerpt was not captured from the CI log.
  • Remaining proof gaps: ELF32 big-endian ARM, AArch64 and MIPS64 remain unsupported and unverified. GOT/PLT completeness on ARM/MIPS is reported as unknown and not asserted.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • Review follow-up at 27dc69f93: npm run check, the pre-push npm run check:fast, Node syntax checking and git diff --check passed on macOS (Node 24.22.1, npm 11.16.0). The new ELF32 symbol/relocation comparisons have not been run against the real provider: isolated Linux x64 setup was stopped after sustained dependency-download throttling, and dispatching the fork's Real offline binary layout workflow was denied with HTTP 403. The successful real-provider run below covers the earlier 1443a20 revision.

  • npx tsc -p . --noEmit (Windows 11, Node 24.18.0): passed.

  • npm run test:local -- <17 binary-diagnostics source test files>: 231 passed, 2 skipped, 1 failed. The failure is ByteEntropyService.test.ts "parent directory moves during streaming", which fails identically on the unchanged base on Windows (rename semantics) and is unrelated.

  • npx oxlint on changed files: no new warnings compared with the base files.

  • npm run verify:module-boundaries: passed.

  • Pre-push hook (typecheck, lint): passed.

  • Real offline binary layout workflow on the fork (ubuntu-24.04, pinned pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2):

    • Run #3 at 1443a20: passed. The verifier step took 232 s, covering the existing ELF64 cases and the eight ELF32 cases.
    • Run #2 at b8426c9: the 20 existing ELF64 cases passed, then the first i686 report was refused as unreadable_output because the relocation check required 24-byte (Elf64_Sym) symbol entries. This was fixed in 1443a20, with regression cases in binaryLayout.test.ts that fail on b8426c9.
    • Run #1 at ebaf7a4: the 20 existing ELF64 cases passed, then the i686 fixture failed to compile because --no-install-recommends skipped the target libc headers. This was fixed in b8426c9.
  • Not run locally: npm run docs:check and boundary tests (test:focused), because the Windows build needs native Windows artifacts absent from a source checkout.

Compatibility, safety, and release

  • Breaking changes or migration steps: none for existing callers. ELF64 x86-64 reports are unchanged. Consumers that assumed architecture was always EM_X86_64 / 64 / little will now also see the ELF32 combinations.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable. Real pwntools coverage is the workflow above.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new authority. Targets are still read from an owned snapshot and never executed.

Author readiness

Human owner: @salmanalmutairi1

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7052 **Author:** [@salmanalmutairi1](https://github.com/salmanalmutairi1) **Created:** 10/25/2026 **Status:** ✅ Merged **Merged:** 10/25/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `feat/elf32-binary-layout` --- ### 📝 Commits (4) - [`ebaf7a4`](https://github.com/morluto/rea/commit/ebaf7a470b218308ca0ff605c364be5cb031754e) feat(binary-layout): admit ELF32 i386, ARM and MIPS layout profiles - [`b8426c9`](https://github.com/morluto/rea/commit/b8426c9299607e106f416506a1b7b2a220235267) ci(binary-layout): install target libc headers for ELF32 fixtures - [`1443a20`](https://github.com/morluto/rea/commit/1443a20fb5d337e4903c0b5877ffcbd6d87fa6c7) fix(binary-layout): size the minimum symbol entry by ELF class - [`27dc69f`](https://github.com/morluto/rea/commit/27dc69f93543feca314e07d882d6dc9cee9678bc) test(binary-layout): compare ELF32 symbols and relocations with readelf ### 📊 Changes **10 files changed** (+291 additions, -25 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/real-binary-layout.yml` (+6 -0) 📝 `bridge/pwntools/layout_impl.py` (+18 -10) 📝 `docs/binary-diagnostics.md` (+6 -3) 📝 `docs/testing.md` (+6 -2) 📝 `scripts/verify-binary-layout.mjs` (+175 -2) 📝 `src/application/binaryDiagnostics/BinaryLayoutService.test.ts` (+22 -0) 📝 `src/application/binaryDiagnostics/BinaryLayoutService.ts` (+7 -1) 📝 `src/contracts/native/binaryDiagnosticsToolContracts.ts` (+1 -1) 📝 `src/domain/native/binaryLayout.test.ts` (+30 -0) 📝 `src/domain/native/binaryLayout.ts` (+20 -6) </details> ### 📄 Description <!-- PR title: feat(binary-layout): admit ELF32 i386, ARM and MIPS layout profiles --> ## Summary Fixes #6934. `inspect_binary_layout` refused every ELF32 object with `unsupported_target`, so ARM32 and MIPS32 router/IoT firmware got no layout facts. It now admits EXEC/DYN/REL objects in these profiles: ELF64 x86-64 (unchanged), ELF32 i386, ELF32 little-endian ARM and ELF32 little- or big-endian MIPS. Other machines, classes and byte orders still return `unsupported_target`, and the message names the admitted profiles. ## Problem and expected behavior `bridge/pwntools/layout_impl.py` accepted only `EI_CLASS == ELFCLASS64`, `EI_DATA == ELFDATA2LSB` and `e_machine == EM_X86_64`, and `binaryLayoutSchema` pinned `architecture` to `EM_X86_64` / 64 / little. The rest of the adapter already goes through class- and byte-order-aware pyelftools structures (`image.structs.Elf_Sym`, `Elf_Dyn`, header sizes), so 32-bit and big-endian layouts need no other parsing change. Expected: an ELF32 ARM or MIPS executable returns the same layout report as an x86-64 one, with `architecture` naming its real machine, class and byte order. ## Change and scope - **Bridge.** `PROFILES` maps each admitted `(e_machine, class bits, byte order)` to the architecture pwntools must also report. The identification fields are read in the declared byte order, ELF32 headers are bounds-checked at 52 bytes, and pwntools' own `arch`/`bits`/`endian` must match the admitted profile after loading. - **Schema.** `architecture` is a union of the exact admitted combinations, so a report such as `EM_386` / 64 or `EM_ARM` / big still fails validation. - **Evidence.** The subject architecture previously always said `x86_64`. It now follows the reported machine (`x86_64`, `x86`, `arm`, `mips`). - **Verification.** `verify:binary:layout` cross-compiles i386, ARM, MIPS and MIPSel executables and relocatable objects. They are never executed. The verifier compares section names, addresses, offsets and sizes with GNU `readelf` through both CLI and MCP. It also compares every symbol value/size and every relocation offset, numeric type and symbol table/index reference, retaining table/entry identities and checking the oracle row counts. The `real-binary-layout` workflow installs the four cross toolchains. Not included: AArch64, ELF32 big-endian ARM, MIPS64/n32, x32, or any machine outside the list above. GOT/PLT maps and mitigation indicators still come from the unchanged upstream heuristics, and their completeness stays `unknown`, as before. Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). This widens target-format admission of an existing read-only offline operation. It adds no execution, injection or mutation authority. Whether widening `architecture` needs a maintainer scope decision is asked on #6934 and is still pending. ## Contract and boundary impact - Semantic owner and earliest changed stage: `PwntoolsLayoutProvider`'s Python adapter (`layout_impl.py`) owns target admission. `binaryLayoutSchema` owns the output contract, and `BinaryLayoutService` composes Evidence. No superseded path remains. - Contribution scope fit and linked maintainer decision: existing read-only capability. Question posted on #6934, no decision recorded yet. - CLI and MCP/tool-catalog contract: input unchanged. The output `architecture` field widens from one literal combination to five admitted combinations, and the tool description states the new profile. - Provider, bridge, target-format, or platform compatibility: same pinned pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2 on Linux x64. Existing ELF64 x86-64 reports are unchanged. - Evidence, artifact, provenance, or reconstruction contract: the Evidence subject architecture now reflects the target instead of a constant `x86_64`. - Target selection, authority, upstream behavior, and cleanup details: unchanged. The target is never launched, and Unicorn PLT emulation is the existing upstream behavior. - Generated metadata, package, or installation impact: none committed. The catalog is regenerated by CI. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - `src/domain/native/binaryLayout.test.ts`: admits the four new combinations and rejects mismatched class/machine/byte-order combinations. It also accepts 16-byte `Elf32_Sym` entries for ELF32 relocations while still refusing undersized ELF32 entries and 16-byte entries in ELF64 reports. - `src/application/binaryDiagnostics/BinaryLayoutService.test.ts`: the Evidence subject names the reported machine. - `scripts/verify-binary-layout.mjs`: real ELF32 cases cross-checked against `readelf`. The existing AArch64 and core `unsupported_target` cases are kept. - Base reproduction or other evidence: on the base revision, `layout_impl.py:119-120` refuses any object with `EI_CLASS != 2` before parsing, as quoted in #6934. - User-visible CLI/MCP output: the real lane asserts, through both `inspect-binary-layout --json` and MCP, that each cross-compiled fixture reports `architecture` `{EM_386, 32, little}`, `{EM_ARM, 32, little}`, `{EM_MIPS, 32, big}` or `{EM_MIPS, 32, little}` and the expected `image_type`, and that every section's name, address, offset and size equals GNU `readelf -SW`. A verbatim report excerpt was not captured from the CI log. - Remaining proof gaps: ELF32 big-endian ARM, AArch64 and MIPS64 remain unsupported and unverified. GOT/PLT completeness on ARM/MIPS is reported as `unknown` and not asserted. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - Review follow-up at `27dc69f93`: `npm run check`, the pre-push `npm run check:fast`, Node syntax checking and `git diff --check` passed on macOS (Node 24.22.1, npm 11.16.0). The new ELF32 symbol/relocation comparisons have **not** been run against the real provider: isolated Linux x64 setup was stopped after sustained dependency-download throttling, and dispatching the fork's `Real offline binary layout` workflow was denied with HTTP 403. The successful real-provider run below covers the earlier `1443a20` revision. - `npx tsc -p . --noEmit` (Windows 11, Node 24.18.0): passed. - `npm run test:local -- <17 binary-diagnostics source test files>`: 231 passed, 2 skipped, 1 failed. The failure is `ByteEntropyService.test.ts` "parent directory moves during streaming", which fails identically on the unchanged base on Windows (rename semantics) and is unrelated. - `npx oxlint` on changed files: no new warnings compared with the base files. - `npm run verify:module-boundaries`: passed. - Pre-push hook (`typecheck`, `lint`): passed. - `Real offline binary layout` workflow on the fork (ubuntu-24.04, pinned pwntools 4.15.0 / pyelftools 0.33 / Unicorn 2.1.2): - [Run #3](https://github.com/salmanalmutairi1/rea/actions/runs/40146961060) at `1443a20`: **passed**. The verifier step took 232 s, covering the existing ELF64 cases and the eight ELF32 cases. - [Run #2](https://github.com/salmanalmutairi1/rea/actions/runs/40146442134) at `b8426c9`: the 20 existing ELF64 cases passed, then the first i686 report was refused as `unreadable_output` because the relocation check required 24-byte (`Elf64_Sym`) symbol entries. This was fixed in `1443a20`, with regression cases in `binaryLayout.test.ts` that fail on `b8426c9`. - [Run #1](https://github.com/salmanalmutairi1/rea/actions/runs/40144681350) at `ebaf7a4`: the 20 existing ELF64 cases passed, then the i686 fixture failed to compile because `--no-install-recommends` skipped the target libc headers. This was fixed in `b8426c9`. - Not run locally: `npm run docs:check` and boundary tests (`test:focused`), because the Windows build needs native Windows artifacts absent from a source checkout. ## Compatibility, safety, and release - Breaking changes or migration steps: none for existing callers. ELF64 x86-64 reports are unchanged. Consumers that assumed `architecture` was always `EM_X86_64` / 64 / little will now also see the ELF32 combinations. - Real Hopper/Ghidra, browser, or OS coverage: not applicable. Real pwntools coverage is the workflow above. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new authority. Targets are still read from an owned snapshot and never executed. ## Author readiness Human owner: @salmanalmutairi1 - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-03 13:39:04 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7607
No description provided.