[PR #2456] [MERGED] feat(native): read embedded cargo-auditable Rust dependency lists #5242

New issue

Closed

opened 2026-10-31 01:05:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:05:05 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2456
Author: @mvanhorn
Created: 10/13/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @morluto

Base: main ← Head: fix/rust-auditable-dependencies

📝 Commits (10+)

03cae44 feat(native): read embedded cargo-auditable Rust dependency lists
25242c6 fix(native): trim PE section padding and compute dependency cycles iteratively
ac9634e fix(native): ignore SHT_NOBITS sections and validate dependency edges linearly
4159005 fix(native): bound compiler path scanning and retained hashes
7594b59 fix(rust): reject unknown formats and invalid dependency edges
f204311 fix(rust): pass package arrays as named parameterized test cases
0b84e34 fix(rust): admit graph expansion and preserve resource units
a1d8431 chore(rust): reconcile captured main and moved README guides
be441d8 fix(rust): reject malformed packages, false cycles, and hidden compiler paths
ba571f8 Merge remote-tracking branch 'origin/main' into fix/rust-auditable-dependencies

📊 Changes

59 files changed (+2767 additions, -12 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+8 -0)
📝 .agents/skills/reverse-engineer-anything/references/native-and-artifacts.md (+12 -0)
📝 README.md (+1 -0)
📝 docs/.vitepress/config.ts (+1 -0)
📝 docs/cli.md (+12 -0)
📝 docs/index.md (+1 -1)
📝 docs/readme/README_ar.md (+1 -0)
📝 docs/readme/README_de.md (+1 -0)
📝 docs/readme/README_es.md (+1 -0)
📝 docs/readme/README_fa.md (+1 -0)
📝 docs/readme/README_fr.md (+1 -0)
📝 docs/readme/README_id.md (+1 -0)
📝 docs/readme/README_it.md (+1 -0)
📝 docs/readme/README_ja.md (+1 -0)
📝 docs/readme/README_ko.md (+1 -0)
📝 docs/readme/README_pl.md (+1 -0)
📝 docs/readme/README_pt-BR.md (+1 -0)
📝 docs/readme/README_ru.md (+1 -0)
📝 docs/readme/README_th.md (+1 -0)
📝 docs/readme/README_uk.md (+1 -0)

...and 39 more files

📄 Description

Summary

Agents can now read the dependency list that cargo auditable embeds in a Rust binary. inspect_rust_binary / inspect-rust-binary reads the .dep-v0 section (__DATA,.dep-v0 on Mach-O) from one ELF, PE or thin Mach-O file without running it, and returns the packages, their versions and sources, the root package and the dependency edges. It answers for Rust the question inspect_go_binary answers for Go.

No issue is linked. This is a new static, read-only primitive within the existing format-inspection scope, proposed here with the implementation.

Demo

Full-quality MP4

Simulated demo rendered from real output. Every value on screen is copied from rea inspect-rust-binary on this branch, run against an auditable and a plain build of the same one-line crate.

Problem and expected behavior

Rust binaries built with cargo auditable carry their full crate dependency tree as zlib-compressed JSON in a .dep-v0 section. REA had no Rust metadata lane, so an agent asked "which crates and versions are in this binary" had nothing to call short of a full disassembly.

With this change, one call returns the section name, file offset, compressed and uncompressed sizes, format version, packages in original order (name, version, source, kind, dependencies, root), the root package index, dependency edges, and any packages that sit on a dependency cycle. When the section is absent, auditable_metadata is null with the limitation "Absence of .dep-v0 does not mean the binary has no dependencies." Read-only /rustc/<commit>/ paths are reported separately as an inferred compiler commit.

Change and scope

Section lookup reuses the ELF, PE and Mach-O readers that inspect_go_binary already validates; they now also expose a list of named sections with their file ranges. PE content length is min(VirtualSize, SizeOfRawData), so file-alignment padding never reaches the decoder, and ELF SHT_NOBITS sections are never treated as stored bytes. The inflate is capped by a fixed ratio and an absolute maximum. The JSON is parsed with a strict schema, dependency indices are range-checked, and cycle members are found with an iterative strongly-connected-component pass, so long chains cannot exhaust the stack. Overlapping read-only sections are scanned once, and more than 64 distinct compiler paths is a typed resource failure rather than unbounded output.

Universal Mach-O files are refused with a typed unsupported error rather than picking a slice. Not included: recovering dependencies from binaries built without cargo auditable (panic strings, symbol names); that would be inference rather than an embedded record.

One naming note for review: the shared container readers keep their existing Go* names (GoImageSection and friends) to keep this diff small. If you would rather they move to neutral names now that two lanes use them, I can do that here or in a follow-up.

Contract and boundary impact

Semantic owner and earliest changed stage: src/rust/RustAuditableReader.ts (section payload), src/rust/RustBinaryImage.ts (section selection and compiler paths), schema in src/domain/rust/rustBinary.ts; section ranges added to src/go/Go{Elf,Pe,Macho}Image.ts.
Contribution scope fit and linked maintainer decision: static format inspection of a caller-selected file; no execution, injection, mutation or protection recovery, so not applicable.
CLI and MCP/tool-catalog contract: new inspect_rust_binary / inspect-rust-binary with one path input; registered next to the Go tool, effects entry read-only, catalog builders and capability inventory updated.
Provider, bridge, target-format, or platform compatibility: ELF, PE and thin Mach-O; cargo auditable format version 1.
Evidence, artifact, provenance, or reconstruction contract: package data is labeled as file content that is not authenticated; the compiler commit is labeled inferred.
Target selection, authority, upstream behavior, and cleanup details: reads one admitted regular file.
Generated metadata, package, or installation impact: skill text in .agents/skills/reverse-engineer-anything/, docs/rust-binaries.md, a README row; build outputs not committed.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated: src/rust/RustAuditableReader.test.ts (schema, inflate bound, out-of-range indices, cycle membership including members reached through finished vertices, a 20,000-package chain), src/application/rust/RustBinaryService.test.ts, tests/boundary/rust/rustBinaryImage.test.ts (ELF, PE with aligned raw size, Mach-O, missing section, universal refused, SHT_NOBITS, compiler-path cap), tests/acceptance/rustBinaryCli.test.ts, tests/boundary/mcp/rustBinaryMcp.test.ts; Go boundary tests rerun unchanged.
Base reproduction or other evidence: executed on two real macOS arm64 builds of a one-line crate that depends on itoa, one with cargo auditable build --release and one with plain cargo build --release.
User-visible CLI/MCP output: the auditable build reports __DATA,.dep-v0 at file offset 298088, 120 compressed and 165 uncompressed bytes, format 1, packages itoa 1.0.18 (crates.io) and sample 0.1.0 (local, root, depends on 0). That matches an independent read of the same section with otool and zlib. The plain build reports auditable_metadata: null with the absence limitation. Both report the inferred compiler commit b940084d7eb6a299eb4bfeb8e34901bc051e7ac4, which matches rustc -vV for the toolchain that built them. A full result is about 960 tokens.
Remaining proof gaps: no real Windows or Linux build was checked; PE and ELF are covered by fixtures.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

All on macOS 27 with Node 26 and the pinned npm 11.16.0, on this branch rebased on a2d8b94:

npm run check                  # pass (typecheck, lint, format, knip, metadata, jscpd, architecture guards)
npm run test:focused -- <Rust and Go boundary, acceptance, MCP and unit tests>   # pass: 11 files, 156 tests
npm run docs:check             # pass
npm run verify:test-discovery  # pass
rea inspect-rust-binary target/release/sample   # both real builds, results above

Compatibility, safety, and release

Breaking changes or migration steps: none. Additive tool; inspect_go_binary output is unchanged.
Real Hopper/Ghidra, browser, or OS coverage: macOS only for real binaries.
Package or release metadata impact: skill text and docs only.
Security, privacy, process, or containment review: reads one caller-selected file; bounded inflate, scan coverage and retained output; no process launch, writes or network.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2456 **Author:** [@mvanhorn](https://github.com/mvanhorn) **Created:** 10/13/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/rust-auditable-dependencies` --- ### 📝 Commits (10+) - [`03cae44`](https://github.com/morluto/rea/commit/03cae44ebbc54a9d546e0445c9f1abdcb9acab2a) feat(native): read embedded cargo-auditable Rust dependency lists - [`25242c6`](https://github.com/morluto/rea/commit/25242c6c52a9eca8cf6dc5f6487dad0a2b5ed16f) fix(native): trim PE section padding and compute dependency cycles iteratively - [`ac9634e`](https://github.com/morluto/rea/commit/ac9634ed56ee0e601e8b55dc7c64034b8b8271b1) fix(native): ignore SHT_NOBITS sections and validate dependency edges linearly - [`4159005`](https://github.com/morluto/rea/commit/415900530122eb7f3caab4ffa8707f72d17f37bb) fix(native): bound compiler path scanning and retained hashes - [`7594b59`](https://github.com/morluto/rea/commit/7594b592efff15c57ef2bfdb5410516368767a59) fix(rust): reject unknown formats and invalid dependency edges - [`f204311`](https://github.com/morluto/rea/commit/f204311a88fce640dd68e364e22a24eef0387a2a) fix(rust): pass package arrays as named parameterized test cases - [`0b84e34`](https://github.com/morluto/rea/commit/0b84e3413ae91b4822e7a715ab17b4109b6b624a) fix(rust): admit graph expansion and preserve resource units - [`a1d8431`](https://github.com/morluto/rea/commit/a1d8431ba854757aeed828df7634a38a861fd689) chore(rust): reconcile captured main and moved README guides - [`be441d8`](https://github.com/morluto/rea/commit/be441d8eec05aac0f70f1ee93060bba5c1dcce02) fix(rust): reject malformed packages, false cycles, and hidden compiler paths - [`ba571f8`](https://github.com/morluto/rea/commit/ba571f8a9d600d66ae27103a2dce1d104ed6e96a) Merge remote-tracking branch 'origin/main' into fix/rust-auditable-dependencies ### 📊 Changes **59 files changed** (+2767 additions, -12 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+8 -0) 📝 `.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md` (+12 -0) 📝 `README.md` (+1 -0) 📝 `docs/.vitepress/config.ts` (+1 -0) 📝 `docs/cli.md` (+12 -0) 📝 `docs/index.md` (+1 -1) 📝 `docs/readme/README_ar.md` (+1 -0) 📝 `docs/readme/README_de.md` (+1 -0) 📝 `docs/readme/README_es.md` (+1 -0) 📝 `docs/readme/README_fa.md` (+1 -0) 📝 `docs/readme/README_fr.md` (+1 -0) 📝 `docs/readme/README_id.md` (+1 -0) 📝 `docs/readme/README_it.md` (+1 -0) 📝 `docs/readme/README_ja.md` (+1 -0) 📝 `docs/readme/README_ko.md` (+1 -0) 📝 `docs/readme/README_pl.md` (+1 -0) 📝 `docs/readme/README_pt-BR.md` (+1 -0) 📝 `docs/readme/README_ru.md` (+1 -0) 📝 `docs/readme/README_th.md` (+1 -0) 📝 `docs/readme/README_uk.md` (+1 -0) _...and 39 more files_ </details> ### 📄 Description ## Summary Agents can now read the dependency list that `cargo auditable` embeds in a Rust binary. `inspect_rust_binary` / `inspect-rust-binary` reads the `.dep-v0` section (`__DATA,.dep-v0` on Mach-O) from one ELF, PE or thin Mach-O file without running it, and returns the packages, their versions and sources, the root package and the dependency edges. It answers for Rust the question `inspect_go_binary` answers for Go. No issue is linked. This is a new static, read-only primitive within the existing format-inspection scope, proposed here with the implementation. ## Demo ![Rust dependency list demo](https://raw.githubusercontent.com/mvanhorn/pr-demo-assets/main/rea/2026-10-12-026-feat-rust-auditable-dependencies/preview.gif) [Full-quality MP4](https://raw.githubusercontent.com/mvanhorn/pr-demo-assets/main/rea/2026-10-12-026-feat-rust-auditable-dependencies/out.mp4) Simulated demo rendered from real output. Every value on screen is copied from `rea inspect-rust-binary` on this branch, run against an auditable and a plain build of the same one-line crate. ## Problem and expected behavior Rust binaries built with `cargo auditable` carry their full crate dependency tree as zlib-compressed JSON in a `.dep-v0` section. REA had no Rust metadata lane, so an agent asked "which crates and versions are in this binary" had nothing to call short of a full disassembly. With this change, one call returns the section name, file offset, compressed and uncompressed sizes, format version, packages in original order (`name`, `version`, `source`, `kind`, `dependencies`, `root`), the root package index, dependency edges, and any packages that sit on a dependency cycle. When the section is absent, `auditable_metadata` is `null` with the limitation "Absence of .dep-v0 does not mean the binary has no dependencies." Read-only `/rustc/<commit>/` paths are reported separately as an inferred compiler commit. ## Change and scope Section lookup reuses the ELF, PE and Mach-O readers that `inspect_go_binary` already validates; they now also expose a list of named sections with their file ranges. PE content length is `min(VirtualSize, SizeOfRawData)`, so file-alignment padding never reaches the decoder, and ELF `SHT_NOBITS` sections are never treated as stored bytes. The inflate is capped by a fixed ratio and an absolute maximum. The JSON is parsed with a strict schema, dependency indices are range-checked, and cycle members are found with an iterative strongly-connected-component pass, so long chains cannot exhaust the stack. Overlapping read-only sections are scanned once, and more than 64 distinct compiler paths is a typed resource failure rather than unbounded output. Universal Mach-O files are refused with a typed unsupported error rather than picking a slice. Not included: recovering dependencies from binaries built without `cargo auditable` (panic strings, symbol names); that would be inference rather than an embedded record. One naming note for review: the shared container readers keep their existing `Go*` names (`GoImageSection` and friends) to keep this diff small. If you would rather they move to neutral names now that two lanes use them, I can do that here or in a follow-up. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/rust/RustAuditableReader.ts` (section payload), `src/rust/RustBinaryImage.ts` (section selection and compiler paths), schema in `src/domain/rust/rustBinary.ts`; section ranges added to `src/go/Go{Elf,Pe,Macho}Image.ts`. - Contribution scope fit and linked maintainer decision: static format inspection of a caller-selected file; no execution, injection, mutation or protection recovery, so not applicable. - CLI and MCP/tool-catalog contract: new `inspect_rust_binary` / `inspect-rust-binary` with one `path` input; registered next to the Go tool, effects entry read-only, catalog builders and capability inventory updated. - Provider, bridge, target-format, or platform compatibility: ELF, PE and thin Mach-O; `cargo auditable` format version 1. - Evidence, artifact, provenance, or reconstruction contract: package data is labeled as file content that is not authenticated; the compiler commit is labeled `inferred`. - Target selection, authority, upstream behavior, and cleanup details: reads one admitted regular file. - Generated metadata, package, or installation impact: skill text in `.agents/skills/reverse-engineer-anything/`, `docs/rust-binaries.md`, a README row; build outputs not committed. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: `src/rust/RustAuditableReader.test.ts` (schema, inflate bound, out-of-range indices, cycle membership including members reached through finished vertices, a 20,000-package chain), `src/application/rust/RustBinaryService.test.ts`, `tests/boundary/rust/rustBinaryImage.test.ts` (ELF, PE with aligned raw size, Mach-O, missing section, universal refused, `SHT_NOBITS`, compiler-path cap), `tests/acceptance/rustBinaryCli.test.ts`, `tests/boundary/mcp/rustBinaryMcp.test.ts`; Go boundary tests rerun unchanged. - Base reproduction or other evidence: executed on two real macOS arm64 builds of a one-line crate that depends on `itoa`, one with `cargo auditable build --release` and one with plain `cargo build --release`. - User-visible CLI/MCP output: the auditable build reports `__DATA,.dep-v0` at file offset 298088, 120 compressed and 165 uncompressed bytes, format 1, packages `itoa` 1.0.18 (crates.io) and `sample` 0.1.0 (local, root, depends on 0). That matches an independent read of the same section with `otool` and zlib. The plain build reports `auditable_metadata: null` with the absence limitation. Both report the inferred compiler commit `b940084d7eb6a299eb4bfeb8e34901bc051e7ac4`, which matches `rustc -vV` for the toolchain that built them. A full result is about 960 tokens. - Remaining proof gaps: no real Windows or Linux build was checked; PE and ELF are covered by fixtures. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed All on macOS 27 with Node 26 and the pinned npm 11.16.0, on this branch rebased on `a2d8b94`: ``` npm run check # pass (typecheck, lint, format, knip, metadata, jscpd, architecture guards) npm run test:focused -- <Rust and Go boundary, acceptance, MCP and unit tests> # pass: 11 files, 156 tests npm run docs:check # pass npm run verify:test-discovery # pass rea inspect-rust-binary target/release/sample # both real builds, results above ``` ## Compatibility, safety, and release - Breaking changes or migration steps: none. Additive tool; `inspect_go_binary` output is unchanged. - Real Hopper/Ghidra, browser, or OS coverage: macOS only for real binaries. - Package or release metadata impact: skill text and docs only. - Security, privacy, process, or containment review: reads one caller-selected file; bounded inflate, scan coverage and retained output; no process launch, writes or network. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:05:05 +00:00