morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #6998] [MERGED] feat(artifacts): inspect and extract Bun standalone module graphs #7560

New issue

Closed

opened 2026-11-03 13:38:42 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:38:42 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6998
Author: @mvanhorn
Created: 10/24/2026
Status: ✅ Merged
Merged: 10/29/2026
Merged by: @gensokyo-dev

Base: main ← Head: fix/bun-standalone-module-graph

📝 Commits (7)

a4daa97 feat(artifacts): inventory and extract Bun standalone executable module graphs
1d7f40f fix(artifacts): bound Bun section probes and reject truncated sections
7578638 feat(artifacts): inspect Bun standalone module graphs with bounded reads
5b59ba9 Merge remote-tracking branch 'refs/remotes/feature-batch/main' into feat/bun-standalone-integration
0aed61b style(artifacts): format Bun reader integration
9bdcf56 fix(contracts): describe static Bun inventory facts
099c511 fix(artifacts): bound Bun metadata reads and combined path growth

📊 Changes

36 files changed (+3192 additions, -16 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+3 -0)
📝 .agents/skills/reverse-engineer-anything/references/native-and-artifacts.md (+17 -0)
➕ docs/bun-standalone.md (+62 -0)
📝 docs/cli.md (+5 -0)
📝 docs/design/limits-and-recovery.md (+14 -0)
📝 package.json (+1 -0)
➕ scripts/verify-bun-standalone.mjs (+113 -0)
📝 src/artifacts/ArtifactEmbeddedFormatProbes.ts (+19 -0)
📝 src/artifacts/ArtifactPaths.test.ts (+10 -0)
📝 src/artifacts/ArtifactPaths.ts (+32 -12)
📝 src/artifacts/ArtifactReader.ts (+1 -0)
📝 src/artifacts/ArtifactReaderAdmission.ts (+3 -1)
➕ src/artifacts/BunStandaloneArtifactReader.ts (+408 -0)
➕ src/artifacts/bun/BunModuleGraph.test.ts (+355 -0)
➕ src/artifacts/bun/BunModuleGraphLayout.ts (+835 -0)
➕ src/artifacts/bun/BunModuleGraphMembers.ts (+365 -0)
➕ src/artifacts/bun/BunStandalone.fixture.ts (+246 -0)
➕ src/artifacts/bun/BunStandaloneAdmission.ts (+9 -0)
➕ src/artifacts/bun/BunStandaloneArtifactReader.test.ts (+64 -0)
📝 src/artifacts/createArtifactReader.ts (+11 -0)

...and 16 more files

📄 Description

Outcome

CLI/MCP artifact inspection and extraction now recover Bun standalone module graphs from Mach-O __BUN,__bun, PE/ELF .bun sections and legacy appended trailers. This completes the existing feature PR within static artifact authority; targets are never executed and no provider is added.

Known 36/52-byte layouts are structural labels, not detected compiler versions. Inventory retains raw flags/enums, names or lossless hex, module identities, source extents and hashes; bytecode and Bun sourcemaps remain opaque stored regions. Malformed pointers, ambiguous layouts and unsupported formats fail explicitly.

Implementation

The current artifact factory serves both adapters. ArtifactFileSource owns selected-path validation, hash binding, buffered streams and cleanup retry. Completed probes are reused only for the same admitted source state. Metadata reads use exact extents with a 256 KiB / 1,024-range cache, plus a 256 MiB cumulative physical-read budget checked before allocation/I/O. Module-count admission precedes table allocation. The shared path owner preflights a 250,000-prefix combined tree budget before allocating destinations; shared prefixes count once.

Reader-owned entry_point_logical_path enables exact retained-inventory lookup and selected content-node reads without reconstructing virtual paths. Full Evidence remains accessible. Unsafe, non-UTF-8 or deeper-than-256-part virtual names use indexed destinations while retaining original source facts and limitations. Capacity refusals name the effective limit and recovery; no runtime override or partial extraction is claimed.

Executed validation

Final 099c511ab: 97 focused tests / 9 files passed, including actual CLI/MCP, shared .NET paths, cleanup and lifecycle checks. Typecheck/lint, docs, catalog generation/check, formatting and discovery of all 2,847 files passed.
Both P1 review regressions fail at 9bdcf5612 and pass after the fix: scattered tiny pointers amplify page reads; disjoint deep paths bypass combined tree admission. A shared-prefix CLI control stays complete. Additional regressions prove cumulative read refusal performs no next I/O and failed path reservations consume no partial capacity. Both threads have root-cause and structural-fix explanations.
The original reader on current contracts fails four earlier boundary regressions: full-payload metadata reads, repeated probes, buffered-producer cleanup and selected-parent replacement. The integrated reader passes them, plus deep-name preservation and pre-allocation count admission.
Six real inputs passed CLI inspection/extraction and SDK stdio MCP checks again after the P1 fixes. Independent LLVM section inventories and Python struct/range hashes verified all 1,373 stored regions, source SHA and extracted bytes. Owned outputs were removed; no guard stops or surviving children occurred.
Inputs: digest-verified Bun 1.3.3/1.4.4 trusted compilers produced legacy appended ELF and current PE/ELF/Mach-O fixtures, bytecode/maps and a 64 MiB asset. Public OpenCode 1.19.5 (214 MB, release digest verified) supplies 1,356 modules. Targets were never executed; Windows/macOS inputs were parsed on Linux. Other Bun releases and REA running on those hosts are unverified.
Earlier broad related checks: 329 passed / 1 Node SEA assertion failed; pure main a108a1b81 independently reproduces both variants (39 passed / 2 failed). No Node SEA behavior was changed. Final 099c511ab passed every selected upstream CI lane (run 40889395271): 21,960 source tests / 16 skipped and all eight built shards. Final Greptile review completed with 5/5 confidence, 36 files reviewed, zero new comments and no unresolved threads. No merge bypass is needed. An earlier PR-owned description-wording failure was corrected and rechecked, not bypassed.

Completed task costs

Two paired fresh-process runs, in alternating/reversed order, found OpenCode's entry module and content digest. Before (a8bc0859a): generic declared-ID lookup through inventory pages. After (099c511ab): reader-owned exact logical path and the same content-node lookup. Both runs selected the artifact,session,application MCP tool families. This includes initialization/instructions, compact input/output schemas and catalog, client validation, open, inspect, retained views and close.

Workflow	Calls / occurrence pages	Whole-frame token estimate	Elapsed / CPU	Peak family RSS
Original reader + ID scan	20 / 16	719,910	6.04–6.05 s / 9.47–9.49 s	586–615 MB
Shared reader + exact path	5 / 1	60,678	5.92–6.01 s / 8.86–9.39 s	549–552 MB

Tokenx 1.3.0 estimates count all serialized JSON-RPC frames, including catalog and both text/structured replies; these are not billed model tokens. Catalog contributions are 51,886 before / 51,884 after. This measured ID-scan workflow uses 91.6% fewer estimated tokens; other caller-derived lookup strategies were not measured. Filesystem caches were warm, with fresh client/server processes; true cold-cache and model-inference costs are unmeasured. Small timing differences do not establish a general speedup.

Separate two-CPU CLI summary + retained-Evidence observations: 64 MiB fixture 4.39 s / 5.44 CPU s / 374 MB before versus 4.18 s / 5.34 CPU s / 348 MB after; OpenCode 6.04 s / 7.27 CPU s / 447 MB before versus 6.53 s / 7.62 CPU s / 424 MB after. These are single warm-cache observations: OpenCode CLI was slower while using less memory. Real-data tasks used a 1.5 GiB process-family guard. Full extraction was correctness-checked separately; independent verification CPU is unmeasured.

Author readiness

Human owner: @mvanhorn

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6998 **Author:** [@mvanhorn](https://github.com/mvanhorn) **Created:** 10/24/2026 **Status:** ✅ Merged **Merged:** 10/29/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `fix/bun-standalone-module-graph` --- ### 📝 Commits (7) - [`a4daa97`](https://github.com/morluto/rea/commit/a4daa972823dfcc4e051f9b7279e65f9b98bc76b) feat(artifacts): inventory and extract Bun standalone executable module graphs - [`1d7f40f`](https://github.com/morluto/rea/commit/1d7f40f10ce87c11e1337fbc53045349560b834e) fix(artifacts): bound Bun section probes and reject truncated sections - [`7578638`](https://github.com/morluto/rea/commit/7578638fcfe0cc297bf2e6e206962a2e9fbf9b2b) feat(artifacts): inspect Bun standalone module graphs with bounded reads - [`5b59ba9`](https://github.com/morluto/rea/commit/5b59ba9973823d0179651088df3dac50ae2f8cab) Merge remote-tracking branch 'refs/remotes/feature-batch/main' into feat/bun-standalone-integration - [`0aed61b`](https://github.com/morluto/rea/commit/0aed61b6ac72500fe3347b3f28ec7eae682564d8) style(artifacts): format Bun reader integration - [`9bdcf56`](https://github.com/morluto/rea/commit/9bdcf5612431771970613bef99b027b3c054594c) fix(contracts): describe static Bun inventory facts - [`099c511`](https://github.com/morluto/rea/commit/099c511abb66a308cc1a301edf642d1d417eafa9) fix(artifacts): bound Bun metadata reads and combined path growth ### 📊 Changes **36 files changed** (+3192 additions, -16 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+3 -0) 📝 `.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md` (+17 -0) ➕ `docs/bun-standalone.md` (+62 -0) 📝 `docs/cli.md` (+5 -0) 📝 `docs/design/limits-and-recovery.md` (+14 -0) 📝 `package.json` (+1 -0) ➕ `scripts/verify-bun-standalone.mjs` (+113 -0) 📝 `src/artifacts/ArtifactEmbeddedFormatProbes.ts` (+19 -0) 📝 `src/artifacts/ArtifactPaths.test.ts` (+10 -0) 📝 `src/artifacts/ArtifactPaths.ts` (+32 -12) 📝 `src/artifacts/ArtifactReader.ts` (+1 -0) 📝 `src/artifacts/ArtifactReaderAdmission.ts` (+3 -1) ➕ `src/artifacts/BunStandaloneArtifactReader.ts` (+408 -0) ➕ `src/artifacts/bun/BunModuleGraph.test.ts` (+355 -0) ➕ `src/artifacts/bun/BunModuleGraphLayout.ts` (+835 -0) ➕ `src/artifacts/bun/BunModuleGraphMembers.ts` (+365 -0) ➕ `src/artifacts/bun/BunStandalone.fixture.ts` (+246 -0) ➕ `src/artifacts/bun/BunStandaloneAdmission.ts` (+9 -0) ➕ `src/artifacts/bun/BunStandaloneArtifactReader.test.ts` (+64 -0) 📝 `src/artifacts/createArtifactReader.ts` (+11 -0) _...and 16 more files_ </details> ### 📄 Description ## Outcome CLI/MCP artifact inspection and extraction now recover Bun standalone module graphs from Mach-O `__BUN,__bun`, PE/ELF `.bun` sections and legacy appended trailers. This completes the existing feature PR within static artifact authority; targets are never executed and no provider is added. Known 36/52-byte layouts are structural labels, not detected compiler versions. Inventory retains raw flags/enums, names or lossless hex, module identities, source extents and hashes; bytecode and Bun sourcemaps remain opaque stored regions. Malformed pointers, ambiguous layouts and unsupported formats fail explicitly. ## Implementation The current artifact factory serves both adapters. `ArtifactFileSource` owns selected-path validation, hash binding, buffered streams and cleanup retry. Completed probes are reused only for the same admitted source state. Metadata reads use exact extents with a 256 KiB / 1,024-range cache, plus a 256 MiB cumulative physical-read budget checked before allocation/I/O. Module-count admission precedes table allocation. The shared path owner preflights a 250,000-prefix combined tree budget before allocating destinations; shared prefixes count once. Reader-owned `entry_point_logical_path` enables exact retained-inventory lookup and selected content-node reads without reconstructing virtual paths. Full Evidence remains accessible. Unsafe, non-UTF-8 or deeper-than-256-part virtual names use indexed destinations while retaining original source facts and limitations. Capacity refusals name the effective limit and recovery; no runtime override or partial extraction is claimed. ## Executed validation - Final `099c511ab`: 97 focused tests / 9 files passed, including actual CLI/MCP, shared .NET paths, cleanup and lifecycle checks. Typecheck/lint, docs, catalog generation/check, formatting and discovery of all 2,847 files passed. - Both P1 review regressions fail at `9bdcf5612` and pass after the fix: scattered tiny pointers amplify page reads; disjoint deep paths bypass combined tree admission. A shared-prefix CLI control stays complete. Additional regressions prove cumulative read refusal performs no next I/O and failed path reservations consume no partial capacity. Both threads have root-cause and structural-fix explanations. - The original reader on current contracts fails four earlier boundary regressions: full-payload metadata reads, repeated probes, buffered-producer cleanup and selected-parent replacement. The integrated reader passes them, plus deep-name preservation and pre-allocation count admission. - Six real inputs passed CLI inspection/extraction and SDK stdio MCP checks again after the P1 fixes. Independent LLVM section inventories and Python struct/range hashes verified all 1,373 stored regions, source SHA and extracted bytes. Owned outputs were removed; no guard stops or surviving children occurred. - Inputs: digest-verified Bun 1.3.3/1.4.4 trusted compilers produced legacy appended ELF and current PE/ELF/Mach-O fixtures, bytecode/maps and a 64 MiB asset. Public OpenCode 1.19.5 (214 MB, release digest verified) supplies 1,356 modules. Targets were never executed; Windows/macOS inputs were parsed on Linux. Other Bun releases and REA running on those hosts are unverified. - Earlier broad related checks: 329 passed / 1 Node SEA assertion failed; pure main `a108a1b81` independently reproduces both variants (39 passed / 2 failed). No Node SEA behavior was changed. Final `099c511ab` passed every selected upstream CI lane ([run 40889395271](https://github.com/morluto/rea/actions/runs/40889395271)): 21,960 source tests / 16 skipped and all eight built shards. Final Greptile review completed with 5/5 confidence, 36 files reviewed, zero new comments and no unresolved threads. No merge bypass is needed. An earlier PR-owned description-wording failure was corrected and rechecked, not bypassed. ## Completed task costs Two paired fresh-process runs, in alternating/reversed order, found OpenCode's entry module and content digest. Before (`a8bc0859a`): generic declared-ID lookup through inventory pages. After (`099c511ab`): reader-owned exact logical path and the same content-node lookup. Both runs selected the `artifact,session,application` MCP tool families. This includes initialization/instructions, compact input/output schemas and catalog, client validation, open, inspect, retained views and close. | Workflow | Calls / occurrence pages | Whole-frame token estimate | Elapsed / CPU | Peak family RSS | | --- | --- | --- | --- | --- | | Original reader + ID scan | 20 / 16 | 719,910 | 6.04–6.05 s / 9.47–9.49 s | 586–615 MB | | Shared reader + exact path | 5 / 1 | 60,678 | 5.92–6.01 s / 8.86–9.39 s | 549–552 MB | Tokenx 1.3.0 estimates count all serialized JSON-RPC frames, including catalog and both text/structured replies; these are not billed model tokens. Catalog contributions are 51,886 before / 51,884 after. This measured ID-scan workflow uses 91.6% fewer estimated tokens; other caller-derived lookup strategies were not measured. Filesystem caches were warm, with fresh client/server processes; true cold-cache and model-inference costs are unmeasured. Small timing differences do not establish a general speedup. Separate two-CPU CLI summary + retained-Evidence observations: 64 MiB fixture 4.39 s / 5.44 CPU s / 374 MB before versus 4.18 s / 5.34 CPU s / 348 MB after; OpenCode 6.04 s / 7.27 CPU s / 447 MB before versus 6.53 s / 7.62 CPU s / 424 MB after. These are single warm-cache observations: OpenCode CLI was slower while using less memory. Real-data tasks used a 1.5 GiB process-family guard. Full extraction was correctness-checked separately; independent verification CPU is unmeasured. ## Author readiness Human owner: @mvanhorn - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:38:42 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7560

No description provided.

Rows
Columns