morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #6170] [MERGED] feat(native): export function pseudocode to resumable directories #7182

New issue

Closed

opened 2026-11-01 21:45:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:45:58 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6170
Author: @CryptoJones
Created: 10/21/2026
Status: ✅ Merged
Merged: 10/23/2026
Merged by: @gensokyo-dev

Base: main ← Head: feat/6090-decompile-directory

📝 Commits (10+)

7720c44 feat(decompile): write function pseudocode to a directory and return a manifest
142d936 fix(decompile): keep unknown export identity unresumable and recover torn journals
c210e7a fix(decompile): refuse resume of a missing export and repair journals by byte offset
dad6705 fix(decompile): keep a function's first file name across failed retries
ac0e72a fix(decompile): never follow a journal symlink and check procedure_info identity
b82e141 fix(decompile): bound returned caveats and keep verified checksums
031762e fix(decompile): keep each function's complete caveats in the index
56ecf41 fix(decompile): report external functions instead of failing them
9b92b9a fix(decompile): report finalization failures after a cancelled export
2741c83 fix(decompile): remove a created output directory that cannot be secured

📊 Changes

33 files changed (+6161 additions, -303 deletions)

View changed files

📝 docs/native-investigation.md (+40 -0)
➕ docs/verification/native-decompile-directory-2026-10-23.md (+223 -0)
📝 src/application/AnalysisProvider.ts (+2 -0)
📝 src/application/CapabilityInventory.ts (+1 -0)
📝 src/application/DirectAnalysis.ts (+19 -7)
📝 src/application/EnhancedTools.ts (+9 -0)
📝 src/application/WorkflowEvidence.test.ts (+153 -0)
📝 src/application/WorkflowEvidence.ts (+73 -1)
📝 src/application/binary/BinarySession.ts (+2 -0)
📝 src/application/binary/BinarySessionRecords.ts (+11 -0)
➕ src/application/decompileDirectory/decompileAll.test.ts (+1706 -0)
➕ src/application/decompileDirectory/decompileAll.ts (+994 -0)
➕ src/application/decompileDirectory/exportResources.ts (+262 -0)
➕ src/application/decompileDirectory/exportTree.ts (+970 -0)
➕ src/application/decompileDirectory/functionFile.ts (+85 -0)
➕ src/application/decompileDirectory/inputError.ts (+10 -0)
➕ src/application/decompileDirectory/selection.ts (+169 -0)
📝 src/cli/coreBinaryCommands.ts (+2 -0)
➕ src/cli/decompileDirectoryCommand.ts (+119 -0)
📝 src/cliCommandNames.ts (+1 -0)

...and 13 more files

📄 Description

Summary

Export native pseudocode into a caller-selected directory while returning only a manifest through decompile_all / rea decompile-all. Refs #6090. Successful files survive partial decompilation failures and can be verified and resumed.

Problem and expected behavior

Returning hundreds of function bodies consumes the agent context before it can inspect the relevant files. The directory workflow keeps names, body ranges, provider/profile identity and complete per-function caveats on disk, with selected/written/external/failed/remaining counts inline.

Change and scope

Reuse existing native provider operations. The shared application workflow owns selection, append-only journal and index; CLI/MCP bind named contracts. Hold the directory lease through cleanup, retain failed file/staging cleanup under the session, and remove only verified owned initialization entries. Caller bounds accept uppercase hex and encoded address spaces; aliases participate in filtering before address coalescing. Journal/index validation shares one schema and committed records retain parsed snapshots. Bound inline failures to 2,048 JSON-encoded UTF-8 bytes and caveats to an encoded-text budget, retaining complete index facts. Case-folded filename reservations avoid cross-platform collisions; resumed journals use the same ownership rule. Stream existing-file hashes.

Contract and boundary impact

Semantic owner: shared directory workflow and its journal/resource owners; adapters only map/validate inputs and project Evidence.
Scope/authority: existing static analysis, provider helper processes and caller-selected local output. No new provider or target execution/injection/mutation; additional maintainer scope decision is not applicable.
CLI/MCP: additive decompile-all / decompile_all, address/name/prefix filters, attempt limit, resume, explicit unavailable DWARF layout with flat address fallback.
Evidence/provenance: manifest-only Evidence retains target/provider/profile, counts and limitations; the index retains full per-function caveats. Resume requires known matching target/profile identities and fresh file hashes. Snapshot replay is disabled for this filesystem workflow.
Compatibility/setup: existing providers remain the source of observations; unavailable procedure_info keeps body coverage unknown. No installer or package changes. Catalogs remain generated and ignored.

Evidence and regression coverage

Real Linux Ghidra 12.1.4 analyzed /usr/bin/sha256sum: 298 inventory entries, 236 files, 61 external entries and one retained decompiler failure. Two exports matched the baseline's addresses, statuses, body ranges, names, pseudocode and complete file SHA-256 values. Another 68 functions were written after the failing address; resume preserved successful output. Three real CLI calls verified uppercase bounds, chunked resume and an empty selection.

Original-PR controls reproduce both initialization lease races, lost failed-close ownership, unexpected-file deletion and uppercase-bound rejection. A production selection control also reproduces a hidden matching alias. A separate negative control caught mutable input retention before committed-index reuse; parsed snapshot ownership fixes it.

Real normalized result excerpt:

{"counts":{"selected":298,"excluded":0,"requested":298,"written":236,"failed":1,"external":61,"skipped":0,"remaining":0}}

Performance and scope: verification report. Count actual catalog discovery and whole completed tasks; tokenx values are estimates, not model-specific usage. Time/CPU vary and no general speedup is claimed. Complete raw verifier artifacts remain in the maintainer workspace.

Validation performed

npm run check — passed after the review fixes on d3e0e9682 (type/lint/format, metadata and architecture guards).
Focused production-boundary/profile suite — 226 passed across ten files after the main refresh. Earlier lease/session cleanup suite passed 112 with four macOS-only cases skipped; overlapping cases are not summed.
npm run docs:check — passed, including nine README conformance cases.
Real production MCP benchmark and three real CLI calls — passed with unchanged target bytes and equivalent exported files; detailed measurements and limits are in the report.
Original/negative controls — expected failures reproduced; repaired ownership, range and alias cases pass. All three new portable-path/long-name regressions fail against the previous published implementation; 94 relevant workflow/filesystem/CLI/MCP/schema tests pass after repair.
GitHub CI/review — final run 40017304578 is red from baseline failures. Exact current base/merge controls reproduce 11 source and 43 built failing cases with identical full diagnostics, plus the same browser checkpoint failure. All failed/cancelled logs are audited with no unclassified assertion failures; cancelled coverage remains unverified. Final attribution and bypass limits. Feature-specific findings are repaired; final automated review is 5/5 with no outstanding findings.

Compatibility, safety, and release

Additive feature. Real-provider verification is Linux Ghidra; real Hopper/IDA/macOS/Windows and very large metadata/journal histories remain unverified. DWARF compilation-unit grouping remains unavailable and is reported explicitly. Latest-record/index memory still scales with retained metadata; no 100,000-function capacity claim is made. Failed cleanup retains exact authority for session retry, including cancellation during finalization.

Author readiness

Human owner: @CryptoJones

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6170 **Author:** [@CryptoJones](https://github.com/CryptoJones) **Created:** 10/21/2026 **Status:** ✅ Merged **Merged:** 10/23/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `feat/6090-decompile-directory` --- ### 📝 Commits (10+) - [`7720c44`](https://github.com/morluto/rea/commit/7720c44cfd8861a546bc5556125bc7bd43d6182c) feat(decompile): write function pseudocode to a directory and return a manifest - [`142d936`](https://github.com/morluto/rea/commit/142d936562449746f6627b00a5e6bdc08a6e24c6) fix(decompile): keep unknown export identity unresumable and recover torn journals - [`c210e7a`](https://github.com/morluto/rea/commit/c210e7ad33dca6a2004513ab16d339ff5d167f3b) fix(decompile): refuse resume of a missing export and repair journals by byte offset - [`dad6705`](https://github.com/morluto/rea/commit/dad6705b8fcfa4ab80d55a8a0aba8c45863e35b9) fix(decompile): keep a function's first file name across failed retries - [`ac0e72a`](https://github.com/morluto/rea/commit/ac0e72a837dfc770f9d38b584c93ba992e81432a) fix(decompile): never follow a journal symlink and check procedure_info identity - [`b82e141`](https://github.com/morluto/rea/commit/b82e14114d3b0d1e7686fc22192288c7403b6ca8) fix(decompile): bound returned caveats and keep verified checksums - [`031762e`](https://github.com/morluto/rea/commit/031762eae0c029d77e4a6406cb44806fbde4d49c) fix(decompile): keep each function's complete caveats in the index - [`56ecf41`](https://github.com/morluto/rea/commit/56ecf4198d64193486609099e98202b03599c879) fix(decompile): report external functions instead of failing them - [`9b92b9a`](https://github.com/morluto/rea/commit/9b92b9ab4710d69acb2fbb4148de905c5c7d7f83) fix(decompile): report finalization failures after a cancelled export - [`2741c83`](https://github.com/morluto/rea/commit/2741c83ff18466cc209d9e94f9b0bf9ae15bd7d9) fix(decompile): remove a created output directory that cannot be secured ### 📊 Changes **33 files changed** (+6161 additions, -303 deletions) <details> <summary>View changed files</summary> 📝 `docs/native-investigation.md` (+40 -0) ➕ `docs/verification/native-decompile-directory-2026-10-23.md` (+223 -0) 📝 `src/application/AnalysisProvider.ts` (+2 -0) 📝 `src/application/CapabilityInventory.ts` (+1 -0) 📝 `src/application/DirectAnalysis.ts` (+19 -7) 📝 `src/application/EnhancedTools.ts` (+9 -0) 📝 `src/application/WorkflowEvidence.test.ts` (+153 -0) 📝 `src/application/WorkflowEvidence.ts` (+73 -1) 📝 `src/application/binary/BinarySession.ts` (+2 -0) 📝 `src/application/binary/BinarySessionRecords.ts` (+11 -0) ➕ `src/application/decompileDirectory/decompileAll.test.ts` (+1706 -0) ➕ `src/application/decompileDirectory/decompileAll.ts` (+994 -0) ➕ `src/application/decompileDirectory/exportResources.ts` (+262 -0) ➕ `src/application/decompileDirectory/exportTree.ts` (+970 -0) ➕ `src/application/decompileDirectory/functionFile.ts` (+85 -0) ➕ `src/application/decompileDirectory/inputError.ts` (+10 -0) ➕ `src/application/decompileDirectory/selection.ts` (+169 -0) 📝 `src/cli/coreBinaryCommands.ts` (+2 -0) ➕ `src/cli/decompileDirectoryCommand.ts` (+119 -0) 📝 `src/cliCommandNames.ts` (+1 -0) _...and 13 more files_ </details> ### 📄 Description ## Summary Export native pseudocode into a caller-selected directory while returning only a manifest through `decompile_all` / `rea decompile-all`. Refs #6090. Successful files survive partial decompilation failures and can be verified and resumed. ## Problem and expected behavior Returning hundreds of function bodies consumes the agent context before it can inspect the relevant files. The directory workflow keeps names, body ranges, provider/profile identity and complete per-function caveats on disk, with selected/written/external/failed/remaining counts inline. ## Change and scope Reuse existing native provider operations. The shared application workflow owns selection, append-only journal and index; CLI/MCP bind named contracts. Hold the directory lease through cleanup, retain failed file/staging cleanup under the session, and remove only verified owned initialization entries. Caller bounds accept uppercase hex and encoded address spaces; aliases participate in filtering before address coalescing. Journal/index validation shares one schema and committed records retain parsed snapshots. Bound inline failures to 2,048 JSON-encoded UTF-8 bytes and caveats to an encoded-text budget, retaining complete index facts. Case-folded filename reservations avoid cross-platform collisions; resumed journals use the same ownership rule. Stream existing-file hashes. ## Contract and boundary impact - Semantic owner: shared directory workflow and its journal/resource owners; adapters only map/validate inputs and project Evidence. - Scope/authority: existing static analysis, provider helper processes and caller-selected local output. No new provider or target execution/injection/mutation; additional maintainer scope decision is not applicable. - CLI/MCP: additive `decompile-all` / `decompile_all`, address/name/prefix filters, attempt limit, resume, explicit unavailable DWARF layout with flat address fallback. - Evidence/provenance: manifest-only Evidence retains target/provider/profile, counts and limitations; the index retains full per-function caveats. Resume requires known matching target/profile identities and fresh file hashes. Snapshot replay is disabled for this filesystem workflow. - Compatibility/setup: existing providers remain the source of observations; unavailable `procedure_info` keeps body coverage unknown. No installer or package changes. Catalogs remain generated and ignored. ## Evidence and regression coverage Real Linux Ghidra 12.1.4 analyzed `/usr/bin/sha256sum`: **298 inventory entries, 236 files, 61 external entries and one retained decompiler failure**. Two exports matched the baseline's addresses, statuses, body ranges, names, pseudocode and complete file SHA-256 values. Another 68 functions were written after the failing address; resume preserved successful output. Three real CLI calls verified uppercase bounds, chunked resume and an empty selection. Original-PR controls reproduce both initialization lease races, lost failed-close ownership, unexpected-file deletion and uppercase-bound rejection. A production selection control also reproduces a hidden matching alias. A separate negative control caught mutable input retention before committed-index reuse; parsed snapshot ownership fixes it. Real normalized result excerpt: ```json {"counts":{"selected":298,"excluded":0,"requested":298,"written":236,"failed":1,"external":61,"skipped":0,"remaining":0}} ``` Performance and scope: [verification report](https://github.com/CryptoJones/rea/blob/a09b1fac23be4a683de9fd9a6d7f7c1691c7d2ed/docs/verification/native-decompile-directory-2026-10-23.md). Count actual catalog discovery and whole completed tasks; `tokenx` values are estimates, not model-specific usage. Time/CPU vary and no general speedup is claimed. Complete raw verifier artifacts remain in the maintainer workspace. ## Validation performed - `npm run check` — passed after the review fixes on `d3e0e9682` (type/lint/format, metadata and architecture guards). - Focused production-boundary/profile suite — 226 passed across ten files after the main refresh. Earlier lease/session cleanup suite passed 112 with four macOS-only cases skipped; overlapping cases are not summed. - `npm run docs:check` — passed, including nine README conformance cases. - Real production MCP benchmark and three real CLI calls — passed with unchanged target bytes and equivalent exported files; detailed measurements and limits are in the report. - Original/negative controls — expected failures reproduced; repaired ownership, range and alias cases pass. All three new portable-path/long-name regressions fail against the previous published implementation; 94 relevant workflow/filesystem/CLI/MCP/schema tests pass after repair. - GitHub CI/review — final run 40017304578 is red from baseline failures. Exact current base/merge controls reproduce 11 source and 43 built failing cases with identical full diagnostics, plus the same browser checkpoint failure. All failed/cancelled logs are audited with no unclassified assertion failures; cancelled coverage remains unverified. [Final attribution and bypass limits](https://github.com/morluto/rea/pull/6170#issuecomment-6374010667). Feature-specific findings are repaired; final automated review is 5/5 with no outstanding findings. ## Compatibility, safety, and release Additive feature. Real-provider verification is Linux Ghidra; real Hopper/IDA/macOS/Windows and very large metadata/journal histories remain unverified. DWARF compilation-unit grouping remains unavailable and is reported explicitly. Latest-record/index memory still scales with retained metadata; no 100,000-function capacity claim is made. Failed cleanup retains exact authority for session retry, including cancellation during finalization. ## Author readiness Human owner: @CryptoJones - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:45:58 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-03 13:39:44 +00:00

[PR #7183] [MERGED] fix(setup): preserve Linux Hopper HTTP failure status #7699

bsakaguchi changed title from ~~[PR #6170] [MERGED] feat(native): export function pseudocode to resumable directories~~ to [PR #6170] feat(native): export function pseudocode to resumable directories

2026-11-04 16:32:36 +00:00

bsakaguchi changed title from ~~[PR #6170] feat(native): export function pseudocode to resumable directories~~ to [PR #6170] [MERGED] feat(native): export function pseudocode to resumable directories

2026-11-04 17:28:40 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7182

No description provided.

Rows
Columns