[GH-ISSUE #6090] [Feature] Decompile every function (or a filtered set) to a directory and return only a manifest #3725

New issue

Closed

opened 2026-10-31 00:43:24 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:43:24 +00:00

Owner

Originally created by @CryptoJones on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6090

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Native analysis or decompilation

Current limitation

The task "decompile this whole program so I can step through it" has no route that keeps the decompiled text out of the agent's context. Every option today returns pseudocode through the tool result: procedure_pseudo_code and analyze_function one function at a time, or batch_decompile for a bounded address list (#5930 is tightening that bound). An agent driving a local model therefore spends its window on function bodies it only needed to have on disk.

Observed (REA 6.7.0, Ghidra 12.1.4, a local 27B model with a 262K context behind an MCP harness that keeps roughly the first 50 KB of a tool result): on Delores (x86-64 ELF, not stripped, DWARF present, 10 MB), the prompt was to decompile the entire program. The model could only proceed function by function. About 45K tokens of the window were already the compact tool catalog, and a handful of analyze_function results exhausted the rest, so it never got past the first few functions. That run is what motivated #5660 (summary-by-default for analyze_function), which fixes the per-call size but not the per-function round trip: with #5660 the agent still needs one call per function and then pages each body.

A second observation from the same run, cause not established: after one oversized result, the next batch_decompile returned target_unavailable with "No app is open", and the model had to open_binary again, which re-imports and re-analyzes. I have not reproduced this in isolation and do not know whether the harness, a cancellation path like the one described in #5340, or something else closed the session. NOT_ENOUGH_INFO on the mechanism; mentioned because a bulk export would be hit by it hardest, and it may deserve its own bug once reproduced.

Use cases

"Give me the decompiled source tree" for a non-stripped binary with DWARF, so a person or agent can grep and read files instead of calling a tool per function.
Large programs (tens of thousands of functions) where the agent's job is to find the ten that matter; it should read an index, not stream everything.
Offline review on a second machine: copy the directory, no provider needed.

Desired outcome and success criteria

A tool on both adapters (MCP and CLI) that writes pseudocode for every function, or a filtered set, into a caller-chosen output directory, one file per function (or per compilation unit when DWARF gives one), and returns only a small manifest: counts (requested, written, failed, skipped), the output root, per-failure reasons, provider identity and analysis profile, and an Evidence ID. Never the code itself.

Success criteria:

The tool result stays under a few KB regardless of program size.
Each written file carries the function identity (name, canonical address, body ranges when non-contiguous) and the provider's limitations for that function, so a file read later is still attributable Evidence.
Per-function failures (decompiler error, deadline, unresolved identity) are recorded in the manifest with reasons and do not abort the run.
Runs on big binaries are resumable or chunked: a second invocation with the same target digest and profile skips functions already written and reports what remains, or the caller can bound a run by count or address range and continue from the manifest.
Optional layout from DWARF source file and line information (directory tree mirroring the compile units) when available; flat by address otherwise. Unavailable debug info is reported, not guessed.
Optional filter by namespace, name pattern, or address range.
Index file in the output root (JSON) that the agent can read selectively.

Expected interaction with targets

Read existing files or captured data, Write or change files/configuration

Target and scope details (optional)

Static provider analysis only; the only writes are to the caller-chosen output directory, which should refuse to overwrite existing files the same way --evidence_output does. No process launch beyond the existing provider session, no target mutation. Non-goal: a new decompiler or a new provider.

Verification available so far

I have a prototype or observed results

Verification details (optional)

Observed results only (the run above); no prototype of the export itself. Checks I can suggest: a non-stripped fixture with DWARF where the number of written files equals the provider's function count minus recorded failures; a run interrupted after N functions resumes without rewriting the first N; a filtered run writes only matching functions and the manifest says how many were excluded; a file's header identity matches analyze_function on the same address.

Alternatives considered

batch_decompile returns every body inline, so it is bounded by response size and by #5930's record cap, and still routes all text through the agent.
With #5660, loop analyze_function (summary) over list_procedures and then export_evidence_bundle: one MCP round trip per function, and the result is one large JSON ledger rather than browsable per-function files; the agent still cannot read "the file for function X" without a view call.
rea decompile per address from a shell loop: works for a person, but the agent is not supposed to shell out, each call starts a provider session unless a snapshot is reused, and the output has no manifest, identity header, or failure record.
#5117 proposes a provider-neutral batch export of machine evidence (bytes, P-code, CFG) for reconstruction and conformance; it is a related batch-selection contract but a different output: a normalized, retained artifact rather than readable source files on disk, and it does not have the "nothing through the agent's context" goal. If the maintainers prefer, this request could be a pseudocode-to-directory mode of that export.
#3751 and #1048 (reusing an analysed Ghidra project) would make repeated or resumed bulk runs much cheaper and are complementary.

Possible direction

Reuse the existing pieces: list_procedures for the selection, the provider's per-function decompile with the per-request deadline from #5340 when it lands, the --evidence_output publish convention for refusing to overwrite, and the retained-Evidence model so the manifest's Evidence ID can be paged later. The CLI form would be rea decompile-all <target> --output <dir> [--filter ...] [--layout dwarf|address] [--resume]; the MCP form the same inputs with the manifest as normalized_result. Treat this as a proposal.

References

#5660 summary-by-default analyze_function (same run, per-call size)
#5032 one large function's pseudocode is too big to return
#5930 batch_decompile accepts unbounded address arrays
#5340 per-function Ghidra decompile deadline instead of closing the session
#5117 provider-neutral batch export for native machine evidence
#3751, #1048 reuse an analysed Ghidra project

Originally created by @CryptoJones on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6090 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Native analysis or decompilation ### Current limitation The task "decompile this whole program so I can step through it" has no route that keeps the decompiled text out of the agent's context. Every option today returns pseudocode through the tool result: `procedure_pseudo_code` and `analyze_function` one function at a time, or `batch_decompile` for a bounded address list (#5930 is tightening that bound). An agent driving a local model therefore spends its window on function bodies it only needed to have on disk. Observed (REA 6.7.0, Ghidra 12.1.4, a local 27B model with a 262K context behind an MCP harness that keeps roughly the first 50 KB of a tool result): on *Delores* (x86-64 ELF, not stripped, DWARF present, 10 MB), the prompt was to decompile the entire program. The model could only proceed function by function. About 45K tokens of the window were already the compact tool catalog, and a handful of `analyze_function` results exhausted the rest, so it never got past the first few functions. That run is what motivated #5660 (summary-by-default for `analyze_function`), which fixes the per-call size but not the per-function round trip: with #5660 the agent still needs one call per function and then pages each body. A second observation from the same run, cause not established: after one oversized result, the next `batch_decompile` returned `target_unavailable` with "No app is open", and the model had to `open_binary` again, which re-imports and re-analyzes. I have not reproduced this in isolation and do not know whether the harness, a cancellation path like the one described in #5340, or something else closed the session. NOT_ENOUGH_INFO on the mechanism; mentioned because a bulk export would be hit by it hardest, and it may deserve its own bug once reproduced. ### Use cases - "Give me the decompiled source tree" for a non-stripped binary with DWARF, so a person or agent can grep and read files instead of calling a tool per function. - Large programs (tens of thousands of functions) where the agent's job is to find the ten that matter; it should read an index, not stream everything. - Offline review on a second machine: copy the directory, no provider needed. ### Desired outcome and success criteria A tool on both adapters (MCP and CLI) that writes pseudocode for every function, or a filtered set, into a caller-chosen output directory, one file per function (or per compilation unit when DWARF gives one), and returns only a small manifest: counts (requested, written, failed, skipped), the output root, per-failure reasons, provider identity and analysis profile, and an Evidence ID. Never the code itself. Success criteria: - The tool result stays under a few KB regardless of program size. - Each written file carries the function identity (name, canonical address, body ranges when non-contiguous) and the provider's limitations for that function, so a file read later is still attributable Evidence. - Per-function failures (decompiler error, deadline, unresolved identity) are recorded in the manifest with reasons and do not abort the run. - Runs on big binaries are resumable or chunked: a second invocation with the same target digest and profile skips functions already written and reports what remains, or the caller can bound a run by count or address range and continue from the manifest. - Optional layout from DWARF source file and line information (directory tree mirroring the compile units) when available; flat by address otherwise. Unavailable debug info is reported, not guessed. - Optional filter by namespace, name pattern, or address range. - Index file in the output root (JSON) that the agent can read selectively. ### Expected interaction with targets Read existing files or captured data, Write or change files/configuration ### Target and scope details (optional) Static provider analysis only; the only writes are to the caller-chosen output directory, which should refuse to overwrite existing files the same way `--evidence_output` does. No process launch beyond the existing provider session, no target mutation. Non-goal: a new decompiler or a new provider. ### Verification available so far I have a prototype or observed results ### Verification details (optional) Observed results only (the run above); no prototype of the export itself. Checks I can suggest: a non-stripped fixture with DWARF where the number of written files equals the provider's function count minus recorded failures; a run interrupted after N functions resumes without rewriting the first N; a filtered run writes only matching functions and the manifest says how many were excluded; a file's header identity matches `analyze_function` on the same address. ### Alternatives considered - `batch_decompile` returns every body inline, so it is bounded by response size and by #5930's record cap, and still routes all text through the agent. - With #5660, loop `analyze_function` (summary) over `list_procedures` and then `export_evidence_bundle`: one MCP round trip per function, and the result is one large JSON ledger rather than browsable per-function files; the agent still cannot read "the file for function X" without a view call. - `rea decompile` per address from a shell loop: works for a person, but the agent is not supposed to shell out, each call starts a provider session unless a snapshot is reused, and the output has no manifest, identity header, or failure record. - #5117 proposes a provider-neutral batch export of machine evidence (bytes, P-code, CFG) for reconstruction and conformance; it is a related batch-selection contract but a different output: a normalized, retained artifact rather than readable source files on disk, and it does not have the "nothing through the agent's context" goal. If the maintainers prefer, this request could be a pseudocode-to-directory mode of that export. - #3751 and #1048 (reusing an analysed Ghidra project) would make repeated or resumed bulk runs much cheaper and are complementary. ### Possible direction Reuse the existing pieces: `list_procedures` for the selection, the provider's per-function decompile with the per-request deadline from #5340 when it lands, the `--evidence_output` publish convention for refusing to overwrite, and the retained-Evidence model so the manifest's Evidence ID can be paged later. The CLI form would be `rea decompile-all <target> --output <dir> [--filter ...] [--layout dwarf|address] [--resume]`; the MCP form the same inputs with the manifest as `normalized_result`. Treat this as a proposal. ### References - #5660 summary-by-default `analyze_function` (same run, per-call size) - #5032 one large function's pseudocode is too big to return - #5930 `batch_decompile` accepts unbounded address arrays - #5340 per-function Ghidra decompile deadline instead of closing the session - #5117 provider-neutral batch export for native machine evidence - #3751, #1048 reuse an analysed Ghidra project

bsakaguchi closed this issue

2026-10-31 00:43:24 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471