[GH-ISSUE #4249] [Feature] Opt-in summary delivery and retained pages for list_strings, list_procedures, list_names, and their search tools #1674

New issue

Closed

opened 2026-10-16 21:08:52 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-16 21:08:52 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4249

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

MCP or agent integration

Current limitation

The provider-wide native inventories list_strings, search_strings, list_procedures, search_procedures, and list_names always return their complete result inline. Their inputs are strict and accept no detail, offset, or limit (src/contracts/officialToolContracts.ts, src/contracts/analysisSearchContract.ts:4-12). Their retained Evidence is also not an eligible inspect_analysis_view parent: the view dispatcher accepts binary layout, JavaScript application, native function, artifact, WASM, Mach-O, Interface Builder, EVM, managed, Android package, Dart AOT, and web or pcap capture parents, and rejects everything else (src/domain/analysisView/analysisView.ts:746-753). When one of these results exceeds the 10 MiB response budget, the only recovery offered is to export Evidence or use CLI JSON (src/server/toolResult.ts:171-205). Clients cut results off long before that budget: in a public Claude Code study reported on #723 (rea 6.4.0, Ghidra 12.1.4), 18 of 72 runs had a result too large to return inline, from list_procedures, procedure_pseudo_code, and list_strings, and median input tokens were 300,695 with REA against 112,967 without it.

Use cases

An agent orienting in a large stripped binary needs the procedure count and the first page of procedures, not tens of thousands of rows in its context.
An agent looking for configuration or network strings needs a page of search_strings matches it can narrow, with the complete result still retained as Evidence.
procedure_pseudo_code and procedure_assembly for one large function are already pageable through analyze_function and the native view; this request covers the whole-binary inventories that have no such route.

Desired outcome and success criteria

The pattern already shipped for inspect_macho (#2473, PR #2634) extended to the native inventories, with complete inline delivery unchanged by default. Success criteria:

Each of the five tools accepts the existing opt-in detail: "summary" and then returns counts, provider and coverage facts, and the retained parent Evidence ID instead of every row.
inspect_analysis_view accepts that Evidence as a parent and returns stable pages of rows with offset and limit, in the same order as complete delivery.
Paging all pages reproduces the complete result exactly, with no rows lost or duplicated.
Coverage and limitations from the complete result (for example Ghidra string coverage disclosure) are present in the summary.
The overflow error for these tools names the working summary or view route instead of only export.
Before-and-after measurements on representative completed tasks report tokens, elapsed time, CPU time, and peak memory, as AGENTS.md requires.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Delivery-shape change over existing provider results; no new provider reads. Non-goals: default caps or truncation, reviving the removed offset/limit inputs on the tools themselves (#723 asks not to; earlier limits were removed in 3.2.0), and catalog or schema scoping (#3246, #2512).

Verification available so far

I can suggest a sample or manual check

Verification details (optional)

Use a large self-built or appropriately licensed binary (for example a statically linked release build of an open-source program) with Ghidra: compare complete list_procedures and list_strings output with the concatenation of all view pages; measure a fixed function-location task with and without summary delivery, using the method from the #723 study.

Alternatives considered

Hard caps with an "N more" footer on every read tool (for example 200 list items, 60 search hits, or 12,000 characters of pseudocode): simple and cheap for context, but it breaks REA's complete-by-default contract and #723's no-arbitrary-caps rule.
Exporting Evidence and reading the bundle: works but moves the cost to a file read and gives no page boundaries.
binary_overview: gives counts only, no rows.

Possible direction

Add the five operations to the retained view projectors with a rows page kind keyed by stable address order, and add analysisDetailSchema to their inputs. Optional exact or literal row matching in the view could cover narrowing without new tool inputs; that is a separate decision.

References

REA baseline: main at 0cc56493be (package 6.5.0): docs/mcp-contracts.md:95-145, src/contracts/analysisDetail.ts, src/domain/analysisView/analysisView.ts, src/server/toolResult.ts, AGENTS.md performance rules.
Related: #723 (discovery and result-repetition umbrella, including the study comment of 2026-10-15), #3134 (measuring existing paging), #2473 and PR #2634 (Mach-O summary and retained pages), #29 (earlier string and procedure paging, later removed), #3246 and #2512 (catalog scoping), PR #2701 and PR #3966 (retained Evidence summaries).
Search: issues and pull requests searched for pagination, paginate, offset limit, list_strings, list_procedures, response size, token budget, large output, and summary detail; no proposal for summary or view delivery of these inventories found.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4249 ### Before submitting - [X] I searched the open and closed issues and pull requests for a related proposal. - [X] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [X] This request does not include credentials, private samples, or sensitive target data. ### Area MCP or agent integration ### Current limitation The provider-wide native inventories `list_strings`, `search_strings`, `list_procedures`, `search_procedures`, and `list_names` always return their complete result inline. Their inputs are strict and accept no `detail`, offset, or limit (`src/contracts/officialToolContracts.ts`, `src/contracts/analysisSearchContract.ts:4-12`). Their retained Evidence is also not an eligible `inspect_analysis_view` parent: the view dispatcher accepts binary layout, JavaScript application, native function, artifact, WASM, Mach-O, Interface Builder, EVM, managed, Android package, Dart AOT, and web or pcap capture parents, and rejects everything else (`src/domain/analysisView/analysisView.ts:746-753`). When one of these results exceeds the 10 MiB response budget, the only recovery offered is to export Evidence or use CLI JSON (`src/server/toolResult.ts:171-205`). Clients cut results off long before that budget: in a public Claude Code study reported on #723 (rea 6.4.0, Ghidra 12.1.4), 18 of 72 runs had a result too large to return inline, from `list_procedures`, `procedure_pseudo_code`, and `list_strings`, and median input tokens were 300,695 with REA against 112,967 without it. ### Use cases - An agent orienting in a large stripped binary needs the procedure count and the first page of procedures, not tens of thousands of rows in its context. - An agent looking for configuration or network strings needs a page of `search_strings` matches it can narrow, with the complete result still retained as Evidence. - `procedure_pseudo_code` and `procedure_assembly` for one large function are already pageable through `analyze_function` and the `native` view; this request covers the whole-binary inventories that have no such route. ### Desired outcome and success criteria The pattern already shipped for `inspect_macho` (#2473, PR #2634) extended to the native inventories, with complete inline delivery unchanged by default. Success criteria: - Each of the five tools accepts the existing opt-in `detail: "summary"` and then returns counts, provider and coverage facts, and the retained parent Evidence ID instead of every row. - `inspect_analysis_view` accepts that Evidence as a parent and returns stable pages of rows with offset and limit, in the same order as complete delivery. - Paging all pages reproduces the complete result exactly, with no rows lost or duplicated. - Coverage and limitations from the complete result (for example Ghidra string coverage disclosure) are present in the summary. - The overflow error for these tools names the working summary or view route instead of only export. - Before-and-after measurements on representative completed tasks report tokens, elapsed time, CPU time, and peak memory, as `AGENTS.md` requires. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) Delivery-shape change over existing provider results; no new provider reads. Non-goals: default caps or truncation, reviving the removed `offset`/`limit` inputs on the tools themselves (#723 asks not to; earlier limits were removed in 3.2.0), and catalog or schema scoping (#3246, #2512). ### Verification available so far I can suggest a sample or manual check ### Verification details (optional) Use a large self-built or appropriately licensed binary (for example a statically linked release build of an open-source program) with Ghidra: compare complete `list_procedures` and `list_strings` output with the concatenation of all view pages; measure a fixed function-location task with and without summary delivery, using the method from the #723 study. ### Alternatives considered - Hard caps with an "N more" footer on every read tool (for example 200 list items, 60 search hits, or 12,000 characters of pseudocode): simple and cheap for context, but it breaks REA's complete-by-default contract and #723's no-arbitrary-caps rule. - Exporting Evidence and reading the bundle: works but moves the cost to a file read and gives no page boundaries. - `binary_overview`: gives counts only, no rows. ### Possible direction Add the five operations to the retained view projectors with a `rows` page kind keyed by stable address order, and add `analysisDetailSchema` to their inputs. Optional exact or literal row matching in the view could cover narrowing without new tool inputs; that is a separate decision. ### References - REA baseline: `main` at 0cc56493be167309bd72040273de603ad75ad9e0 (package 6.5.0): `docs/mcp-contracts.md:95-145`, `src/contracts/analysisDetail.ts`, `src/domain/analysisView/analysisView.ts`, `src/server/toolResult.ts`, `AGENTS.md` performance rules. - Related: #723 (discovery and result-repetition umbrella, including the study comment of 2026-10-15), #3134 (measuring existing paging), #2473 and PR #2634 (Mach-O summary and retained pages), #29 (earlier string and procedure paging, later removed), #3246 and #2512 (catalog scoping), PR #2701 and PR #3966 (retained Evidence summaries). - Search: issues and pull requests searched for pagination, paginate, offset limit, `list_strings`, `list_procedures`, response size, token budget, large output, and summary detail; no proposal for summary or view delivery of these inventories found.

bsakaguchi commented

2026-10-16 21:08:53 +00:00

Author

Owner

@pablodiazjorge commented on GitHub (Oct 16, 2026):

Some numbers for this request from the study cited above, rea-lab study 3: rea 6.4.0 with Ghidra 12.1.4, Claude Code 2.1.296 and 72 runs with rea on four stripped x64 programs.

One correction first. The 18 runs are the ones where the agent read a saved rea result back. Claude Code wrote at least one rea result to a file instead of passing it inline in 32 of the 72 runs, 43 results in all:

Tool	Calls	Written to a file	Size of those results
`list_procedures`	21	21, every call	51 to 224 KB, median 52 KB
`procedure_pseudo_code`	306	18	58 to 471 KB, median 194 KB
`search_procedures`	16	2	52 and 108 KB
`list_strings`	1	1	60 KB
`procedure_references`	1	1	95 KB
`search_strings`	101	0

Two observations that bear on the design:

No run called analyze_function or inspect_analysis_view. The pageable route for one large function was available but never used; the agents went to procedure_pseudo_code every time. My inference is that a summary mode on the inventories helps only if agents find it, so a pointer to it in the oversized-result path or in the tool descriptions may matter as much as the mode itself.
The smallest result written to a file was 51 KB, far below the 10 MiB response budget, so with this client the practical limit is the client's, as the issue says.

The counts come from the saved files and the run transcripts in my local run folders. They are not part of the published dataset, because the transcripts contain local paths.

@pablodiazjorge commented on GitHub (Oct 16, 2026): Some numbers for this request from the study cited above, [rea-lab study 3](https://github.com/pablodiazjorge/rea-lab/tree/main/writeups/2026-10-study-3): rea 6.4.0 with Ghidra 12.1.4, Claude Code 2.1.296 and 72 runs with rea on four stripped x64 programs. One correction first. The 18 runs are the ones where the agent read a saved rea result back. Claude Code wrote at least one rea result to a file instead of passing it inline in 32 of the 72 runs, 43 results in all: | Tool | Calls | Written to a file | Size of those results | |---|---|---|---| | `list_procedures` | 21 | 21, every call | 51 to 224 KB, median 52 KB | | `procedure_pseudo_code` | 306 | 18 | 58 to 471 KB, median 194 KB | | `search_procedures` | 16 | 2 | 52 and 108 KB | | `list_strings` | 1 | 1 | 60 KB | | `procedure_references` | 1 | 1 | 95 KB | | `search_strings` | 101 | 0 | | Two observations that bear on the design: - No run called `analyze_function` or `inspect_analysis_view`. The pageable route for one large function was available but never used; the agents went to `procedure_pseudo_code` every time. My inference is that a summary mode on the inventories helps only if agents find it, so a pointer to it in the oversized-result path or in the tool descriptions may matter as much as the mode itself. - The smallest result written to a file was 51 KB, far below the 10 MiB response budget, so with this client the practical limit is the client's, as the issue says. The counts come from the saved files and the run transcripts in my local run folders. They are not part of the published dataset, because the transcripts contain local paths.

bsakaguchi closed this issue

2026-10-31 00:32:09 +00:00

bsakaguchi referenced this issue

2026-10-31 00:54:35 +00:00

[PR #1674] [CLOSED] docs(website): add a step-by-step setup guide for first-time users #4744

bsakaguchi referenced this issue

2026-10-31 00:54:39 +00:00

[PR #1676] [CLOSED] docs(website): make the analysis guides easier to start #4748