morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6426] [Feature] Smaller MCP responses for native analysis: repeated Evidence envelopes and unpaged lists #3842

New issue

Closed

opened 2026-10-31 00:44:33 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:44:33 +00:00

Owner

Originally created by @Kendy619 on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6426

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

MCP or agent integration

Current limitation

Before submitting: check all three boxes after searching for related issues and PRs.

For native analysis through MCP (measured with the Ghidra provider on PS2
Emotion Engine executables), most response bytes repeat context that agents
already have, and some complete results exceed the 10 MiB stdio budget.

Measured response bytes (MCP content text; tokens not measured):

Call	Useful payload	Response
`procedure_callees` returning 1 address	~12 B	~4.9–5.7 KB
`resolve_containing_procedure` (one procedure)	~60 B	~5.6 KB
`get_call_graph` backward, 518 nodes	~43 KB	253 KB
`procedure_callers` with an unknown name (error)	one message	38 KB
`inspect_native_load_image`, ELF with 56,477 relocations	—	> 10 MiB, refused (CLI JSON 17 MB)
`list_strings` on a 28k-function executable	—	> 10 MiB, refused

Every primitive response embeds the complete analysis profile and every
session limitation (~5 KB). One 66-call agent session received 800 KB.
In get_call_graph, three distinct limitation texts repeat on every node:
83% of that 518-node response.
The unknown-name error embeds the provider's whole launch stdout
(details.diagnostics.stdout).
inspect_native_load_image lists every relocation with no summary or
paging. In the CLI JSON the list appears twice (raw_result and
normalized_result).

Use cases

Agents exploring a large native target: hundreds of small callers/callees/
xref queries per task, where envelope bytes dominate the context cost.
ELF images with many relocations (here a Metrowerks CodeWarrior build) or
many strings, where the complete answer cannot be delivered at all over MCP.

Desired outcome and success criteria

The smallest sufficient answer with provenance, as AGENTS.md "Performance"
asks. For example:

after the first response, reference the session's analysis profile and
limitations by digest, with the full record kept retrievable;
deduplicate limitation texts in graph results;
keep launch diagnostics out of input-validation errors, or bound them;
offer summary/paging for relocation and string inventories instead of
refusing.

Success would mean equivalent workflows return the same facts and provenance
in measurably fewer bytes, and the two refused tasks complete.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Only the Ghidra provider was measured. Whether other providers behave the
same is not established. Whether MCP clients rely on the inline profile in
every record is not established either.

Verification available so far

I have a prototype or observed results

Verification details (optional)

Observed with REA 6.7.0 (fork with the PS2 profile from the related draft PR),
Ghidra 12.1.3, Linux x64 (WSL2), MCP client 2.3.1. The numbers come from
recorded MCP sessions on caller-owned executables. No samples are attached.

Alternatives considered

Raising REA_MCP_MAX_RESPONSE_BYTES also requires a matching client
receive buffer. With the default client it closed the connection.
search_strings works around list_strings for targeted queries, but not
for complete inventories.
Agents can avoid get_call_graph, but then pay the per-call envelope on
many smaller queries.

Possible direction

A per-session Evidence context referenced by digest, plus summary/paging views
for the relocation and string inventories. This is similar to the existing
detail: summary on search_procedures.

References

Related draft PR: PS2 Emotion Engine profile: https://github.com/morluto/rea/pull/6422

Originally created by @Kendy619 on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6426 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area MCP or agent integration ### Current limitation Before submitting: check all three boxes after searching for related issues and PRs. For native analysis through MCP (measured with the Ghidra provider on PS2 Emotion Engine executables), most response bytes repeat context that agents already have, and some complete results exceed the 10 MiB stdio budget. Measured response bytes (MCP `content` text; tokens not measured): | Call | Useful payload | Response | | --- | --- | --- | | `procedure_callees` returning 1 address | ~12 B | ~4.9–5.7 KB | | `resolve_containing_procedure` (one procedure) | ~60 B | ~5.6 KB | | `get_call_graph` backward, 518 nodes | ~43 KB | 253 KB | | `procedure_callers` with an unknown name (error) | one message | 38 KB | | `inspect_native_load_image`, ELF with 56,477 relocations | — | > 10 MiB, refused (CLI JSON 17 MB) | | `list_strings` on a 28k-function executable | — | > 10 MiB, refused | - Every primitive response embeds the complete analysis profile and every session limitation (~5 KB). One 66-call agent session received 800 KB. - In `get_call_graph`, three distinct limitation texts repeat on every node: 83% of that 518-node response. - The unknown-name error embeds the provider's whole launch stdout (`details.diagnostics.stdout`). - `inspect_native_load_image` lists every relocation with no summary or paging. In the CLI JSON the list appears twice (`raw_result` and `normalized_result`). ### Use cases - Agents exploring a large native target: hundreds of small callers/callees/ xref queries per task, where envelope bytes dominate the context cost. - ELF images with many relocations (here a Metrowerks CodeWarrior build) or many strings, where the complete answer cannot be delivered at all over MCP. ### Desired outcome and success criteria The smallest sufficient answer with provenance, as AGENTS.md "Performance" asks. For example: - after the first response, reference the session's analysis profile and limitations by digest, with the full record kept retrievable; - deduplicate limitation texts in graph results; - keep launch diagnostics out of input-validation errors, or bound them; - offer summary/paging for relocation and string inventories instead of refusing. Success would mean equivalent workflows return the same facts and provenance in measurably fewer bytes, and the two refused tasks complete. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) Only the Ghidra provider was measured. Whether other providers behave the same is not established. Whether MCP clients rely on the inline profile in every record is not established either. ### Verification available so far I have a prototype or observed results ### Verification details (optional) Observed with REA 6.7.0 (fork with the PS2 profile from the related draft PR), Ghidra 12.1.3, Linux x64 (WSL2), MCP client 2.3.1. The numbers come from recorded MCP sessions on caller-owned executables. No samples are attached. ### Alternatives considered - Raising `REA_MCP_MAX_RESPONSE_BYTES` also requires a matching client receive buffer. With the default client it closed the connection. - `search_strings` works around `list_strings` for targeted queries, but not for complete inventories. - Agents can avoid `get_call_graph`, but then pay the per-call envelope on many smaller queries. ### Possible direction A per-session Evidence context referenced by digest, plus summary/paging views for the relocation and string inventories. This is similar to the existing `detail: summary` on `search_procedures`. ### References Related draft PR: PS2 Emotion Engine profile: https://github.com/morluto/rea/pull/6422

bsakaguchi

2026-10-31 00:44:33 +00:00

closed this issue
added the
triage

enhancement
labels

bsakaguchi commented

2026-10-31 00:44:34 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 25, 2026):

Implemented and merged in #7134, #7140 and #7146. Native load-image inventories now have retained summaries and selected/paged collections; expected Ghidra input errors use request-scoped diagnostics; primitive/graph summaries refer to retrievable source context, with graph limitation text deduplicated per page. Complete Evidence, source indices, coverage and unknowns remain accessible through the shared CLI/MCP owners.

Real Ghidra 12.1.4 checks on Linux: a libcrypto load image with 21,118 relocation observations completes through summary/paging where the complete inline response was refused. The existing retained string-inventory path also completed on stripped LLVM 14 with 177,830 strings and a 39,856,151-byte complete exported bundle; selected first/last pages match the retained source. A paired cold/warm /usr/bin/ls investigation preserved all 65 source Evidence records, observations, profiles and limitations. The whole serialized MCP text estimate decreased 0.68% after counting catalog discovery and additional calls; the graph first reply decreased from 63,888 to 5,150 cl100k tokens. Actual model usage was not measured, and no speed or memory improvement is claimed.

Reproducible drivers, elapsed/CPU/process-family memory measurements and limitations are recorded in the linked PRs and experiments/analysis-efficiency/native-observation-views.md. The original caller-owned PS2 executable was unavailable; that target/profile was not reproduced.

@gensokyo-dev commented on GitHub (Oct 25, 2026): Implemented and merged in #7134, #7140 and #7146. Native load-image inventories now have retained summaries and selected/paged collections; expected Ghidra input errors use request-scoped diagnostics; primitive/graph summaries refer to retrievable source context, with graph limitation text deduplicated per page. Complete Evidence, source indices, coverage and unknowns remain accessible through the shared CLI/MCP owners. Real Ghidra 12.1.4 checks on Linux: a libcrypto load image with 21,118 relocation observations completes through summary/paging where the complete inline response was refused. The existing retained string-inventory path also completed on stripped LLVM 14 with 177,830 strings and a 39,856,151-byte complete exported bundle; selected first/last pages match the retained source. A paired cold/warm /usr/bin/ls investigation preserved all 65 source Evidence records, observations, profiles and limitations. The whole serialized MCP text estimate decreased 0.68% after counting catalog discovery and additional calls; the graph first reply decreased from 63,888 to 5,150 cl100k tokens. Actual model usage was not measured, and no speed or memory improvement is claimed. Reproducible drivers, elapsed/CPU/process-family memory measurements and limitations are recorded in the linked PRs and experiments/analysis-efficiency/native-observation-views.md. The original caller-owned PS2 executable was unavailable; that target/profile was not reproduced.

bsakaguchi referenced this issue

2026-10-31 01:10:37 +00:00

[PR #3842] [CLOSED] fix(mcp): give the compact any-value schema an item schema #5942

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3842

No description provided.

Rows
Columns