[GH-ISSUE #723] [Design] Reduce MCP discovery and result repetition while preserving complete contracts

bsakaguchi commented

2026-10-15 18:23:52 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 6, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/723

Before submitting

I searched the open and closed issues and relevant pull requests for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Decision needed

What compatible result/schema representation makes complete REA discovery and tiny-target findings easier for agents to consume without dropping Evidence, weakening validation, or hiding tools?

Context and evidence

Real SDK discovery returned all 117 tools. Compact UTF-8 measurements:

Surface	Bytes
Complete tool definitions, including output schemas	1,949,695
Names, descriptions, and input schemas	699,835
Three-file JS CLI output	303,361
Three-file JS MCP output	759,454

The JS fixture contained 422 text bytes and 65 visited AST nodes. The MCP result exactly repeated evidence.normalized_result, and text repeated the structured envelope. Initial binary_session took 7.90 s and returned approximately 305 KB.

The input-facing projection measured 167,799 cl100k_base proxy tokens. This is not the actual tokenizer, tool filtering, or context cost of every model host. Essential parameters in expanded Evidence-heavy schemas required manual projection to locate. No automatic model-host selection benchmark was performed.

Boundary and invariants

Preserve the complete stable canonical catalog, exact input/output types, Evidence normalized_result, useful inline findings, source identity, confidence, limitations, and unknowns. Retain text/structured semantic equivalence and validate schemas after SDK conversion.

Do not revive removed filtering APIs, assume cross-tool JSON references work in every client, remove output schemas, introduce arbitrary caps, or force resource reads for the next useful result. Separately validate CLI/MCP semantics and host compatibility.

Options and tradeoffs

Improve required-field presentation, semantic descriptions, and schema-local reuse where supported; validate actual client consumers rather than only serialized bytes.
Provide concise question-relevant summaries alongside complete Evidence and reduce repeated representations only where client compatibility allows.
Keep the current full envelopes and add clearer consumption examples. This is compatible but leaves measured discovery/repetition costs.

Retained-reference inputs are a separate contract decision. They may reduce repeated schema/input material, but do not by themselves address result readability.

Success criteria and open questions

Report discovery and result sizes with representative public CLI/MCP tasks, not only a fixed byte budget.
Demonstrate that an agent/client can identify required inputs and read module/function/change facts without manually traversing expanded envelopes.
Preserve complete Evidence, successful output validation, catalog identity, and accurate advertised availability.
Test representative host/client treatment of structured output, text fallback, and schema references before changing their representation.
Keep useful findings inline and make any incomplete coverage explicit.

Open question: which repeated forms are required for current client compatibility, and which can be removed or represented more efficiently without losing information?

References

Hands-on public CLI/MCP evaluation, findings F02. Baseline: REA 4.0.1, source revision c3bc38bf6c74d2175a4680530dcadd6cab0e7791; evaluation date 2026-10-06.

Follow-up measurements to #264. Its historical text/structured mismatch is fixed in this baseline; the new concern is consumption/repetition under current complete contracts. #551 requires complete inline Evidence. #654 and #664 establish current stable-catalog guidance; do not reintroduce obsolete filtering claims. docs/mcp-contracts.md

Related evaluation issues: #722.

Originally created by @N0zoM1z0 on GitHub (Oct 6, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/723 ### Before submitting - [x] I searched the open and closed issues and relevant pull requests for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Decision needed What compatible result/schema representation makes complete REA discovery and tiny-target findings easier for agents to consume without dropping Evidence, weakening validation, or hiding tools? ### Context and evidence Real SDK discovery returned all 117 tools. Compact UTF-8 measurements: | Surface | Bytes | | --- | ---: | | Complete tool definitions, including output schemas | 1,949,695 | | Names, descriptions, and input schemas | 699,835 | | Three-file JS CLI output | 303,361 | | Three-file JS MCP output | 759,454 | The JS fixture contained 422 text bytes and 65 visited AST nodes. The MCP `result` exactly repeated `evidence.normalized_result`, and text repeated the structured envelope. Initial binary_session took 7.90 s and returned approximately 305 KB. The input-facing projection measured 167,799 cl100k_base proxy tokens. This is not the actual tokenizer, tool filtering, or context cost of every model host. Essential parameters in expanded Evidence-heavy schemas required manual projection to locate. No automatic model-host selection benchmark was performed. ### Boundary and invariants Preserve the complete stable canonical catalog, exact input/output types, Evidence normalized_result, useful inline findings, source identity, confidence, limitations, and unknowns. Retain text/structured semantic equivalence and validate schemas after SDK conversion. Do not revive removed filtering APIs, assume cross-tool JSON references work in every client, remove output schemas, introduce arbitrary caps, or force resource reads for the next useful result. Separately validate CLI/MCP semantics and host compatibility. ### Options and tradeoffs 1. Improve required-field presentation, semantic descriptions, and schema-local reuse where supported; validate actual client consumers rather than only serialized bytes. 2. Provide concise question-relevant summaries alongside complete Evidence and reduce repeated representations only where client compatibility allows. 3. Keep the current full envelopes and add clearer consumption examples. This is compatible but leaves measured discovery/repetition costs. Retained-reference inputs are a separate contract decision. They may reduce repeated schema/input material, but do not by themselves address result readability. ### Success criteria and open questions - Report discovery and result sizes with representative public CLI/MCP tasks, not only a fixed byte budget. - Demonstrate that an agent/client can identify required inputs and read module/function/change facts without manually traversing expanded envelopes. - Preserve complete Evidence, successful output validation, catalog identity, and accurate advertised availability. - Test representative host/client treatment of structured output, text fallback, and schema references before changing their representation. - Keep useful findings inline and make any incomplete coverage explicit. Open question: which repeated forms are required for current client compatibility, and which can be removed or represented more efficiently without losing information? ### References Hands-on public CLI/MCP evaluation, findings F02. Baseline: REA 4.0.1, source revision [c3bc38bf6c74d2175a4680530dcadd6cab0e7791](https://github.com/morluto/rea/commit/c3bc38bf6c74d2175a4680530dcadd6cab0e7791); evaluation date 2026-10-06. Follow-up measurements to [#264](https://github.com/morluto/rea/issues/264). Its historical text/structured mismatch is fixed in this baseline; the new concern is consumption/repetition under current complete contracts. [#551](https://github.com/morluto/rea/issues/551) requires complete inline Evidence. [#654](https://github.com/morluto/rea/issues/654) and [#664](https://github.com/morluto/rea/pull/664) establish current stable-catalog guidance; do not reintroduce obsolete filtering claims. [docs/mcp-contracts.md](https://github.com/morluto/rea/blob/c3bc38bf6c74d2175a4680530dcadd6cab0e7791/docs/mcp-contracts.md) Related evaluation issues: [#722](https://github.com/morluto/rea/issues/722).

bsakaguchi added the

P1: high

area: mcp

enhancement

labels

2026-10-15 18:23:52 +00:00

bsakaguchi commented

2026-10-15 18:23:53 +00:00

Author

Owner

@hippoley commented on GitHub (Oct 8, 2026):

I’m testing #723 from the downstream-consumer side rather than from another MCP host.

One thing that seems worth preserving even if the result envelope becomes more compact is a small stable evidence projection that an external verifier can consume without depending on REA internals:

stable evidence identity
artifact identity/hash
provider identity + observed version
operation + normalized parameters
normalized result
confidence
authority
execution environment
explicit limitations / residual unknowns
typed source locations when present

The reason is that a downstream compatibility/proof tool should be able to say “this claim is supported by these exact REA observations” without copying the full envelope or silently upgrading an inferred/truncated observation into a positive verdict.

I wrote the smallest consumer-profile experiment here:
https://github.com/hippoley/CounterProof/pull/172
(specifically experiments/behavior_witness/interop/REA_EVIDENCE_V2.md)

The fail-closed rule I’m testing is: if the comparison depends on an unknown/truncated/redacted dimension, incompatible comparison contract, missing provenance, or an inferred observation phrased as direct observation, the downstream result stays UNKNOWN rather than PASS.

This is not a proposal to change REA’s Evidence v2 schema. I’m trying to answer one narrower question relevant to #723: is there a compact, explicitly supported consumer projection that can remain stable even if REA changes how much of the full Evidence envelope it renders inline for MCP clients?

If you already consider a subset of Evidence v2 to be the intended external-consumer boundary, pointing me at that contract would let me delete assumptions rather than grow another adapter.

@hippoley commented on GitHub (Oct 8, 2026): I’m testing #723 from the downstream-consumer side rather than from another MCP host. One thing that seems worth preserving even if the result envelope becomes more compact is a **small stable evidence projection** that an external verifier can consume without depending on REA internals: - stable evidence identity - artifact identity/hash - provider identity + observed version - operation + normalized parameters - normalized result - confidence - authority - execution environment - explicit limitations / residual unknowns - typed source locations when present The reason is that a downstream compatibility/proof tool should be able to say “this claim is supported by these exact REA observations” without copying the full envelope or silently upgrading an inferred/truncated observation into a positive verdict. I wrote the smallest consumer-profile experiment here: https://github.com/hippoley/CounterProof/pull/172 (specifically `experiments/behavior_witness/interop/REA_EVIDENCE_V2.md`) The fail-closed rule I’m testing is: if the comparison depends on an unknown/truncated/redacted dimension, incompatible comparison contract, missing provenance, or an inferred observation phrased as direct observation, the downstream result stays `UNKNOWN` rather than `PASS`. This is not a proposal to change REA’s Evidence v2 schema. I’m trying to answer one narrower question relevant to #723: **is there a compact, explicitly supported consumer projection that can remain stable even if REA changes how much of the full Evidence envelope it renders inline for MCP clients?** If you already consider a subset of Evidence v2 to be the intended external-consumer boundary, pointing me at that contract would let me delete assumptions rather than grow another adapter.

bsakaguchi commented

2026-10-15 18:23:53 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 9, 2026):

The latest DX-Ball audit adds a concrete native follow-up in #1247. Npm latest 6.1.0 and main f4c99ab8 now reject an oversized complete native dossier with a typed transport resource constraint, retained Evidence reference and export recovery; the default SDK connection remains usable. Native selected-facet inspection is still missing because the current shared view workflow covers JavaScript applications and ELF layouts.

#1247 focuses on useful native views and finding eligible retained dossiers, preserving complete Evidence rather than silently trimming it. #1249 separately investigates snapshot materialization/storage cost. Historical repeated catalog archives were the project's helper copies, so they were not filed as REA automatically duplicating session catalogs. This complements #1065 without treating serialized bytes alone as agent context cost.

@N0zoM1z0 commented on GitHub (Oct 9, 2026): The latest DX-Ball audit adds a concrete native follow-up in #1247. Npm latest 6.1.0 and main `f4c99ab8` now reject an oversized complete native dossier with a typed transport resource constraint, retained Evidence reference and export recovery; the default SDK connection remains usable. Native selected-facet inspection is still missing because the current shared view workflow covers JavaScript applications and ELF layouts. #1247 focuses on useful native views and finding eligible retained dossiers, preserving complete Evidence rather than silently trimming it. #1249 separately investigates snapshot materialization/storage cost. Historical repeated catalog archives were the project's helper copies, so they were not filed as REA automatically duplicating session catalogs. This complements #1065 without treating serialized bytes alone as agent context cost.

bsakaguchi commented

2026-10-15 18:23:53 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

#1433 is merged after 19/19 final-head checks. Stable content/dialect-bound output-schema IDs let the actual default SDK 2.3.1 Ajv provider reuse validators across unchanged list refreshes and reconnects; real stdio measurements show zero new compilations afterward. Changed schemas, availability and reconnect validation are covered. Complete discovery and default text/structured results remain intact.

This addresses repeated validator compilation, not catalog size or actual model context: the full 139-tool catalog is 1,884,470 compact bytes, and the first ordinary call still compiles 118 distinct outputs in about 2.43 seconds on two CPUs. That distinct eager cold cost is tracked at https://github.com/modelcontextprotocol/typescript-sdk/issues/2997. This broader design issue remains open.

@N0zoM1z0 commented on GitHub (Oct 10, 2026): #1433 is merged after 19/19 final-head checks. Stable content/dialect-bound output-schema IDs let the actual default SDK 2.3.1 Ajv provider reuse validators across unchanged list refreshes and reconnects; real stdio measurements show zero new compilations afterward. Changed schemas, availability and reconnect validation are covered. Complete discovery and default text/structured results remain intact. This addresses repeated validator compilation, not catalog size or actual model context: the full 139-tool catalog is 1,884,470 compact bytes, and the first ordinary call still compiles 118 distinct outputs in about 2.43 seconds on two CPUs. That distinct eager cold cost is tracked at https://github.com/modelcontextprotocol/typescript-sdk/issues/2997. This broader design issue remains open.

bsakaguchi commented

2026-10-15 18:23:53 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

A fresh actual small-target CLI/SDK stdio MCP measurement from #1537 supplies another result-size baseline: a 2,421-byte JavaScript fixture with nine exports returns 785,206 CLI bytes and 1,644,176 bytes for the MCP reply. Full normalized results, Evidence IDs, text/structured parity, complete return-shape coverage, unchanged input and a subsequent ping all pass. Main and the indexed returned-reference implementation produce the same results.

Within the compact normalized result, the application graph occupies 64,768 bytes and the semantic graph 717,325 bytes (444 nodes, 359 relations, 88 fingerprints and 82 explicit unknowns). This fixture intentionally includes a 64-method receiver as well as duplicate/computed/spread/capture cases; these measurements are not a minimal single-export baseline or model billing tokens. The shared Evidence context table is already present (seven contexts), so it would be inaccurate to describe all repeated provenance as still unnormalized.

#1537 addresses redundant property scans and removes an exactly duplicated test group; it does not reduce these result bytes. I am keeping this compatible result/readability concern associated with #723 rather than treating a CPU optimization as a token fix.

@N0zoM1z0 commented on GitHub (Oct 10, 2026): A fresh actual small-target CLI/SDK stdio MCP measurement from #1537 supplies another result-size baseline: a 2,421-byte JavaScript fixture with nine exports returns 785,206 CLI bytes and 1,644,176 bytes for the MCP reply. Full normalized results, Evidence IDs, text/structured parity, complete return-shape coverage, unchanged input and a subsequent ping all pass. Main and the indexed returned-reference implementation produce the same results. Within the compact normalized result, the application graph occupies 64,768 bytes and the semantic graph 717,325 bytes (444 nodes, 359 relations, 88 fingerprints and 82 explicit unknowns). This fixture intentionally includes a 64-method receiver as well as duplicate/computed/spread/capture cases; these measurements are not a minimal single-export baseline or model billing tokens. The shared Evidence context table is already present (seven contexts), so it would be inaccurate to describe all repeated provenance as still unnormalized. #1537 addresses redundant property scans and removes an exactly duplicated test group; it does not reduce these result bytes. I am keeping this compatible result/readability concern associated with #723 rather than treating a CPU optimization as a token fix.

bsakaguchi commented

2026-10-15 18:23:53 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 11, 2026):

Current measurement on morluto/rea@21650455a9c90a26e725c3505a3c17054cc4516c: complete discovery is 170 tools / 1,954,693 JSON bytes. That payload is unchanged by the SDK cache fix.

The related cold-call cost is now verified against the real catalog: the existing upstream change reduces validator preparation from 170 to 1 and actual first-call compilation from 148 to 1, while full discovery and output validation stay intact. Five ordered experimental pairs measured first-call medians 6.286 s → 0.724 s. Flameox evidence, exact source revisions and verification scope.

This is an isolated test of the exact upstream owning module, not a deployed dependency change. Upstream issue #2997 being closed does not mean this has shipped: PR #2998 remains unmerged and the official client release remains 2.3.1. Keep catalog/result representation work in this issue separate from that pending client fix; no tool omission, schema bypass or payload reduction was used in the measurement.

@N0zoM1z0 commented on GitHub (Oct 11, 2026): Current measurement on `morluto/rea@21650455a9c90a26e725c3505a3c17054cc4516c`: complete discovery is **170 tools / 1,954,693 JSON bytes**. That payload is unchanged by the SDK cache fix. The related cold-call cost is now verified against the real catalog: the existing upstream change reduces validator preparation from **170 to 1** and actual first-call compilation from **148 to 1**, while full discovery and output validation stay intact. Five ordered experimental pairs measured first-call medians **6.286 s → 0.724 s**. [Flameox evidence, exact source revisions and verification scope](https://github.com/modelcontextprotocol/typescript-sdk/pull/2998#issuecomment-6106691706). This is an isolated test of the exact upstream owning module, not a deployed dependency change. Upstream [issue #2997](https://github.com/modelcontextprotocol/typescript-sdk/issues/2997) being closed does not mean this has shipped: [PR #2998](https://github.com/modelcontextprotocol/typescript-sdk/pull/2998) remains unmerged and the official client release remains 2.3.1. Keep catalog/result representation work in this issue separate from that pending client fix; no tool omission, schema bypass or payload reduction was used in the measurement.

bsakaguchi commented

2026-10-15 18:23:53 +00:00

Author

Owner

@wu546526 commented on GitHub (Oct 13, 2026):

Adding a fresh data point to this design discussion from a real compatibility failure — full write-up in #2512.

rea-agents@6.4.0 tools/list arrives as a single JSON-RPC line of 1,954,727 bytes (170 tools): outputSchema 1,493,900 (76%), inputSchema 365,646, description 58,941, annotations 15,243, names+titles 7,352.

A real host that caps one stdio line at 1 MB (pi-web-ui 0.104.4's built-in MCP bridge) kills the server at every handshake and loads 0 tools, while the pi CLI on the same machine (SDK 10 MB receive buffer) loads all 170 in 2.96 s. Two existing levers don't get under the cap: REA_MCP_INPUT_SCHEMA_PROFILE=compact → 1,772,799 bytes (trims inputSchema only), and #1143's shipped $ref sharing (~1.9 MB — per-tool output contracts are largely distinct).

So today there is a hard compatibility floor: ≤1 MB-frame clients cannot load REA at all. Proposal and success criteria in #2512 (opt-in output-schema-light advertisement, or tools/list pagination).

@wu546526 commented on GitHub (Oct 13, 2026): Adding a fresh data point to this design discussion from a real compatibility failure — full write-up in #2512. rea-agents@6.4.0 `tools/list` arrives as a single JSON-RPC line of **1,954,727 bytes** (170 tools): `outputSchema` 1,493,900 (76%), `inputSchema` 365,646, `description` 58,941, `annotations` 15,243, names+titles 7,352. A real host that caps one stdio line at 1 MB ([pi-web-ui](https://github.com/xing-shuyin/pi-web-ui) 0.104.4's built-in MCP bridge) kills the server at every handshake and loads 0 tools, while the pi CLI on the same machine (SDK 10 MB receive buffer) loads all 170 in 2.96 s. Two existing levers don't get under the cap: `REA_MCP_INPUT_SCHEMA_PROFILE=compact` → 1,772,799 bytes (trims inputSchema only), and #1143's shipped `$ref` sharing (~1.9 MB — per-tool output contracts are largely distinct). So today there is a hard compatibility floor: ≤1 MB-frame clients cannot load REA at all. Proposal and success criteria in #2512 (opt-in output-schema-light advertisement, or `tools/list` pagination).

bsakaguchi commented

2026-10-16 21:02:23 +00:00

Author

Owner

@pablodiazjorge commented on GitHub (Oct 15, 2026):

A real-host data point on context cost, from three preregistered Claude Code studies run on Windows with rea 6.4.0 and Ghidra 12.1.4 (rea-lab). In the latest one (study 3), 72 runs located functions in four stripped C programs with rea and 72 without it. Runs with rea used a median of 300,695 input tokens against 112,967 without it, and 2.3 times the median equivalent cost, for similar accuracy (68 and 71 of 72). In 32 of the 72 runs with rea, at least one result was too large for Claude Code to return inline and was saved to a file instead: 43 results, from list_procedures, procedure_pseudo_code, search_procedures, list_strings and procedure_references. The agent read a saved result back in 18 of those runs. The study's addendum has the breakdown by tool. Per-run token usage and tool calls are in bench/results/study-3.

Edit (16 October 2026): the first version said results were saved to a file in 18 runs. That is the number of runs that read one back; results were saved in 32.

@pablodiazjorge commented on GitHub (Oct 15, 2026): A real-host data point on context cost, from three preregistered Claude Code studies run on Windows with rea 6.4.0 and Ghidra 12.1.4 ([rea-lab](https://github.com/pablodiazjorge/rea-lab)). In the latest one ([study 3](https://github.com/pablodiazjorge/rea-lab/tree/main/writeups/2026-10-study-3)), 72 runs located functions in four stripped C programs with rea and 72 without it. Runs with rea used a median of 300,695 input tokens against 112,967 without it, and 2.3 times the median equivalent cost, for similar accuracy (68 and 71 of 72). In 32 of the 72 runs with rea, at least one result was too large for Claude Code to return inline and was saved to a file instead: 43 results, from `list_procedures`, `procedure_pseudo_code`, `search_procedures`, `list_strings` and `procedure_references`. The agent read a saved result back in 18 of those runs. The study's [addendum](https://github.com/pablodiazjorge/rea-lab/tree/main/writeups/2026-10-study-3#addendum-oversized-results-and-catalog-cost-post-hoc) has the breakdown by tool. Per-run token usage and tool calls are in [bench/results/study-3](https://github.com/pablodiazjorge/rea-lab/tree/main/bench/results/study-3). Edit (16 October 2026): the first version said results were saved to a file in 18 runs. That is the number of runs that read one back; results were saved in 32.

bsakaguchi referenced this issue

2026-10-16 21:05:48 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-10-16 21:05:49 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-10-17 21:01:17 +00:00

bsakaguchi commented

2026-10-31 00:29:01 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 18, 2026):

Checked on Linux: part of this is fixed on main by 6395e8d30 (#1988). The commit's message says "Refs #723", which is why GitHub left this issue open.

Its 11 test files pass on main 36d156ff0 on Node 22.23.1, 24.18.0 and 26.11.1 (117 tests, 2 skipped).
On the commit's parent, 34 of 81 fail, with ZodError and wrong capabilities. Some fail only for missing helpers.

The tests cover grouped session and discovery facts, not repeated results or size. So part of this may still be open.

Thanks, N0zoM1z0, for the clear report.

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

@CassieFox23 commented on GitHub (Oct 18, 2026): Checked on Linux: part of this is fixed on main by 6395e8d30 (#1988). The commit's message says "Refs #723", which is why GitHub left this issue open. - Its 11 test files pass on main 36d156ff0 on Node 22.23.1, 24.18.0 and 26.11.1 (117 tests, 2 skipped). - On the commit's parent, 34 of 81 fail, with `ZodError` and wrong `capabilities`. Some fail only for missing helpers. The tests cover grouped session and discovery facts, not repeated results or size. So part of this may still be open. Thanks, N0zoM1z0, for the clear report. 🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

bsakaguchi commented

2026-10-31 00:29:02 +00:00

Author

Owner

@cathrynlavery commented on GitHub (Oct 19, 2026):

Size measurements for binary_session, since the existing tests for #1988 cover grouping but not size.

Setup: macOS 27.0 arm64, Node 24.18.0, stdio MCP via @modelcontextprotocol/client, no Ghidra/Hopper/IDA installed. Sequence: open_binary {"path": "/System/Applications/Calculator.app"}, then binary_session {}. Sizes are characters of the result's text content; structuredContent serializes to the same length.

Field	6.4.0 (published)	main `b4938aed5`
Whole result	227,787	56,752
`tool_availability`	100,397	27,327
`analysis_provider_candidates`	69,425	26,993
`server_identity`	64,117	1,116
`capabilities`	8,802	3,346

With no target open, main returns 54,278 characters.

In a Claude Code session on 6.4.0, the result was too large to return inline and Claude Code saved it to a file instead. I have not checked whether Claude Code accepts the 56,752-character result from main inline. On main, tool_availability and analysis_provider_candidates together are 96% of the result, so those two are where any further reduction would come from.

@cathrynlavery commented on GitHub (Oct 19, 2026): Size measurements for `binary_session`, since the existing tests for #1988 cover grouping but not size. Setup: macOS 27.0 arm64, Node 24.18.0, stdio MCP via `@modelcontextprotocol/client`, no Ghidra/Hopper/IDA installed. Sequence: `open_binary {"path": "/System/Applications/Calculator.app"}`, then `binary_session {}`. Sizes are characters of the result's text content; `structuredContent` serializes to the same length. | Field | 6.4.0 (published) | main b4938aed5 | | --- | ---: | ---: | | Whole result | 227,787 | 56,752 | | `tool_availability` | 100,397 | 27,327 | | `analysis_provider_candidates` | 69,425 | 26,993 | | `server_identity` | 64,117 | 1,116 | | `capabilities` | 8,802 | 3,346 | With no target open, main returns 54,278 characters. In a Claude Code session on 6.4.0, the result was too large to return inline and Claude Code saved it to a file instead. I have not checked whether Claude Code accepts the 56,752-character result from main inline. On main, `tool_availability` and `analysis_provider_candidates` together are 96% of the result, so those two are where any further reduction would come from.

bsakaguchi commented

2026-10-31 00:29:02 +00:00

Author

Owner

@shockstricken commented on GitHub (Oct 28, 2026):

Another data point for result repetition, on rea-agents 6.8.0 (Ghidra provider, Linux x86_64, MCP over HTTP).

analyze_function returns the same limitations list twice: once at the Evidence top level and again inside normalized_result (the function dossier). The list is about 25 entries of fixed provider text, so it is most of the reply for a small function. Projecting to fields: ["evidence_id"] returns only the ID, so projection is a working workaround.

Observed on a raw-binary target; the steps below are the minimal equivalent and were not run verbatim.

Reproduction:

printf 'int main(void){return 0;}\n' > t.c && cc -O0 -o t t.c
open_binary {"path": "<abs path>/t"}
analyze_function on main with no fields.
Compare limitations with normalized_result.limitations. Both hold the same entries.

Source reading on main: src/domain/functionDossierEvidence.ts builds the top-level list as new Set([...evidence.limitations, ...dossier.value.limitations]), while the dossier in normalized_result keeps its own copy. #6426's fix deduplicated graph limitation text per page. Function dossiers appear to still carry both copies.

@shockstricken commented on GitHub (Oct 28, 2026): Another data point for result repetition, on rea-agents 6.8.0 (Ghidra provider, Linux x86_64, MCP over HTTP). `analyze_function` returns the same `limitations` list twice: once at the Evidence top level and again inside `normalized_result` (the function dossier). The list is about 25 entries of fixed provider text, so it is most of the reply for a small function. Projecting to `fields: ["evidence_id"]` returns only the ID, so projection is a working workaround. Observed on a raw-binary target; the steps below are the minimal equivalent and were not run verbatim. Reproduction: 1. `printf 'int main(void){return 0;}\n' > t.c && cc -O0 -o t t.c` 2. `open_binary` `{"path": "<abs path>/t"}` 3. `analyze_function` on `main` with no `fields`. 4. Compare `limitations` with `normalized_result.limitations`. Both hold the same entries. Source reading on `main`: `src/domain/functionDossierEvidence.ts` builds the top-level list as `new Set([...evidence.limitations, ...dossier.value.limitations])`, while the dossier in `normalized_result` keeps its own copy. #6426's fix deduplicated graph limitation text per page. Function dossiers appear to still carry both copies.

bsakaguchi commented

2026-10-31 00:29:02 +00:00

Author

Owner

@jgt87 commented on GitHub (Oct 28, 2026):

Confirmed on rea-agents 6.8.0 (released) and on main 6a226bd09.

One module item view (88-file fixture, refused projection)	6.8.0 (npm, Node 26.12.1)	main `6a226bd09` (source, Node 24.22.1)
Whole result (bytes)	72,459	72,459
`limitations` and `normalized_result.limitations` (identical lists)	28 + 28 strings, 4,859 + 4,859 bytes	28 + 28 strings, 4,859 + 4,859 bytes

Measurement

rea-agents 6.8.0 (npm, Node 26.12.1) and upstream main at 6a226bd09 (source, Node 24.22.1), Windows 11 Pro
(10.0.26300). Synthetic Vite-style
build (generator gen-vite-like.mjs, attached to #7735, 88 files):

node gen-vite-like.mjs ./fx/vite87 87 456
rea analyze-javascript-application ./fx/vite87/assets --artifact-format directory \
  --max-heap-mb 4096 --detail summary --evidence_output ./ev-87.json --format json > summary.json
# view.json: {"source":{"kind":"evidence-file","path":"<abs>/ev-87.json","evidence_id":"<parent_evidence_id>"},
#             "view":{"kind":"item","collection":"modules","selector":{"node_id":"<one module node>"}}}
rea inspect-analysis-view view.json --format json

One module item view (a file whose application projection was refused):

Part	Bytes
Whole result	72,459
`normalized_result.item` (mostly `static_analysis.storage`, 272 entries)	56,143
`limitations` (28 strings)	4,859
`normalized_result.limitations` (the same 28 strings)	4,859
`normalized_result.unknowns` (31 entries)	4,856

Five limitations arrays in the result hold 58 strings, 30 of them distinct.
The 28 parent limitations appear twice in every item view, and again in each
further item the caller requests. For a fully projected module the view is
12,286 bytes, of which 4,975 bytes (40 %) are limitations.

On 6.4.0 through MCP, the same kind of view was about 110 KB per item, mostly
146 repeated limitation strings. Main is smaller, so the 6.4.0 numbers are
history, not a current reproduction.

Possible direction

Return the parent's limitations once per parent (by digest or reference) and
only item-specific limitations in each item view, as #723 proposes for other
results.

@jgt87 commented on GitHub (Oct 28, 2026): Confirmed on rea-agents 6.8.0 (released) and on main 6a226bd09. | One module item view (88-file fixture, refused projection) | 6.8.0 (npm, Node 26.12.1) | main 6a226bd09 (source, Node 24.22.1) | | --- | ---: | ---: | | Whole result (bytes) | 72,459 | 72,459 | | `limitations` and `normalized_result.limitations` (identical lists) | 28 + 28 strings, 4,859 + 4,859 bytes | 28 + 28 strings, 4,859 + 4,859 bytes | ## Measurement rea-agents 6.8.0 (npm, Node 26.12.1) and upstream `main` at 6a226bd09 (source, Node 24.22.1), Windows 11 Pro (10.0.26300). Synthetic Vite-style build (generator `gen-vite-like.mjs`, attached to #7735, 88 files): ```sh node gen-vite-like.mjs ./fx/vite87 87 456 rea analyze-javascript-application ./fx/vite87/assets --artifact-format directory \ --max-heap-mb 4096 --detail summary --evidence_output ./ev-87.json --format json > summary.json # view.json: {"source":{"kind":"evidence-file","path":"<abs>/ev-87.json","evidence_id":"<parent_evidence_id>"}, # "view":{"kind":"item","collection":"modules","selector":{"node_id":"<one module node>"}}} rea inspect-analysis-view view.json --format json ``` One module item view (a file whose application projection was refused): | Part | Bytes | | --- | ---: | | Whole result | 72,459 | | `normalized_result.item` (mostly `static_analysis.storage`, 272 entries) | 56,143 | | `limitations` (28 strings) | 4,859 | | `normalized_result.limitations` (the same 28 strings) | 4,859 | | `normalized_result.unknowns` (31 entries) | 4,856 | Five `limitations` arrays in the result hold 58 strings, 30 of them distinct. The 28 parent limitations appear twice in every item view, and again in each further item the caller requests. For a fully projected module the view is 12,286 bytes, of which 4,975 bytes (40 %) are limitations. On 6.4.0 through MCP, the same kind of view was about 110 KB per item, mostly 146 repeated limitation strings. Main is smaller, so the 6.4.0 numbers are history, not a current reproduction. ## Possible direction Return the parent's limitations once per parent (by digest or reference) and only item-specific limitations in each item view, as #723 proposes for other results.

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-01 21:10:52 +00:00

bsakaguchi commented

2026-11-01 21:10:53 +00:00

Author

Owner

@morluto commented on GitHub (Nov 1, 2026):

A related real-provider workflow exposes repetition inside a single reconstruction claim, beyond the transport duplication described above. @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Observed: On REA PR #8443 at 8dd4cf4a9, I analyzed _main in two controlled, unexecuted Mach-O fixtures with Ghidra 12.1.4, retained/exported the dossiers, then used the compact-profile public stdio MCP compare_functions → verify_reconstruction workflow. The call comparison was unchanged for the observed sets; the verification correctly returned unknown because static discovery was unreported. The single normalized claim serialized to 6,514 UTF-8 bytes, of which its 35-element limitations array occupied 5,855 bytes. All 15 Left: provider limitations were repeated verbatim under Right:; further source-specific discovery limitations were also present. These are JSON byte measurements, not token counts or an end-to-end agent benchmark. The fixture hashes were unchanged. Replaying the same real dossiers in a target-free server at the parent commit produced a 6,046-byte Ghidra claim with 31 limitations; the new discovery qualifications account for four additional entries. Both full and compact profiles produced the same claim sizes. The analogous Kuna 1.986 run produced an unknown observed call comparison and an unknown verification; its claim was 2,781 bytes with 15 limitations.

Source-derived explanation: Function comparison prefixes every source limitation with its side, and verification includes the whole comparison limitation list alongside dimension and source-specific limitations. This propagation already exists in the parent of #8443; the new discovery facts add useful qualifications, so removing them would be incorrect.

Design question: Can the result group identical side limitations and surface the limitations relevant to the selected claim while preserving side/source attribution, independently readable Evidence, complete access to the original caveats, and honest unknowns? Measure a complete representative task and validate full/compact schemas and client rendering before changing the contract. Whether the extra information changes agent decisions has not been measured.

@morluto commented on GitHub (Nov 1, 2026): A related real-provider workflow exposes repetition inside a single reconstruction claim, beyond the transport duplication described above. @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 **Observed:** On REA PR #8443 at `8dd4cf4a9`, I analyzed `_main` in two controlled, unexecuted Mach-O fixtures with Ghidra 12.1.4, retained/exported the dossiers, then used the compact-profile public stdio MCP `compare_functions` → `verify_reconstruction` workflow. The call comparison was `unchanged` for the observed sets; the verification correctly returned `unknown` because static discovery was unreported. The single normalized claim serialized to 6,514 UTF-8 bytes, of which its 35-element `limitations` array occupied 5,855 bytes. All 15 `Left:` provider limitations were repeated verbatim under `Right:`; further source-specific discovery limitations were also present. These are JSON byte measurements, not token counts or an end-to-end agent benchmark. The fixture hashes were unchanged. Replaying the same real dossiers in a target-free server at the parent commit produced a 6,046-byte Ghidra claim with 31 limitations; the new discovery qualifications account for four additional entries. Both full and compact profiles produced the same claim sizes. The analogous Kuna 1.986 run produced an unknown observed call comparison and an unknown verification; its claim was 2,781 bytes with 15 limitations. **Source-derived explanation:** Function comparison prefixes every source limitation with its side, and verification includes the whole comparison limitation list alongside dimension and source-specific limitations. This propagation already exists in the parent of #8443; the new discovery facts add useful qualifications, so removing them would be incorrect. **Design question:** Can the result group identical side limitations and surface the limitations relevant to the selected claim while preserving side/source attribution, independently readable Evidence, complete access to the original caveats, and honest unknowns? Measure a complete representative task and validate full/compact schemas and client rendering before changing the contract. Whether the extra information changes agent decisions has not been measured.

Rows
Columns

[GH-ISSUE #723] [Design] Reduce MCP discovery and result repetition while preserving complete contracts #324

Before submitting

Decision needed

Context and evidence

Boundary and invariants

Options and tradeoffs

Success criteria and open questions

References

Measurement

Possible direction