morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #1560] [MERGED] fix(mcp): stop advertising complete Evidence in input examples #2972

New issue

Closed

opened 2026-10-17 21:12:24 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:12:24 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1560
Author: @kaoru0822-kitauji
Created: 10/10/2026
Status: ✅ Merged
Merged: 10/10/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: fix/mcp-advertised-example-evidence

📝 Commits (1)

1d9a95f fix(mcp): stop advertising complete Evidence in input examples

📊 Changes

5 files changed (+171 additions, -3 deletions)

View changed files

📝 docs/mcp-contracts.md (+10 -1)
➕ src/contracts/advertisedInputExamples.test.ts (+80 -0)
➕ src/contracts/advertisedInputExamples.ts (+75 -0)
📝 src/contracts/toolSchemaMetadata.ts (+3 -1)
📝 tests/boundary/mcp/toolSchemaValidity.test.ts (+3 -1)

📄 Description

Problem

Users report that REA uses too much context. Hosts send every tool's name, description and inputSchema to the model with each request, so the advertised input catalog is a fixed cost on every turn.

Measured with the pinned SDK 2.3.1 client against the stdio server built from main (2ec93ba1). All 139 tools; compact UTF-8 bytes:

Surface	Before	After
Input-facing catalog (name + description + inputSchema)	507,455	393,263 (−22.5%)
All advertised `inputSchema`s	450,463	336,271 (−25.3%)
Root `examples` in input schemas	129,457	~13,000

Root examples made up 29% of the advertised input schemas. 17 tools had an example that embedded a complete producer Evidence record: compare_javascript_export_shapes (22 KB), compare_application_versions (21 KB), reconcile_javascript_runtime, correlate_static_and_runtime, compare_source_to_bundle, trace_application_feature, trace_javascript_semantics, compare_process_captures, find_changed_behavior and others. Together these were 116 KB. An agent never writes such an example by hand. It passes Evidence it already received, and the example shows nothing about choosing arguments that the schema and descriptions don't already state. It can also lead an agent to paste whole Evidence inline when a retained reference would work.

Change

The advertised projection (toolInputSchemaWithMetadata) now builds examples with a new helper, advertisedInputExamples (src/contracts/advertisedInputExamples.ts):

An embedded complete Evidence record is replaced by its exact {"kind":"retained-evidence","evidence_id":...} reference, but only when the canonical input schema accepts that form. This applies to the five JavaScript application workflows, which take evidenceInputSchema.
An example that can only carry inline Evidence is left out of the advertisement. Twelve tools end up with no advertised examples, and the examples keyword is omitted for them.
Duplicates are removed, so every advertised example is a distinct valid canonical input.

Unchanged: canonical contract.examples (still executed by boundary tests), canonical validation, the advertised validation keywords, property descriptions, output schemas, the tool inventory and the compact profile.

docs/mcp-contracts.md documents the rule.

Not in scope: per-result size and the duplication between text and structured content (#723), and the remaining validation structure of large input schemas such as compare_web_captures (52 KB).

Relates to #723, #1065, #1554. This reduces the static catalog cost #1554 hit, but on its own it may not clear that client's limit: the issue reports that omitting root examples alone still failed with gpt-4.1 metadata.

Verification

npm run check:changed and npm run check:fast pass
npm run docs:check passes
npm run test:focused -- tests/boundary/mcp/*.test.ts: 85 files passed, 2 skipped (330 tests)
New src/contracts/advertisedInputExamples.test.ts covers:
- rewriting Evidence to retained references, including inside arrays, and removing duplicates;
- omitting an example when only inline Evidence is accepted;
- a check across the whole catalog that no advertised example embeds an Evidence record and every advertised example parses canonically.
toolSchemaValidity now asserts the advertised examples equal the projection, and still checks every canonical example against the advertised schema with Ajv.

No real Hopper/Ghidra/IDA verification was performed (they are not affected). No real host-client lanes (Copilot, Kimi) were run. Byte counts are wire measurements, not model token counts.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1560 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/10/2026 **Status:** ✅ Merged **Merged:** 10/10/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `fix/mcp-advertised-example-evidence` --- ### 📝 Commits (1) - [`1d9a95f`](https://github.com/morluto/rea/commit/1d9a95f440c3c0082936093a8ffd0dce3de82705) fix(mcp): stop advertising complete Evidence in input examples ### 📊 Changes **5 files changed** (+171 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `docs/mcp-contracts.md` (+10 -1) ➕ `src/contracts/advertisedInputExamples.test.ts` (+80 -0) ➕ `src/contracts/advertisedInputExamples.ts` (+75 -0) 📝 `src/contracts/toolSchemaMetadata.ts` (+3 -1) 📝 `tests/boundary/mcp/toolSchemaValidity.test.ts` (+3 -1) </details> ### 📄 Description ## Problem Users report that REA uses too much context. Hosts send every tool's name, description and `inputSchema` to the model with each request, so the advertised input catalog is a fixed cost on every turn. Measured with the pinned SDK 2.3.1 client against the stdio server built from `main` (2ec93ba1). All 139 tools; compact UTF-8 bytes: | Surface | Before | After | | --- | ---: | ---: | | Input-facing catalog (name + description + inputSchema) | 507,455 | 393,263 (−22.5%) | | All advertised `inputSchema`s | 450,463 | 336,271 (−25.3%) | | Root `examples` in input schemas | 129,457 | ~13,000 | Root examples made up 29% of the advertised input schemas. 17 tools had an example that embedded a **complete producer Evidence record**: `compare_javascript_export_shapes` (22 KB), `compare_application_versions` (21 KB), `reconcile_javascript_runtime`, `correlate_static_and_runtime`, `compare_source_to_bundle`, `trace_application_feature`, `trace_javascript_semantics`, `compare_process_captures`, `find_changed_behavior` and others. Together these were 116 KB. An agent never writes such an example by hand. It passes Evidence it already received, and the example shows nothing about choosing arguments that the schema and descriptions don't already state. It can also lead an agent to paste whole Evidence inline when a retained reference would work. ## Change The advertised projection (`toolInputSchemaWithMetadata`) now builds `examples` with a new helper, `advertisedInputExamples` (`src/contracts/advertisedInputExamples.ts`): - An embedded complete Evidence record is replaced by its exact `{"kind":"retained-evidence","evidence_id":...}` reference, but only when the canonical input schema accepts that form. This applies to the five JavaScript application workflows, which take `evidenceInputSchema`. - An example that can only carry inline Evidence is left out of the advertisement. Twelve tools end up with no advertised examples, and the `examples` keyword is omitted for them. - Duplicates are removed, so every advertised example is a distinct valid canonical input. Unchanged: canonical `contract.examples` (still executed by boundary tests), canonical validation, the advertised validation keywords, property descriptions, output schemas, the tool inventory and the compact profile. `docs/mcp-contracts.md` documents the rule. Not in scope: per-result size and the duplication between text and structured content (#723), and the remaining validation structure of large input schemas such as `compare_web_captures` (52 KB). Relates to #723, #1065, #1554. This reduces the static catalog cost #1554 hit, but on its own it may not clear that client's limit: the issue reports that omitting root examples alone still failed with `gpt-4.1` metadata. ## Verification - `npm run check:changed` and `npm run check:fast` pass - `npm run docs:check` passes - `npm run test:focused -- tests/boundary/mcp/*.test.ts`: 85 files passed, 2 skipped (330 tests) - New `src/contracts/advertisedInputExamples.test.ts` covers: - rewriting Evidence to retained references, including inside arrays, and removing duplicates; - omitting an example when only inline Evidence is accepted; - a check across the whole catalog that no advertised example embeds an Evidence record and every advertised example parses canonically. - `toolSchemaValidity` now asserts the advertised examples equal the projection, and still checks every canonical example against the advertised schema with Ajv. No real Hopper/Ghidra/IDA verification was performed (they are not affected). No real host-client lanes (Copilot, Kimi) were run. Byte counts are wire measurements, not model token counts. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:12:24 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 01:07:20 +00:00

[PR #2972] [CLOSED] fix(ci): fail required gate after current-head cancellation #5529

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2972

No description provided.

Rows
Columns