[PR #1162] [MERGED] perf(mcp): reuse input schemas without losing caller guidance #2695

New issue

Closed

opened 2026-10-17 21:09:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:09:11 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1162
Author: @N0zoM1z0
Created: 10/9/2026
Status: ✅ Merged
Merged: 10/9/2026
Merged by: @N0zoM1z0

Base: main ← Head: perf/mcp-input-schema-references

📝 Commits (1)

71f70cc perf(mcp): reuse input schemas without losing caller guidance

📊 Changes

6 files changed (+599 additions, -158 deletions)

View changed files

📝 docs/mcp-contracts.md (+5 -3)
➕ src/contracts/inputSchemaPresentation.ts (+163 -0)
➕ src/contracts/toolInputReferences.test.ts (+247 -0)
📝 src/contracts/toolSchemaMetadata.ts (+22 -89)
📝 tests/boundary/mcp/toolSchemaValidity.test.ts (+22 -3)
📝 tests/fixtures/localJsonSchemaReferences.ts (+140 -63)

📄 Description

Summary

Reduce repeated input-schema definitions in MCP discovery while retaining complete caller guidance. Partial T3 follow-up for #1065 and #723; both roadmaps remain open.

Problem and expected behavior

The frozen baseline advertises 552,049 UTF-8 bytes of input-facing tool metadata (all metadata retained, only outputSchema removed). Naively enabling references breaks root-union object presentation and introduces redundant deeper anyOf branches. Generic annotation traversal also mistakes literal default/example objects for schema nodes. Complete tools and parameter meaning must remain available.

Change and scope

Share repeated definitions through finite schema-local references. Keep root unions inline with the existing object presentation. Traverse schema keywords only, retain literal annotation data, and copy specialized referenced property descriptions to the use site. Preserve prototype-named own properties. Memoization and explicit inline projection remain available.

Actual pinned SDK comparison of all 138 tools against frozen source 089bbf58:

Surface	Before	After
Complete tools/list bytes	1,758,587	1,722,483
Input-facing metadata bytes	552,049	515,945
Input-schema bytes	478,286	442,182
cl100k_base input-facing proxy tokens	133,884	129,296

Input-facing bytes improve 6.54%; proxy tokens improve 3.43%. These proxy measurements do not establish automatic tool selection or every model host's context use.

Contract and boundary impact

Semantic owner and earliest changed stage: contracts, advertised JSON Schema projection.
CLI and MCP/tool-catalog contract: complete inventory and exact canonical input/output validation retained; input wire representation uses local references. Output schemas and all remaining tool metadata are byte equivalent to the control after actual SDK conversion.
Provider, bridge, target-format, or platform compatibility: provider operations unchanged; complete Notion and Obsidian workflows passed on native Windows.
Evidence, artifact, provenance, or reconstruction contract: unchanged complete result and Evidence contracts.
Process execution, authorization, cleanup, or containment impact: none; no new execution authority or approval fields.
Generated metadata/package impact: build regenerates ignored catalog and skills; no generated outputs are committed.

Sanitized actual catalog example from compare_application_versions:

{
  "left": {
    "$ref": "#/$defs/__schema20",
    "description": "Left-hand input used for comparison or differential execution."
  },
  "right": {
    "$ref": "#/$defs/__schema20",
    "description": "Right-hand input used for comparison or differential execution."
  }
}

The shared definition remains in that same tool's input schema.

Evidence and regression coverage

Executed: all input meanings and guidance match after finite-reference expansion; output schemas and remaining metadata match the actual SDK baseline. Both supported JSON Schema dialects preserve examples, strict objects, native-selector patterns and numeric bounds.
Executed: 40 focused tests across eight files, including actual SDK object roots/depth profile, referenced descriptions, incomplete round-trip rejection and reversed presentation.
Test-only normalization handles known primitive string intersections and local annotation precedence; object/tuple evaluation scopes and unexpected conflicts remain protected.
Remaining proof gaps: isolated packaged CLI/MCP acceptance passed. Other model-host converters and live selection tasks are not exercised. Native Windows complete Notion and Obsidian analysis, full-export byte parity, retained tracing and cleanup have passed. No live Muse/opencode or other model-host selection benchmark is claimed. Parser cancellation, client eager output compilation and result-view/transport changes remain separate.
Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npm run test:focused -- src/contracts/toolInputReferences.test.ts src/contracts/toolOutputReferences.test.ts src/contracts/toolSchemaMetadata.test.ts tests/boundary/mcp/toolSchemaValidity.test.ts tests/boundary/mcp/outputSchemaSemanticConstraints.test.ts tests/boundary/mcp/wireInputSchemaLimits.test.ts tests/boundary/providers/auxiliaryDeclarations.test.ts tests/acceptance/contractPresentation.test.ts — 40 passed.
npm run test:local -- src/contracts/toolInputReferences.test.ts src/contracts/toolOutputReferences.test.ts src/contracts/toolSchemaMetadata.test.ts — passed after final test-helper refactor.
npm run check — six static tasks passed; final diff has no new lint warnings.
npm run docs:check — passed.
npm pack --json — passed; isolated Windows offline installation passed.
npm run verify:package — passed (227.4 seconds; guarded family peak 1,523,920,896 bytes).
Native Windows complete Notion/Obsidian control/candidate workflows — passed; same-directory diagnostic controls preserved complete retained identities and queries.
All 22 CI checks passed at exact head 71f70cc942ec8a7f67c5624d00524afbb640a5ff.

Real-app correctness and performance investigation

Native Windows full exports match the same-source control byte-for-byte: Notion 336,236,424 bytes and Obsidian 429,232,879 bytes, with identical Evidence identities and retained trace digests. Malformed requests reject; default-buffer large-result recovery preserves the connection; ping, trace, close and owned-output cleanup pass. All exported files were deleted after hashing.

Two initial separate-directory Obsidian pairs showed slower candidate elapsed times and higher memory. We kept this PR draft and investigated rather than claiming a throughput gain. A same-product-directory comparison changes only the compiled metadata module: analysis 155.886 -> 156.400 seconds. A reverse-order control with identical GC telemetry gives 159.862 seconds for the original presentation versus 156.400 for the candidate, with family peaks 1,783,914,496 versus 1,581,506,560 bytes. All analysis modules are byte-identical. This narrower experiment does not reproduce a consistent regression; no analysis-throughput or memory-reduction claim is made from these variable samples.

Tests ran sequentially with two CPUs, BelowNormal priority, a 1.5 GiB server heap, 3 GiB family guard, 2 GiB host-memory reserve and disk checks. One initial pre-tool startup request timed out at the unchanged 60-second SDK deadline; its exact stage was not captured. A stage-instrumented retry at the same deadline passed, as did the subsequent controls. This PR does not claim to fix that unresolved startup timeout.

Compatibility, safety, and release

Breaking changes or migration steps: none for canonical calls; input discovery now uses finite local references. Both declared dialects, the pinned SDK and the existing input object/depth profile pass. Input-reference support in other model-host converters remains unverified; output-reference support alone does not establish it.
Real Hopper/Ghidra/browser coverage: not performed; no provider-operation changes. Real native Windows Notion and Obsidian analysis/export/query workflows passed.
Package or release metadata impact: generated catalog identity changes with its wire representation.
Security/privacy/process review: no target execution, new subprocesses or host configuration changes. Public examples contain no private host paths or credentials.

Review checklist

Concrete agreed enhancement and related issues linked.
One focused outcome and conventional title.
Meaningful success and failure regression coverage.
Owning docs and generated metadata workflow updated.
Representative wire output included.
Final diff checked for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1162 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/9/2026 **Status:** ✅ Merged **Merged:** 10/9/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `perf/mcp-input-schema-references` --- ### 📝 Commits (1) - [`71f70cc`](https://github.com/morluto/rea/commit/71f70cc942ec8a7f67c5624d00524afbb640a5ff) perf(mcp): reuse input schemas without losing caller guidance ### 📊 Changes **6 files changed** (+599 additions, -158 deletions) <details> <summary>View changed files</summary> 📝 `docs/mcp-contracts.md` (+5 -3) ➕ `src/contracts/inputSchemaPresentation.ts` (+163 -0) ➕ `src/contracts/toolInputReferences.test.ts` (+247 -0) 📝 `src/contracts/toolSchemaMetadata.ts` (+22 -89) 📝 `tests/boundary/mcp/toolSchemaValidity.test.ts` (+22 -3) 📝 `tests/fixtures/localJsonSchemaReferences.ts` (+140 -63) </details> ### 📄 Description ## Summary Reduce repeated input-schema definitions in MCP discovery while retaining complete caller guidance. Partial T3 follow-up for #1065 and #723; both roadmaps remain open. ## Problem and expected behavior The frozen baseline advertises 552,049 UTF-8 bytes of input-facing tool metadata (all metadata retained, only outputSchema removed). Naively enabling references breaks root-union object presentation and introduces redundant deeper anyOf branches. Generic annotation traversal also mistakes literal default/example objects for schema nodes. Complete tools and parameter meaning must remain available. ## Change and scope Share repeated definitions through finite schema-local references. Keep root unions inline with the existing object presentation. Traverse schema keywords only, retain literal annotation data, and copy specialized referenced property descriptions to the use site. Preserve prototype-named own properties. Memoization and explicit inline projection remain available. Actual pinned SDK comparison of all 138 tools against frozen source `089bbf58`: | Surface | Before | After | | --- | ---: | ---: | | Complete tools/list bytes | 1,758,587 | 1,722,483 | | Input-facing metadata bytes | 552,049 | 515,945 | | Input-schema bytes | 478,286 | 442,182 | | cl100k_base input-facing proxy tokens | 133,884 | 129,296 | Input-facing bytes improve 6.54%; proxy tokens improve 3.43%. These proxy measurements do not establish automatic tool selection or every model host's context use. ## Contract and boundary impact - Semantic owner and earliest changed stage: contracts, advertised JSON Schema projection. - CLI and MCP/tool-catalog contract: complete inventory and exact canonical input/output validation retained; input wire representation uses local references. Output schemas and all remaining tool metadata are byte equivalent to the control after actual SDK conversion. - Provider, bridge, target-format, or platform compatibility: provider operations unchanged; complete Notion and Obsidian workflows passed on native Windows. - Evidence, artifact, provenance, or reconstruction contract: unchanged complete result and Evidence contracts. - Process execution, authorization, cleanup, or containment impact: none; no new execution authority or approval fields. - Generated metadata/package impact: build regenerates ignored catalog and skills; no generated outputs are committed. Sanitized actual catalog example from compare_application_versions: ```json { "left": { "$ref": "#/$defs/__schema20", "description": "Left-hand input used for comparison or differential execution." }, "right": { "$ref": "#/$defs/__schema20", "description": "Right-hand input used for comparison or differential execution." } } ``` The shared definition remains in that same tool's input schema. ## Evidence and regression coverage - Executed: all input meanings and guidance match after finite-reference expansion; output schemas and remaining metadata match the actual SDK baseline. Both supported JSON Schema dialects preserve examples, strict objects, native-selector patterns and numeric bounds. - Executed: 40 focused tests across eight files, including actual SDK object roots/depth profile, referenced descriptions, incomplete round-trip rejection and reversed presentation. - Test-only normalization handles known primitive string intersections and local annotation precedence; object/tuple evaluation scopes and unexpected conflicts remain protected. - Remaining proof gaps: isolated packaged CLI/MCP acceptance passed. Other model-host converters and live selection tasks are not exercised. Native Windows complete Notion and Obsidian analysis, full-export byte parity, retained tracing and cleanup have passed. No live Muse/opencode or other model-host selection benchmark is claimed. Parser cancellation, client eager output compilation and result-view/transport changes remain separate. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:focused -- src/contracts/toolInputReferences.test.ts src/contracts/toolOutputReferences.test.ts src/contracts/toolSchemaMetadata.test.ts tests/boundary/mcp/toolSchemaValidity.test.ts tests/boundary/mcp/outputSchemaSemanticConstraints.test.ts tests/boundary/mcp/wireInputSchemaLimits.test.ts tests/boundary/providers/auxiliaryDeclarations.test.ts tests/acceptance/contractPresentation.test.ts` — 40 passed. - `npm run test:local -- src/contracts/toolInputReferences.test.ts src/contracts/toolOutputReferences.test.ts src/contracts/toolSchemaMetadata.test.ts` — passed after final test-helper refactor. - `npm run check` — six static tasks passed; final diff has no new lint warnings. - `npm run docs:check` — passed. - `npm pack --json` — passed; isolated Windows offline installation passed. - `npm run verify:package` — passed (227.4 seconds; guarded family peak 1,523,920,896 bytes). - Native Windows complete Notion/Obsidian control/candidate workflows — passed; same-directory diagnostic controls preserved complete retained identities and queries. - All 22 CI checks passed at exact head `71f70cc942ec8a7f67c5624d00524afbb640a5ff`. ## Real-app correctness and performance investigation Native Windows full exports match the same-source control byte-for-byte: Notion 336,236,424 bytes and Obsidian 429,232,879 bytes, with identical Evidence identities and retained trace digests. Malformed requests reject; default-buffer large-result recovery preserves the connection; ping, trace, close and owned-output cleanup pass. All exported files were deleted after hashing. Two initial separate-directory Obsidian pairs showed slower candidate elapsed times and higher memory. We kept this PR draft and investigated rather than claiming a throughput gain. A same-product-directory comparison changes only the compiled metadata module: analysis 155.886 -> 156.400 seconds. A reverse-order control with identical GC telemetry gives 159.862 seconds for the original presentation versus 156.400 for the candidate, with family peaks 1,783,914,496 versus 1,581,506,560 bytes. All analysis modules are byte-identical. This narrower experiment does not reproduce a consistent regression; no analysis-throughput or memory-reduction claim is made from these variable samples. Tests ran sequentially with two CPUs, BelowNormal priority, a 1.5 GiB server heap, 3 GiB family guard, 2 GiB host-memory reserve and disk checks. One initial pre-tool startup request timed out at the unchanged 60-second SDK deadline; its exact stage was not captured. A stage-instrumented retry at the same deadline passed, as did the subsequent controls. This PR does not claim to fix that unresolved startup timeout. ## Compatibility, safety, and release - Breaking changes or migration steps: none for canonical calls; input discovery now uses finite local references. Both declared dialects, the pinned SDK and the existing input object/depth profile pass. Input-reference support in other model-host converters remains unverified; output-reference support alone does not establish it. - Real Hopper/Ghidra/browser coverage: not performed; no provider-operation changes. Real native Windows Notion and Obsidian analysis/export/query workflows passed. - Package or release metadata impact: generated catalog identity changes with its wire representation. - Security/privacy/process review: no target execution, new subprocesses or host configuration changes. Public examples contain no private host paths or credentials. ## Review checklist - [x] Concrete agreed enhancement and related issues linked. - [x] One focused outcome and conventional title. - [x] Meaningful success and failure regression coverage. - [x] Owning docs and generated metadata workflow updated. - [x] Representative wire output included. - [x] Final diff checked for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:09:11 +00:00