[GH-ISSUE #8620] test: avoid whole-catalog schema compilation in deferred runtime checks #7496

New issue

Closed

opened 2026-11-03 00:30:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 00:30:13 +00:00

Owner

Originally created by @gensokyo-dev on GitHub (Nov 2, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8620

Problem

On main 00c4f44a1c002f9801ee535386be60c30b23d20e, tests/boundary/mcp/deferredRuntime.test.ts repeatedly lets a fresh MCP SDK client compile every advertised tool's output schema when it calls one tool. The complete catalog is already compared independently in this file. The SDK supports validating the actual named toolDefinition instead.

Observed: the file cost 117.7s in main CI. A local complete 28-case run passed in 126.0s, with 150.5s descendant CPU and 2.25 GiB peak sampled process-group RSS.

Reproduction executed

Pinned Node 24.22.1 / npm 11.16.0, Linux x64, prepared runtime and catalog, four CPU affinity, two Vitest workers:

npm run test:prepare
node node_modules/vitest/vitest.mjs run tests/boundary/mcp/deferredRuntime.test.ts --reporter=verbose

RSS was summed over the owned process group at nominal 100ms intervals; shared pages may be counted twice. CPU comes from waited descendants. A 3 GiB sampled-RSS stop and 300s wall deadline bounded the local run. These are file-level measurements, not a CI-wide speedup or agent-token measurement.

Proposed correction

Use the existing advertised-schema call helper for schema-bearing calls and select the connection's named discovery definition for cases also covering output schemas disabled. Preserve every case, separate real stdio sessions, full catalog comparisons, concurrent first requests, canonical argument rejection, startup loading checks, and cleanup regressions. Validate malformed-output rejection as well as performance.

Related historical audit: #2190. Latest main CI is now green; this is a measured efficiency follow-up.

Originally created by @gensokyo-dev on GitHub (Nov 2, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8620 ## Problem On main `00c4f44a1c002f9801ee535386be60c30b23d20e`, `tests/boundary/mcp/deferredRuntime.test.ts` repeatedly lets a fresh MCP SDK client compile every advertised tool's output schema when it calls one tool. The complete catalog is already compared independently in this file. The SDK supports validating the actual named `toolDefinition` instead. Observed: the file cost 117.7s in [main CI](https://github.com/morluto/rea/actions/runs/41380640209). A local complete 28-case run passed in 126.0s, with 150.5s descendant CPU and 2.25 GiB peak sampled process-group RSS. ## Reproduction executed Pinned Node 24.22.1 / npm 11.16.0, Linux x64, prepared runtime and catalog, four CPU affinity, two Vitest workers: ```sh npm run test:prepare node node_modules/vitest/vitest.mjs run tests/boundary/mcp/deferredRuntime.test.ts --reporter=verbose ``` RSS was summed over the owned process group at nominal 100ms intervals; shared pages may be counted twice. CPU comes from waited descendants. A 3 GiB sampled-RSS stop and 300s wall deadline bounded the local run. These are file-level measurements, not a CI-wide speedup or agent-token measurement. ## Proposed correction Use the existing advertised-schema call helper for schema-bearing calls and select the connection's named discovery definition for cases also covering output schemas disabled. Preserve every case, separate real stdio sessions, full catalog comparisons, concurrent first requests, canonical argument rejection, startup loading checks, and cleanup regressions. Validate malformed-output rejection as well as performance. Related historical audit: #2190. Latest main CI is now green; this is a measured efficiency follow-up.

bsakaguchi closed this issue

2026-11-03 00:30:13 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7496

No description provided.

Rows
Columns