morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #5658] [MERGED] perf(test): reuse one stdio MCP server per JavaScript application file #7016

New issue

Closed

opened 2026-11-01 21:44:44 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:44:44 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5658
Author: @morluto
Created: 10/20/2026
Status: ✅ Merged
Merged: 10/20/2026
Merged by: @morluto

Base: main ← Head: cursor/vitest-mcp-session-reuse-b482

📝 Commits (5)

0c2e5c5 perf(test): reuse one stdio MCP server per JavaScript application file
8ea6fb2 fix(test): close in-flight shared MCP startup with the file
81c56af fix(test): keep a failed MCP handshake when cleanup fails
c04ed74 fix(test): fail shared MCP cleanup when the rejection has no reason
d99ce91 fix(test): keep one MCP server while failed cleanup is retained

📊 Changes

4 files changed (+473 additions, -7 deletions)

View changed files

📝 docs/testing.md (+11 -0)
➕ tests/boundary/cli/sharedJavaScriptMcpSession.test.ts (+264 -0)
📝 tests/support/cli/cliFixture.ts (+181 -2)
📝 tests/support/cli/javascriptApplication.ts (+17 -5)

📄 Description

Summary

The slowest built Vitest shard is about 15 minutes because file-duration weight is balanced near 24.6 minutes and CI runs two workers. The three slowest files were starting a full stdio MCP server for every JavaScript application case. Those cases now share one file-scoped server.

Human owner: @morluto

Problem and expected behavior

On main run 39408744482 (052956af), Test shard (6/6) took 15.03 minutes, of which the portable suite step was 14.60 minutes. Historical test-timings (1785 files, 147.5 minutes of file weight) pack evenly to 24.58 minutes per shard. With two workers and process-boundary serial after the other projects, the estimated test makespan is 13.04 minutes.

The slowest files were almost all test execution, not imports:

File	CI duration
`tests/acceptance/applications/javascriptLiteralPercentPaths.test.ts`	574 s
`tests/acceptance/applications/javascriptLoaderReviewBoundaries.test.ts`	370 s
`tests/acceptance/applications/javascriptIndirectLoaderOwners.test.ts`	238 s

Each stdio MCP case in those files called connectLocalToolsMcp(), which starts rea mcp, lists tools, and exits. A warm repeated call on one server is about 0.5 s locally; a fresh case was about 2.3–2.8 s.

process-boundary stays serial. It is 4.78 minutes across 72 files, and the heaviest packed serial tail is 1.51 minutes, because those tests observe the host process table. Splitting non-sampling files out of that project would risk perturbing the sampling oracle, so this change does not do that.

Change and scope

cliTest keeps one file-scoped stdio server, started on the first sharedMcpCall(), with the worker's Node compile cache. analyzeJavaScriptDirectory uses it when the case does not pass mcpCall. CLI cases still start a fresh process. electronWrappedCallPaths keeps its explicit caller.

File teardown marks the session closed before waiting. Startup still in the dynamic import does not open a server. A server that registers its cleanup after close has begun is still closed, including when a failed handshake is already draining. The later-case reuse test records the session when it is the first selected case, and compares identity when an earlier case in the file already recorded it.

A failed handshake stays the error returned to the case when its cleanup also fails, including a rejection without a reason. The cleanup failure is attached on an AggregateError whose cause is the handshake. That cleanup stays owned until a later close or call releases it. Another server does not open while it remains. Close caches completion only after every retained cleanup succeeds, and a cleanup that already succeeded is not run again.

Rebased onto main 0add6a5 (#5634). The diff against main is still only the shared stdio session.

Contribution scope: not applicable. This is the existing test harness. No new provider, target execution, injection, mutation, or protection recovery.

Contract and boundary impact

Semantic owner and earliest changed stage: test harness (cliTest / analyzeJavaScriptDirectory). No product stage changes.
Contribution scope fit and linked maintainer decision: not applicable.
CLI and MCP/tool-catalog contract: unchanged. Tests still call the public CLI and analyze_javascript_application.
Provider, bridge, target-format, or platform compatibility: none.
Evidence, artifact, provenance, or reconstruction contract: none.
Target selection, authority, upstream behavior, and cleanup details (if applicable): the shared server is closed when the file finishes, including startup still in flight. A failed handshake closes that attempt. If that cleanup fails, the handshake remains the caller error, and the next case retries the cleanup before opening another server.
Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Test-only. No new product target interaction.

Evidence and regression coverage

Tests added or updated: tests/boundary/cli/sharedJavaScriptMcpSession.test.ts checks explicit callers, same-file reuse, a filtered later case, startup that loses the race with close, a server registered after close starts, a failed handshake whose cleanup also fails, a reasonless cleanup rejection retried by a later close, and a blocked reopen while that cleanup remains.
Base reproduction or other evidence: local before/after on the three files above, Node 24.18.0, 4 vCPU, no coverage, 398 tests passed both times.

File	Before	After
`javascriptLiteralPercentPaths.test.ts`	378.8 s	191.9 s
`javascriptLoaderReviewBoundaries.test.ts`	241.8 s	123.0 s
`javascriptIndirectLoaderOwners.test.ts`	158.7 s	80.8 s
Sum	779.3 s	395.6 s

Applying those ratios to the historical durations and rebalancing six shards (two workers, process-boundary still serial) moves the estimated max makespan from 13.04 minutes to 12.30 minutes. A startup model for the other analyzeJavaScriptDirectory files, which were checked for correctness but not re-timed, adds about 3.5 minutes of weight and moves that estimate to about 11.95 minutes. The second figure is a model. Coverage instrumentation was not part of the local timing.

User-visible CLI/MCP output (if applicable): none.
Remaining proof gaps: no full CI shard was re-run here. CLI cases are still about 1.4 s each and now dominate these files. process-boundary is unchanged.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npx vitest run on the three files above, before the change — 398 passed, 779.3 s of file time.
Same command after the change — 398 passed, 395.6 s of file time.
npx vitest run on 39 related files (sharedJavaScriptMcpSession, cliCompileCache, electronWrappedCallPaths, and the other analyzeJavaScriptDirectory suites) — 243 passed.
After rebase onto 0add6a5: npm run test:focused -- tests/boundary/cli/sharedJavaScriptMcpSession.test.ts — 9 passed. The later case took 337 ms after the first case paid startup.
After rebase onto 0add6a5: npm run check:fast — typecheck and lint passed.
node scripts/run-exact-tests.mjs run --project cli-boundary tests/boundary/cli/sharedJavaScriptMcpSession.test.ts -t "reuses that stdio MCP session" — 1 passed, 5 skipped, on an earlier head before the cleanup-failure tests.

Compatibility, safety, and release

Breaking changes or migration steps: none. A case that needs its own server, environment, or lifecycle passes mcpCall.
Real Hopper/Ghidra, browser, or OS coverage: not applicable.
Package or release metadata impact: none.
Security, privacy, process, or containment review: the shared server lives until the file ends and is closed by the fixture, including when startup is still in flight. It uses the same default environment as the previous per-case server.

Author readiness

Human owner: @morluto

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

No linked issue: this follows the requested CI shard investigation rather than an open bug.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5658 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/20/2026 **Status:** ✅ Merged **Merged:** 10/20/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `cursor/vitest-mcp-session-reuse-b482` --- ### 📝 Commits (5) - [`0c2e5c5`](https://github.com/morluto/rea/commit/0c2e5c5a4acbe5a7206bf74b4c20967df33d6bcd) perf(test): reuse one stdio MCP server per JavaScript application file - [`8ea6fb2`](https://github.com/morluto/rea/commit/8ea6fb297cf2806423b86326dd2654f4ca83374e) fix(test): close in-flight shared MCP startup with the file - [`81c56af`](https://github.com/morluto/rea/commit/81c56afdfdc89f7d345a065ee7cacbc263e61c4e) fix(test): keep a failed MCP handshake when cleanup fails - [`c04ed74`](https://github.com/morluto/rea/commit/c04ed74286f1b1e368067f76ba95748e39595dcd) fix(test): fail shared MCP cleanup when the rejection has no reason - [`d99ce91`](https://github.com/morluto/rea/commit/d99ce913e0a4c0b93a56601e2d1e3593ff36236b) fix(test): keep one MCP server while failed cleanup is retained ### 📊 Changes **4 files changed** (+473 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `docs/testing.md` (+11 -0) ➕ `tests/boundary/cli/sharedJavaScriptMcpSession.test.ts` (+264 -0) 📝 `tests/support/cli/cliFixture.ts` (+181 -2) 📝 `tests/support/cli/javascriptApplication.ts` (+17 -5) </details> ### 📄 Description  ## Summary The slowest built Vitest shard is about 15 minutes because file-duration weight is balanced near 24.6 minutes and CI runs two workers. The three slowest files were starting a full stdio MCP server for every JavaScript application case. Those cases now share one file-scoped server. Human owner: @morluto ## Problem and expected behavior On main run [39408744482](https://github.com/morluto/rea/actions/runs/39408744482) (`052956af`), Test shard (6/6) took 15.03 minutes, of which the portable suite step was 14.60 minutes. Historical `test-timings` (1785 files, 147.5 minutes of file weight) pack evenly to 24.58 minutes per shard. With two workers and `process-boundary` serial after the other projects, the estimated test makespan is 13.04 minutes. The slowest files were almost all test execution, not imports: | File | CI duration | | --- | ---: | | `tests/acceptance/applications/javascriptLiteralPercentPaths.test.ts` | 574 s | | `tests/acceptance/applications/javascriptLoaderReviewBoundaries.test.ts` | 370 s | | `tests/acceptance/applications/javascriptIndirectLoaderOwners.test.ts` | 238 s | Each stdio MCP case in those files called `connectLocalToolsMcp()`, which starts `rea mcp`, lists tools, and exits. A warm repeated call on one server is about 0.5 s locally; a fresh case was about 2.3–2.8 s. `process-boundary` stays serial. It is 4.78 minutes across 72 files, and the heaviest packed serial tail is 1.51 minutes, because those tests observe the host process table. Splitting non-sampling files out of that project would risk perturbing the sampling oracle, so this change does not do that. ## Change and scope `cliTest` keeps one file-scoped stdio server, started on the first `sharedMcpCall()`, with the worker's Node compile cache. `analyzeJavaScriptDirectory` uses it when the case does not pass `mcpCall`. CLI cases still start a fresh process. `electronWrappedCallPaths` keeps its explicit caller. File teardown marks the session closed before waiting. Startup still in the dynamic import does not open a server. A server that registers its cleanup after close has begun is still closed, including when a failed handshake is already draining. The later-case reuse test records the session when it is the first selected case, and compares identity when an earlier case in the file already recorded it. A failed handshake stays the error returned to the case when its cleanup also fails, including a rejection without a reason. The cleanup failure is attached on an `AggregateError` whose cause is the handshake. That cleanup stays owned until a later close or call releases it. Another server does not open while it remains. Close caches completion only after every retained cleanup succeeds, and a cleanup that already succeeded is not run again. Rebased onto main `0add6a5` (#5634). The diff against main is still only the shared stdio session. Contribution scope: not applicable. This is the existing test harness. No new provider, target execution, injection, mutation, or protection recovery. ## Contract and boundary impact - Semantic owner and earliest changed stage: test harness (`cliTest` / `analyzeJavaScriptDirectory`). No product stage changes. - Contribution scope fit and linked maintainer decision: not applicable. - CLI and MCP/tool-catalog contract: unchanged. Tests still call the public CLI and `analyze_javascript_application`. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: none. - Target selection, authority, upstream behavior, and cleanup details (if applicable): the shared server is closed when the file finishes, including startup still in flight. A failed handshake closes that attempt. If that cleanup fails, the handshake remains the caller error, and the next case retries the cleanup before opening another server. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. Test-only. No new product target interaction. ## Evidence and regression coverage - Tests added or updated: `tests/boundary/cli/sharedJavaScriptMcpSession.test.ts` checks explicit callers, same-file reuse, a filtered later case, startup that loses the race with close, a server registered after close starts, a failed handshake whose cleanup also fails, a reasonless cleanup rejection retried by a later close, and a blocked reopen while that cleanup remains. - Base reproduction or other evidence: local before/after on the three files above, Node 24.18.0, 4 vCPU, no coverage, 398 tests passed both times. | File | Before | After | | --- | ---: | ---: | | `javascriptLiteralPercentPaths.test.ts` | 378.8 s | 191.9 s | | `javascriptLoaderReviewBoundaries.test.ts` | 241.8 s | 123.0 s | | `javascriptIndirectLoaderOwners.test.ts` | 158.7 s | 80.8 s | | Sum | 779.3 s | 395.6 s | Applying those ratios to the historical durations and rebalancing six shards (two workers, `process-boundary` still serial) moves the estimated max makespan from 13.04 minutes to 12.30 minutes. A startup model for the other `analyzeJavaScriptDirectory` files, which were checked for correctness but not re-timed, adds about 3.5 minutes of weight and moves that estimate to about 11.95 minutes. The second figure is a model. Coverage instrumentation was not part of the local timing. - User-visible CLI/MCP output (if applicable): none. - Remaining proof gaps: no full CI shard was re-run here. CLI cases are still about 1.4 s each and now dominate these files. `process-boundary` is unchanged. For evidence-bearing changes: - [ ] Observed, derived, and inferred claims remain distinguishable. - [ ] Artifact identity, source provenance, and failed attempts remain preserved. - [ ] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npx vitest run` on the three files above, before the change — 398 passed, 779.3 s of file time. - Same command after the change — 398 passed, 395.6 s of file time. - `npx vitest run` on 39 related files (`sharedJavaScriptMcpSession`, `cliCompileCache`, `electronWrappedCallPaths`, and the other `analyzeJavaScriptDirectory` suites) — 243 passed. - After rebase onto `0add6a5`: `npm run test:focused -- tests/boundary/cli/sharedJavaScriptMcpSession.test.ts` — 9 passed. The later case took 337 ms after the first case paid startup. - After rebase onto `0add6a5`: `npm run check:fast` — typecheck and lint passed. - `node scripts/run-exact-tests.mjs run --project cli-boundary tests/boundary/cli/sharedJavaScriptMcpSession.test.ts -t "reuses that stdio MCP session"` — 1 passed, 5 skipped, on an earlier head before the cleanup-failure tests. ## Compatibility, safety, and release - Breaking changes or migration steps: none. A case that needs its own server, environment, or lifecycle passes `mcpCall`. - Real Hopper/Ghidra, browser, or OS coverage: not applicable. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the shared server lives until the file ends and is closed by the fixture, including when startup is still in flight. It uses the same default environment as the previous per-case server. ## Author readiness Human owner: @morluto - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. No linked issue: this follows the requested CI shard investigation rather than an open bug.  <div><a href="https://cursor.com/agents/bc-94ab3e99-a14c-5a19-9ab9-6b6bcbf8b482?cursor_ref=pr_footer&cursor_cta=open_in_web"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-web-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-web-light.png"><img alt="Open in Web" width="114" height="28" src="https://cursor.com/assets/images/open-in-web-dark.png"></picture></a> <a href="https://cursor.com/background-agent?bcId=bc-94ab3e99-a14c-5a19-9ab9-6b6bcbf8b482&cursor_ref=pr_footer&cursor_cta=open_in_cursor"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/assets/images/open-in-cursor-dark.png"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/assets/images/open-in-cursor-light.png"><img alt="Open in Cursor" width="131" height="28" src="https://cursor.com/assets/images/open-in-cursor-dark.png"></picture></a> </div> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:44:44 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-03 13:38:49 +00:00

[PR #7016] [MERGED] test(javascript): bound route scan work independently of timing #7576

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7016

No description provided.

Rows
Columns