[PR #5323] test(mcp): check advertised defaults against every input group #6879

Open
opened 2026-11-01 21:43:45 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5323
Author: @darshjoshi
Created: 10/20/2026
Status: 🔄 Open

Base: main ← Head: fix/5322-compare-web-captures-normalization-default


📝 Commits (5)

  • f5d8e5b fix(mcp): advertise defaults only for properties every input group accepts
  • b4dc58a test(mcp): send advertised defaults with every input group
  • 2f46aef Merge remote-tracking branch 'upstream/main' into fix/5322-compare-web-captures-normalization-default
  • 6a82075 revert(mcp): drop default filtering now covered on main
  • 4add762 test(mcp): leave the compare_web_captures case to its boundary test

📊 Changes

1 file changed (+47 additions, -0 deletions)

View changed files

📝 tests/boundary/mcp/toolSchemaValidity.test.ts (+47 -0)

📄 Description

Summary

Refs #5322.

88e37806 on main already stops advertising defaults that are not safe for every strict input group, which fixes #5322. As suggested in review, this PR is now narrowed to the coverage main doesn't have. I dropped the overlapping change to src/contracts/inputSchemaPresentation.ts, so the diff is test-only.

What the test adds

Main already checks presentInputJsonSchema on a synthetic union (src/contracts/toolInputReferences.test.ts) and checks compare_web_captures with filled-in defaults (tests/boundary/mcp/browserComparisonInput.test.ts). This PR adds a catalog-wide check to tests/boundary/mcp/toolSchemaValidity.test.ts. It lists every tool over an in-memory MCP connection, then for each tool contract:

  • merges all advertised top-level defaults into each example input, the way a client that fills in defaults would, and checks that the canonical schema still accepts the result;
  • for root unions, checks that every strict group defines each property that has an advertised default.

So a new tool, or a new default on an existing one, that reintroduces this bug fails the test even if no test targets that tool.

Follow the contribution scope. Unresolved scope, authority, side effects, or required verification block integration. Link the required maintainer scope decision for new providers or capabilities involving target execution, injection, mutation, or protection recovery; ordinary fixes within existing boundaries may state "not applicable".

Contract and boundary impact

  • Semantic owner and earliest changed stage: tests only (tests/boundary/mcp/toolSchemaValidity.test.ts)
  • Contribution scope fit and linked maintainer decision: not applicable, test coverage for an existing MCP contract
  • CLI and MCP/tool-catalog contract: unchanged
  • Provider, bridge, target-format, or platform compatibility: none
  • Evidence, artifact, provenance, or reconstruction contract: none
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: describe("MCP advertised input defaults") in tests/boundary/mcp/toolSchemaValidity.test.ts.
  • Base reproduction or other evidence: executed. On current main with this branch merged, all 17 tests in the file pass. With only src/contracts/inputSchemaPresentation.ts restored to 88e37806^, the new test fails with compare_web_captures: default for normalization: expected [ 'before', 'after' ] to include 'normalization'.
  • User-visible CLI/MCP output (if applicable): none; test-only.
  • Remaining proof gaps: none known for the test itself; no real MCP client was run.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Node 24.22.1, macOS arm64, branch merged with current main:

  • npm run test:focused -- tests/boundary/mcp/toolSchemaValidity.test.ts: 17 passed
  • npm run check: typecheck, lint, format and metadata pass. knip reports two unused exports in files this PR doesn't touch (staticFileUrlPath in src/domain/javascript/javascriptStaticAnalysisHelpers.ts, ObjcClassImportBindingCoverage in src/domain/native/objcSwiftMetadata.ts).

Compatibility, safety, and release

  • Breaking changes or migration steps: none
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable
  • Package or release metadata impact: none
  • Security, privacy, process, or containment review: no new effects

Author readiness

Human owner:

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5323 **Author:** [@darshjoshi](https://github.com/darshjoshi) **Created:** 10/20/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/5322-compare-web-captures-normalization-default` --- ### 📝 Commits (5) - [`f5d8e5b`](https://github.com/morluto/rea/commit/f5d8e5b876c4cf90c3d49e72d75d7906e0edb681) fix(mcp): advertise defaults only for properties every input group accepts - [`b4dc58a`](https://github.com/morluto/rea/commit/b4dc58aebcb19a5b0548ffa321ae82bed5945dc9) test(mcp): send advertised defaults with every input group - [`2f46aef`](https://github.com/morluto/rea/commit/2f46aefdc47e348cdae6076d7c45d13615ee14e0) Merge remote-tracking branch 'upstream/main' into fix/5322-compare-web-captures-normalization-default - [`6a82075`](https://github.com/morluto/rea/commit/6a820759e1ceff649ac00b9c89d7c1b2da5b097a) revert(mcp): drop default filtering now covered on main - [`4add762`](https://github.com/morluto/rea/commit/4add7622faa5973b91d8e4ebe1515cca92892af5) test(mcp): leave the compare_web_captures case to its boundary test ### 📊 Changes **1 file changed** (+47 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `tests/boundary/mcp/toolSchemaValidity.test.ts` (+47 -0) </details> ### 📄 Description ## Summary Refs #5322. 88e37806 on main already stops advertising defaults that are not safe for every strict input group, which fixes #5322. As suggested in review, this PR is now narrowed to the coverage main doesn't have. I dropped the overlapping change to `src/contracts/inputSchemaPresentation.ts`, so the diff is test-only. ## What the test adds Main already checks `presentInputJsonSchema` on a synthetic union (`src/contracts/toolInputReferences.test.ts`) and checks `compare_web_captures` with filled-in defaults (`tests/boundary/mcp/browserComparisonInput.test.ts`). This PR adds a catalog-wide check to `tests/boundary/mcp/toolSchemaValidity.test.ts`. It lists every tool over an in-memory MCP connection, then for each tool contract: - merges all advertised top-level defaults into each example input, the way a client that fills in defaults would, and checks that the canonical schema still accepts the result; - for root unions, checks that every strict group defines each property that has an advertised default. So a new tool, or a new default on an existing one, that reintroduces this bug fails the test even if no test targets that tool. Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). Unresolved scope, authority, side effects, or required verification block integration. Link the required maintainer scope decision for new providers or capabilities involving target execution, injection, mutation, or protection recovery; ordinary fixes within existing boundaries may state "not applicable". ## Contract and boundary impact - Semantic owner and earliest changed stage: tests only (`tests/boundary/mcp/toolSchemaValidity.test.ts`) - Contribution scope fit and linked maintainer decision: not applicable, test coverage for an existing MCP contract - CLI and MCP/tool-catalog contract: unchanged - Provider, bridge, target-format, or platform compatibility: none - Evidence, artifact, provenance, or reconstruction contract: none - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [ ] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [x] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: `describe("MCP advertised input defaults")` in `tests/boundary/mcp/toolSchemaValidity.test.ts`. - Base reproduction or other evidence: executed. On current main with this branch merged, all 17 tests in the file pass. With only `src/contracts/inputSchemaPresentation.ts` restored to `88e37806^`, the new test fails with `compare_web_captures: default for normalization: expected [ 'before', 'after' ] to include 'normalization'`. - User-visible CLI/MCP output (if applicable): none; test-only. - Remaining proof gaps: none known for the test itself; no real MCP client was run. For evidence-bearing changes: - [ ] Observed, derived, and inferred claims remain distinguishable. - [ ] Artifact identity, source provenance, and failed attempts remain preserved. - [ ] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Node 24.22.1, macOS arm64, branch merged with current main: - `npm run test:focused -- tests/boundary/mcp/toolSchemaValidity.test.ts`: 17 passed - `npm run check`: typecheck, lint, format and metadata pass. `knip` reports two unused exports in files this PR doesn't touch (`staticFileUrlPath` in `src/domain/javascript/javascriptStaticAnalysisHelpers.ts`, `ObjcClassImportBindingCoverage` in `src/domain/native/objcSwiftMetadata.ts`). ## Compatibility, safety, and release - Breaking changes or migration steps: none - Real Hopper/Ghidra, browser, or OS coverage: not applicable - Package or release metadata impact: none - Security, privacy, process, or containment review: no new effects ## Author readiness <!-- The human owner completes these declarations before requesting review. The PR author is the owner by default. A different owner must explicitly accept ownership on this PR. AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA. Draft PRs may leave them unchecked. These declarations record initial readiness when opening the PR for review; later pushes do not require a new human sign-off. Keep the description and validation evidence current as the work changes. See CONTRIBUTING.md#pull-request-ownership-and-readiness. --> Human owner: <!-- @GitHub-handle --> - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge <!-- Draft PRs may leave pending items unchecked. Resolve material scope and authority questions before integration. --> - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6879
No description provided.