[PR #7148] [CLOSED] test(skill): guard bundled SKILL.md references against the advertised MCP catalog #7675

New issue

Closed

opened 2026-11-03 13:39:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:33 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7148
Author: @akushonkamen
Created: 10/25/2026
Status: ❌ Closed

Base: main ← Head: fix/5244-skill-mcp-catalog-consistency

📝 Commits (1)

5d41501 test(skill): guard bundled SKILL.md references against the advertised MCP catalog

📊 Changes

1 file changed (+146 additions, -0 deletions)

View changed files

➕ tests/boundary/skillCatalogConsistency.test.ts (+146 -0)

📄 Description

Problem

Bundled skill v42 documented tools the npm 6.7.0 server did not advertise (inspect_pcap_capture, inspect_sqlite_database, ...). The skill and the MCP tool catalog are produced by separate pipelines, so doctor's catalog digest can report aligned while the skill prose references phantom tools, and every agent following the skill fails with Unknown tool.

Changes

Add a CPU-only boundary test that walks every .agents/skills/**/SKILL.md, extracts backtick-quoted snake_case identifiers, and requires each to be part of the advertised surface: an MCP tool name, an advertised tool input or output schema property or enum value, a stable error code, a CLI command, or a supported client name.
A phantom reference now fails CI before packaging instead of shipping.

Testing

New test tests/boundary/skillCatalogConsistency.test.ts passes on the branch (2/2, verified locally).
The test is CPU-only and does not require a packaged server or MCP runtime.

Fixes #5244

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7148 **Author:** [@akushonkamen](https://github.com/akushonkamen) **Created:** 10/25/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/5244-skill-mcp-catalog-consistency` --- ### 📝 Commits (1) - [`5d41501`](https://github.com/morluto/rea/commit/5d41501f2a84b1258ffc8dc8cb363c12123b1d94) test(skill): guard bundled SKILL.md references against the advertised MCP catalog ### 📊 Changes **1 file changed** (+146 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `tests/boundary/skillCatalogConsistency.test.ts` (+146 -0) </details> ### 📄 Description ## Problem Bundled skill v42 documented tools the npm 6.7.0 server did not advertise (`inspect_pcap_capture`, `inspect_sqlite_database`, ...). The skill and the MCP tool catalog are produced by separate pipelines, so `doctor`'s catalog digest can report aligned while the skill prose references phantom tools, and every agent following the skill fails with `Unknown tool`. ## Changes - Add a CPU-only boundary test that walks every `.agents/skills/**/SKILL.md`, extracts backtick-quoted snake_case identifiers, and requires each to be part of the advertised surface: an MCP tool name, an advertised tool input or output schema property or enum value, a stable error code, a CLI command, or a supported client name. - A phantom reference now fails CI before packaging instead of shipping. ## Testing - New test `tests/boundary/skillCatalogConsistency.test.ts` passes on the branch (2/2, verified locally). - The test is CPU-only and does not require a packaged server or MCP runtime. Fixes #5244 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:39:33 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7675

No description provided.

Rows
Columns