morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5244] [Bug] Bundled skill v42 documents MCP tools absent from npm 6.7.0 server #3209

New issue

Open

opened 2026-10-31 00:37:52 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-31 00:37:52 +00:00

Owner

Originally created by @rezenderaul on GitHub (Oct 19, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5244

Versão e ambiente

rea-agents 6.7.0 (npm, latest), Linux x64, Ghidra 12.1.4. Skill instalada via setup: v42, 207 tools, doctor diz aligned.

Resultado observado

A skill v42 documenta tools que o servidor MCP 6.7.0 não anuncia. Exemplos verificados:
inspect_pcap_capture, inspect_packet_capture, inspect_packet_fields, inspect_sqlite_database, inspect_wasm_artifact, inspect_evm_interface, inspect_go_binary, inspect_rust_binary, signal decoders.

Reprodução mínima

binary_session com {} → tool_availability lista 170 nomes, nenhum dos acima.
Chamada direta retorna: Unknown tool 'rea.inspect_pcap_capture'.
Servidor íntegro: mcp doctor → 207/207 tools, catalog digest confere.

Esperado

Ou a skill empacotada no 6.7.0 reflita o catálogo real do servidor, ou os tools existam. Do jeito atual, o agente segue instruções para tools que não existem.

Originally created by @rezenderaul on GitHub (Oct 19, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5244 ## Versão e ambiente rea-agents 6.7.0 (npm, latest), Linux x64, Ghidra 12.1.4. Skill instalada via `setup`: v42, 207 tools, `doctor` diz `aligned`. ## Resultado observado A skill v42 documenta tools que o servidor MCP 6.7.0 não anuncia. Exemplos verificados: `inspect_pcap_capture`, `inspect_packet_capture`, `inspect_packet_fields`, `inspect_sqlite_database`, `inspect_wasm_artifact`, `inspect_evm_interface`, `inspect_go_binary`, `inspect_rust_binary`, signal decoders. ## Reprodução mínima 1. `binary_session` com `{}` → `tool_availability` lista 170 nomes, nenhum dos acima. 2. Chamada direta retorna: `Unknown tool 'rea.inspect_pcap_capture'`. 3. Servidor íntegro: `mcp doctor` → 207/207 tools, catalog digest confere. ## Esperado Ou a skill empacotada no 6.7.0 reflita o catálogo real do servidor, ou os tools existam. Do jeito atual, o agente segue instruções para tools que não existem.

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 19, 2026):

Claiming this — I'll add a consistency guard that verifies every tool name referenced by bundled SKILL.md files is present in the MCP server catalog.

@akushonkamen commented on GitHub (Oct 19, 2026): Claiming this — I'll add a consistency guard that verifies every tool name referenced by bundled `SKILL.md` files is present in the MCP server catalog.

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 19, 2026):

Root cause and fix progress: the bundled reverse-engineer-anything skill (v42) and the MCP tool catalog are produced by separate pipelines, so doctor's catalog digest can report aligned while the skill prose references tools the packaged server never advertises (e.g. inspect_pcap_capture, inspect_sqlite_database) — agents following the skill then fail with Unknown tool. On current main all referenced tools do exist in TOOL_CONTRACTS; the drift shipped in the 6.7.0 package.

I've added a CPU-only consistency guard (tests/boundary/skillCatalogConsistency.test.ts) that walks every .agents/skills/**/SKILL.md, extracts backtick-quoted snake_case identifiers, and asserts each is part of the advertised surface (MCP tool names, advertised input/output schema properties and enum values, stable error codes, CLI commands, supported client names). Verified: green on main, and turns red when a phantom tool reference is injected into the skill.

Fix is complete on branch fix/5244-skill-mcp-catalog-consistency (pushed to fork); I'll open the PR once the current PR-creation restriction is lifted.

@akushonkamen commented on GitHub (Oct 19, 2026): Root cause and fix progress: the bundled `reverse-engineer-anything` skill (v42) and the MCP tool catalog are produced by separate pipelines, so `doctor`'s catalog digest can report aligned while the skill prose references tools the packaged server never advertises (e.g. `inspect_pcap_capture`, `inspect_sqlite_database`) — agents following the skill then fail with `Unknown tool`. On current `main` all referenced tools do exist in `TOOL_CONTRACTS`; the drift shipped in the 6.7.0 package. I've added a CPU-only consistency guard (`tests/boundary/skillCatalogConsistency.test.ts`) that walks every `.agents/skills/**/SKILL.md`, extracts backtick-quoted snake_case identifiers, and asserts each is part of the advertised surface (MCP tool names, advertised input/output schema properties and enum values, stable error codes, CLI commands, supported client names). Verified: green on main, and turns red when a phantom tool reference is injected into the skill. Fix is complete on branch [`fix/5244-skill-mcp-catalog-consistency`](https://github.com/akushonkamen/rea/tree/fix/5244-skill-mcp-catalog-consistency) (pushed to fork); I'll open the PR once the current PR-creation restriction is lifted.

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 21, 2026):

Thanks, rezenderaul, for the 170 versus 207 count. From reading, not a run, I can't settle this one. At the rea-agents-6.7.0 tag (27dc42d102), the file src/contracts/toolContracts.ts:56 includes the pcap contracts in TOOL_CONTRACTS. The file src/application/CapabilityInventory.ts:181 lists every contract in tool_availability. So the tagged source does not explain a 170-name list. The skill on main at 01788bac8 is v44, and akushonkamen's consistency test is not on main. What would help: the tool count from the server's own tools/list, and the path of the rea-agents package that served it. Related: #654 (closed).

🤖 Written with Claude Code (Fable 5.1, verified by Opus 5.5), posted with my OK 🦊

@CassieFox23 commented on GitHub (Oct 21, 2026): Thanks, rezenderaul, for the 170 versus 207 count. From reading, not a run, I can't settle this one. At the rea-agents-6.7.0 tag (27dc42d102), the file `src/contracts/toolContracts.ts:56` includes the pcap contracts in `TOOL_CONTRACTS`. The file `src/application/CapabilityInventory.ts:181` lists every contract in `tool_availability`. So the tagged source does not explain a 170-name list. The skill on main at 01788bac8 is v44, and akushonkamen's consistency test is not on main. What would help: the tool count from the server's own tools/list, and the path of the rea-agents package that served it. Related: #654 (closed). 🤖 Written with Claude Code (Fable 5.1, verified by Opus 5.5), posted with my OK 🦊

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@rezenderaul commented on GitHub (Oct 22, 2026):

Atualização com a causa raiz verificada aqui (Linux x64, pacote npm 6.7.0 local):

Os contratos existem no bundle instalado (dist/mcp/toolContracts-*.js, TsharkCaptureProvider). O que acontece é gate de ambiente na hora de anunciar: p. ex. packetCaptureEnabled = ... || (linux && isAbsolute(env.REA_TSHARK_COMMAND)). Nosso MCP rodava sem nenhuma variável REA_*, então as tools sumiam do binary_session (170 nomes) e a chamada direta falhava com Unknown tool — enquanto o doctor reportava 207/207 (conta contratos, não o que foi anunciado).

Após adicionar REA_TSHARK_COMMAND=/usr/bin/tshark + REA_CAPINFOS_COMMAND=/usr/bin/capinfos ao MCP e reiniciar, o binary_session passou a anunciar 207 nomes incluindo inspect_pcap_capture, inspect_packet_*, inspect_pcap_client (+ sqlite/wasm/evm/go/rust/signal), e validei e2e com um pcap real (2 pacotes DNS → endpoints/DNS/protocolos com Evidence, tshark 4.2.2 via env).

Sugestão, além/instead do guard de catálogo: quando o gate de ambiente ocultar tools, anunciá-las como indisponíveis com o motivo (falta X env) em vez de sumir com elas — o 'Unknown tool' sem dica foi o que tornou o diagnóstico caro. E o doctor poderia distinguir 'contratos no pacote' de 'tools anunciadas nesta sessão'.

@rezenderaul commented on GitHub (Oct 22, 2026): Atualização com a causa raiz verificada aqui (Linux x64, pacote npm 6.7.0 local): Os contratos **existem** no bundle instalado (`dist/mcp/toolContracts-*.js`, `TsharkCaptureProvider`). O que acontece é gate de ambiente na hora de anunciar: p. ex. `packetCaptureEnabled = ... || (linux && isAbsolute(env.REA_TSHARK_COMMAND))`. Nosso MCP rodava sem nenhuma variável `REA_*`, então as tools sumiam do `binary_session` (170 nomes) e a chamada direta falhava com `Unknown tool` — enquanto o `doctor` reportava 207/207 (conta contratos, não o que foi anunciado). Após adicionar `REA_TSHARK_COMMAND=/usr/bin/tshark` + `REA_CAPINFOS_COMMAND=/usr/bin/capinfos` ao MCP e reiniciar, o `binary_session` passou a anunciar **207 nomes** incluindo `inspect_pcap_capture`, `inspect_packet_*`, `inspect_pcap_client` (+ sqlite/wasm/evm/go/rust/signal), e validei e2e com um pcap real (2 pacotes DNS → endpoints/DNS/protocolos com Evidence, tshark 4.2.2 via env). Sugestão, além/instead do guard de catálogo: quando o gate de ambiente ocultar tools, anunciá-las como indisponíveis com o motivo (falta X env) em vez de sumir com elas — o 'Unknown tool' sem dica foi o que tornou o diagnóstico caro. E o `doctor` poderia distinguir 'contratos no pacote' de 'tools anunciadas nesta sessão'.

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 22, 2026):

Thanks, rezenderaul, for the root cause and the numbers. I read main at fd5cd1b05 and the rea-agents-6.7.0 tag, and the gate you quote is real. But it covers only inspect_packet_capture and inspect_packet_fields, which are always registered and are listed as unavailable with a reason. The pcap tools are always reported available. The wasm, signal and EVM tools you saw return have their own gates, and none reads REA_TSHARK_COMMAND. Could you paste the package path and launch command for the run that showed 170 names and the run that showed 207?

🤖 Written with Claude Code (Sonnet 5.5, verified by Opus 5.5), posted with my OK 🦊

@CassieFox23 commented on GitHub (Oct 22, 2026): Thanks, rezenderaul, for the root cause and the numbers. I read main at fd5cd1b05 and the rea-agents-6.7.0 tag, and the [gate you quote](https://github.com/morluto/rea/blob/fd5cd1b058dd549b5f5c6d707edeaebcc21c1165/src/server/createServer.ts#L248-L250) is real. But it covers only `inspect_packet_capture` and `inspect_packet_fields`, which are always [registered](https://github.com/morluto/rea/blob/fd5cd1b058dd549b5f5c6d707edeaebcc21c1165/src/server/createServer.ts#L668-L674) and are listed as unavailable with a reason. The pcap tools are always [reported available](https://github.com/morluto/rea/blob/fd5cd1b058dd549b5f5c6d707edeaebcc21c1165/src/application/CapabilityInventory.ts#L591-L592). The wasm, signal and EVM tools you saw return have [their own gates](https://github.com/morluto/rea/blob/fd5cd1b058dd549b5f5c6d707edeaebcc21c1165/src/server/createServer.ts#L251-L258), and none reads `REA_TSHARK_COMMAND`. Could you paste the package path and launch command for the run that showed 170 names and the run that showed 207? 🤖 Written with Claude Code (Sonnet 5.5, verified by Opus 5.5), posted with my OK 🦊

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@henriquebalesteros commented on GitHub (Oct 23, 2026):

Ran the published 6.7.0 package on a third platform, since the runs in this thread are Linux-only and the two root-cause candidates point at different layers.

Environment: Windows 10 x64 (10.0.19045), Node v24.16.0, npm install rea-agents@6.7.0 (registry integrity sha512-yhIx2ioJYYT1C/tONfYeitC6LhnKmCh+ZwRUOgYuhSc733lGhkkt73zhug9WmLgxoMdKuqIYGCoBfD4fXkOhFw==), default environment with no REA_* variables set, server started with rea --mcp over stdio, MCP initialize (negotiated 2025-06-18), then plain tools/list / tools/call from a ~60-line Node driver.

Measured:

tools/list advertises 207 tools, including every tool named in this issue — inspect_pcap_capture, inspect_packet_capture, inspect_packet_fields, inspect_sqlite_database, inspect_wasm_artifact, inspect_evm_interface, inspect_go_binary, inspect_rust_binary, inspect_signal_decoders, decode_signal_recording. The full response is 3,158,368 bytes of JSON. So on this platform the packaged skill's tool_count: 207 matches the server's advertised surface exactly, and the headline symptom ("tools absent from npm 6.7.0 server") does not reproduce.
A direct tools/call of inspect_pcap_capture with {} does not return "Unknown tool" — the server accepts it and returns a structured invalid_request (missing_argument: path, retryable: true). If the reporter saw Unknown tool 'rea.inspect_pcap_capture', that error shape points to a client-side filter upstream of the server (declining a tool absent from the availability list it saw), not to a server response.
binary_session's tool_availability lists all 207 names here too — 78 available, 129 unavailable — and every unavailable group carries a reason plus a remediation string (e.g. provider_missing → "On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND…"). That connects the two existing reports: @rezenderaul's proposed "announce unavailable tools with the reason" already exists in the Windows availability surface today, while the 170-name lists seen on Linux suggest that path emits fewer entries for the same command. I did not measure Linux, so that part is inference — but two numbers from one session on the affected install would settle it: the tools/list count and the binary_session tool_availability entry count. If the first is 207 and the second is 170, this is a platform-inconsistent availability report rather than skill drift, and @akushonkamen's consistency guard would be worth extending to assert per-platform surface equality (the packaged skill already tells agents to trust "the connected server's actual tool list", so keeping that list complete and uniform across platforms is the contract that matters).

Happy to paste the driver and full outputs if useful.

(Comment drafted with AI assistance; every number above was verified by me against the runs described.)

@henriquebalesteros commented on GitHub (Oct 23, 2026): Ran the published 6.7.0 package on a third platform, since the runs in this thread are Linux-only and the two root-cause candidates point at different layers. Environment: Windows 10 x64 (10.0.19045), Node v24.16.0, `npm install rea-agents@6.7.0` (registry integrity `sha512-yhIx2ioJYYT1C/tONfYeitC6LhnKmCh+ZwRUOgYuhSc733lGhkkt73zhug9WmLgxoMdKuqIYGCoBfD4fXkOhFw==`), default environment with no `REA_*` variables set, server started with `rea --mcp` over stdio, MCP `initialize` (negotiated `2025-06-18`), then plain `tools/list` / `tools/call` from a ~60-line Node driver. Measured: 1. **`tools/list` advertises 207 tools**, including every tool named in this issue — `inspect_pcap_capture`, `inspect_packet_capture`, `inspect_packet_fields`, `inspect_sqlite_database`, `inspect_wasm_artifact`, `inspect_evm_interface`, `inspect_go_binary`, `inspect_rust_binary`, `inspect_signal_decoders`, `decode_signal_recording`. The full response is 3,158,368 bytes of JSON. So on this platform the packaged skill's `tool_count: 207` matches the server's advertised surface exactly, and the headline symptom ("tools absent from npm 6.7.0 server") does not reproduce. 2. **A direct `tools/call` of `inspect_pcap_capture` with `{}` does not return "Unknown tool"** — the server accepts it and returns a structured `invalid_request` (`missing_argument: path`, `retryable: true`). If the reporter saw `Unknown tool 'rea.inspect_pcap_capture'`, that error shape points to a client-side filter upstream of the server (declining a tool absent from the availability list it saw), not to a server response. 3. **`binary_session`'s `tool_availability` lists all 207 names here too** — 78 available, 129 unavailable — and every unavailable group carries a `reason` plus a `remediation` string (e.g. `provider_missing` → "On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND…"). That connects the two existing reports: @rezenderaul's proposed "announce unavailable tools with the reason" already exists in the Windows availability surface today, while the 170-name lists seen on Linux suggest that path emits fewer entries for the same command. I did not measure Linux, so that part is inference — but two numbers from one session on the affected install would settle it: the `tools/list` count and the `binary_session` `tool_availability` entry count. If the first is 207 and the second is 170, this is a platform-inconsistent availability report rather than skill drift, and @akushonkamen's consistency guard would be worth extending to assert per-platform surface equality (the packaged skill already tells agents to trust "the connected server's actual tool list", so keeping that list complete and uniform across platforms is the contract that matters). Happy to paste the driver and full outputs if useful. (Comment drafted with AI assistance; every number above was verified by me against the runs described.)

bsakaguchi commented

2026-10-31 00:37:53 +00:00

Author

Owner

@morluto commented on GitHub (Oct 26, 2026):

Obrigado pelo relato. O binary_session.tool_availability descreve as operações da sessão binária e não é, por si só, o catálogo completo de tools/list; isso pode explicar a diferença entre 170 e 207 nomes. A chamada que retornou Unknown tool, porém, ainda precisa ser esclarecida. Você poderia compartilhar o nome exato anunciado pelo cliente em tools/list e a chamada completa com dados privados removidos? O prefixo exibido pode pertencer ao cliente. Vou manter a issue aberta até confirmar o caminho da chamada.

@morluto commented on GitHub (Oct 26, 2026): Obrigado pelo relato. O `binary_session.tool_availability` descreve as operações da sessão binária e não é, por si só, o catálogo completo de `tools/list`; isso pode explicar a diferença entre 170 e 207 nomes. A chamada que retornou `Unknown tool`, porém, ainda precisa ser esclarecida. Você poderia compartilhar o nome exato anunciado pelo cliente em `tools/list` e a chamada completa com dados privados removidos? O prefixo exibido pode pertencer ao cliente. Vou manter a issue aberta até confirmar o caminho da chamada.

bsakaguchi referenced this issue

2026-10-31 00:48:17 +00:00

[GH-ISSUE #7077] Linux Hopper setup fails for current vendor release: fixed 120 s download deadline exceeded and feed main_exe_hash does not match the published launcher #4266

bsakaguchi referenced this issue

2026-10-31 00:48:18 +00:00

[GH-ISSUE #7077] Linux Hopper setup fails for current vendor release: fixed 120 s download deadline exceeded and feed main_exe_hash does not match the published launcher #4266

bsakaguchi referenced this issue

2026-10-31 01:09:14 +00:00

[PR #3472] [CLOSED] fix(setup): keep the completed Hopper installation when the final open fails #5763