[GH-ISSUE #3222] [Bug] MCP catalog alignment misses changes to guided prompt objectives #1160

New issue

Closed

opened 2026-10-15 18:31:37 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:31:37 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3222

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

A freshly initialized REA MCP server reports aligned when the caller supplies a catalog digest from a guided workflow with a different objective. prompts/get returns the changed objective, but prompts_sha256 and combined_sha256 remain equal to the original catalog. binary_session returns an empty alignment-reason list and no remediation.

The independent verifier reproduced this through the production server and the MCP SDK's in-memory transport. Each variant ran in a separate process and changed its contract before server and catalog construction. A title-only control changed both prompt-related digests and correctly returned mcp_server_restart_required with catalog_digest_mismatch.

Reproduction

Start from a clean checkout of the verified revision and install its locked dependencies. Use the repository's declared Node.js and npm toolchain.

git clone https://github.com/morluto/rea.git rea-catalog-repro
cd rea-catalog-repro
git checkout 58300885207133422af235e0f2b10658cf958b03
npm ci
npm run build:cached
mkdir -p .repro-catalog/home/config .repro-catalog/home/cache .repro-catalog/home/data

Create the inline fixture. It starts the production server with an empty provider registry, retrieves investigate_feature, and calls binary_session. The objective and title variants model builds differing in one contract field without editing source or runtime files.

cat > .repro-catalog/repro.mjs <<'JS'
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const fromRoot = path => import(pathToFileURL(resolve(path)).href);
const require = createRequire(resolve('package.json'));
const { Client, InMemoryTransport } = await import(require.resolve('@modelcontextprotocol/client'));
const variant = process.argv[2];
const expected = variant === 'baseline' ? undefined : JSON.parse(await readFile(process.argv[3], 'utf8')).digests.combined_sha256;
const { PROMPT_CONTRACTS } = await fromRoot('dist/contracts/promptContracts.js');
const contract = PROMPT_CONTRACTS.find(prompt => prompt.name === 'investigate_feature');
const originalObjective = contract.objective;
if (variant === 'objective') contract.objective += ' Objective-only verification marker.';
if (variant === 'title') contract.title += ' Title-only verification marker.';
const { createServer } = await fromRoot('dist/server/createServer.js');
const { BinarySession } = await fromRoot('dist/application/binary/BinarySession.js');
const { SessionProviderRouter } = await fromRoot('dist/application/binary/SessionProviderRouter.js');
const { AnalysisProviderRegistry } = await fromRoot('dist/application/binary/AnalysisProviderRegistry.js');
const session = new BinarySession(SessionProviderRouter.selectable(new AnalysisProviderRegistry([]), []));
const server = createServer({ kind: 'session', session }, { environment: {} });
const client = new Client({ name: 'catalog-verifier', version: '1' });
const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair();
try {
  await server.connect(serverTransport);
  await client.connect(clientTransport);
  const prompt = await client.getPrompt({ name: 'investigate_feature', arguments: { feature: 'offline search' } });
  const text = prompt.messages[0].content.text;
  assert.ok(text.includes(`Objective: ${contract.objective}`));
  const reply = await client.callTool({ name: 'binary_session', arguments: expected === undefined ? {} : { expected_catalog_digest: expected } });
  assert.notEqual(reply.isError, true);
  const identity = reply.structuredContent.result.server_identity;
  console.log(JSON.stringify({ variant, objectiveChanged: contract.objective !== originalObjective, renderedObjective: text.split('\n').find(line => line.startsWith('Objective:')), digests: identity.catalog.digests, alignment: identity.alignment }, null, 2));
} finally {
  await client.close();
  await server.close();
  await session.close();
}
JS

Run three fresh Node.js processes with isolated configuration directories. The first saves the baseline digest; the other two supply it as expected_catalog_digest.

python3 - <<'PYTHON'
import os
import subprocess
from pathlib import Path

repro_root = Path('.repro-catalog').resolve()
repro_environment = dict(os.environ)
repro_environment.update({
    'HOME': str(repro_root / 'home'),
    'XDG_CONFIG_HOME': str(repro_root / 'home/config'),
    'XDG_CACHE_HOME': str(repro_root / 'home/cache'),
    'XDG_DATA_HOME': str(repro_root / 'home/data'),
})
for variant in ['baseline', 'objective', 'title']:
    command = ['node', str(repro_root / 'repro.mjs'), variant]
    if variant != 'baseline':
        command.append(str(repro_root / 'baseline.json'))
    with (repro_root / f'{variant}.json').open('w') as output:
        subprocess.run(command, env=repro_environment, stdout=output, check=True)
PYTHON

Compare the returned objective, digests, and alignment state.

cat .repro-catalog/baseline.json .repro-catalog/objective.json .repro-catalog/title.json

The objective variant contains Objective-only verification marker. in the retrieved prompt but retains the baseline digests and reports aligned. The title variant changes both prompt-related digests and reports a catalog mismatch. The baseline reports unknown because it intentionally supplies no expectation. The tool digest stays unchanged in all three variants.

The verifier executed the three Node.js variants against prepared dependencies and the built baseline. The clean-checkout dependency installation and build steps above are setup instructions; the verifier did not rerun them during verification.

Expected behavior

An objective-only guided workflow change should change prompts_sha256 and combined_sha256. Supplying the previous combined digest should return mcp_server_restart_required with catalog_digest_mismatch.

docs/mcp-contracts.md:366-375 documents comparison of the live session with the caller's expected_catalog_digest and the catalog identity summary. It describes schema digests and does not explicitly promise a hash of every rendered prompt byte. The applicable public workflow contract declares objective in src/contracts/promptContracts.ts:28-36 and renders it at src/contracts/promptContracts.ts:78-80. The existing prompt projection already includes rendered workflow steps at src/catalogIdentity.ts:46-52. The objective should also participate in that guided workflow identity.

Impact and scope

This is a bounded catalog-comparison defect. An MCP caller cannot detect an objective-only guidance difference through the supported catalog expectation input and receives a false alignment result. The reproduction establishes no core binary-analysis failure, downstream agent harm, or crossed security boundary.

A caller can retrieve the prompt with the same arguments and compare its text with the known expected prompt. The reproduced responses expose the changed objective directly, so that comparison distinguishes these variants. No real analysis provider, deployed historical package pair, or real MCP client application was tested.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node.js v24.21.0; macOS arm64. The checkout declares Node.js v24.18.0, but the verifier did not use that exact version. The draft does not state a macOS release number.

Target and artifact context

No binary target, analysis engine, external service, or network server is needed. The fixture uses the production BinarySession, an empty AnalysisProviderRegistry, production createServer, and the installed MCP SDK's Client and InMemoryTransport. Each fresh process changes at most one field of the exported investigate_feature contract before server and catalog initialization.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

baseline:
  prompts_sha256: 5296f0301566cadaf89243e774512011f11045af36d48405f76a3ff9a8947ac2
  combined_sha256: e4eea9fc91fc4df1418c84cd9bcfd46bfe200494a8f5bac4fe1bcd58719553a9
  alignment.state: unknown

objective:
  objectiveChanged: true
  retrieved objective contains: Objective-only verification marker.
  prompts_sha256: 5296f0301566cadaf89243e774512011f11045af36d48405f76a3ff9a8947ac2
  combined_sha256: e4eea9fc91fc4df1418c84cd9bcfd46bfe200494a8f5bac4fe1bcd58719553a9
  alignment.state: aligned
  alignment.reasons: []
  alignment.remediation: null

title control:
  prompts_sha256: cbde01e0df40194cf8b54e8aa6ee9895d3c45ab209d464536686ecd6dc1d0f2f
  combined_sha256: 9df90512c3f7e79b2be56ba1d547f309023ae908d8d3e7d88b7e02bf5924e851
  alignment.state: mcp_server_restart_required
  alignment.reasons: [catalog_digest_mismatch]

All three Node.js processes exited 0. tools_sha256 stayed unchanged.

Typed outcome or recovery expected

For the objective variant, binary_session should return server_identity.alignment.state as mcp_server_restart_required, include catalog_digest_mismatch in its reasons, and provide the existing restart-and-compare remediation. An unchanged contract should retain the same identity. The title-only control should continue to detect a mismatch, and these prompt-only changes should preserve the tool digest.

Analysis or suspected cause

Source-derived at baseline 58300885207133422af235e0f2b10658cf958b03: src/catalogIdentity.ts:46-52 projects name, title, description, arguments, and steps into the prompt catalog but omits objective. Lines 72-77 use that projection for both prompts_sha256 and combined_sha256. src/serverIdentity.ts:41-45 compares the caller's expectation only with the combined digest.

The public path registers prompts through src/server/createServer.ts:570-572 and src/server/registerPrompts.ts:108-115, renders the objective at src/contracts/promptContracts.ts:80, and forwards the status-tool expectation at src/server/registerSessionStatusTool.ts:76-84. These source locators explain the reproduced false alignment. Fresh-process initialization and the title-only control distinguish this field omission from a cached-catalog refresh problem.

Additional context

Related reports are #2578 and #115. #2578 concerns input examples omitted from the tool digest, while #115 introduced stale-server identity reporting; this report concerns objective coverage in the separate prompt projection and requires an objective-only change to alter the prompt and combined digests.

The verifier assessed impact as P2 because the comparison returns an incorrect result for a bounded input difference and direct prompt-text comparison provides an available workaround.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3222 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area MCP or agent integration ### Observed behavior A freshly initialized REA MCP server reports `aligned` when the caller supplies a catalog digest from a guided workflow with a different objective. `prompts/get` returns the changed objective, but `prompts_sha256` and `combined_sha256` remain equal to the original catalog. `binary_session` returns an empty alignment-reason list and no remediation. The independent verifier reproduced this through the production server and the MCP SDK's in-memory transport. Each variant ran in a separate process and changed its contract before server and catalog construction. A title-only control changed both prompt-related digests and correctly returned `mcp_server_restart_required` with `catalog_digest_mismatch`. ### Reproduction 1. Start from a clean checkout of the verified revision and install its locked dependencies. Use the repository's declared Node.js and npm toolchain. ```sh git clone https://github.com/morluto/rea.git rea-catalog-repro cd rea-catalog-repro git checkout 58300885207133422af235e0f2b10658cf958b03 npm ci npm run build:cached mkdir -p .repro-catalog/home/config .repro-catalog/home/cache .repro-catalog/home/data ``` 2. Create the inline fixture. It starts the production server with an empty provider registry, retrieves `investigate_feature`, and calls `binary_session`. The objective and title variants model builds differing in one contract field without editing source or runtime files. ```sh cat > .repro-catalog/repro.mjs <<'JS' import assert from 'node:assert/strict'; import { readFile } from 'node:fs/promises'; import { createRequire } from 'node:module'; import { resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; const fromRoot = path => import(pathToFileURL(resolve(path)).href); const require = createRequire(resolve('package.json')); const { Client, InMemoryTransport } = await import(require.resolve('@modelcontextprotocol/client')); const variant = process.argv[2]; const expected = variant === 'baseline' ? undefined : JSON.parse(await readFile(process.argv[3], 'utf8')).digests.combined_sha256; const { PROMPT_CONTRACTS } = await fromRoot('dist/contracts/promptContracts.js'); const contract = PROMPT_CONTRACTS.find(prompt => prompt.name === 'investigate_feature'); const originalObjective = contract.objective; if (variant === 'objective') contract.objective += ' Objective-only verification marker.'; if (variant === 'title') contract.title += ' Title-only verification marker.'; const { createServer } = await fromRoot('dist/server/createServer.js'); const { BinarySession } = await fromRoot('dist/application/binary/BinarySession.js'); const { SessionProviderRouter } = await fromRoot('dist/application/binary/SessionProviderRouter.js'); const { AnalysisProviderRegistry } = await fromRoot('dist/application/binary/AnalysisProviderRegistry.js'); const session = new BinarySession(SessionProviderRouter.selectable(new AnalysisProviderRegistry([]), [])); const server = createServer({ kind: 'session', session }, { environment: {} }); const client = new Client({ name: 'catalog-verifier', version: '1' }); const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); try { await server.connect(serverTransport); await client.connect(clientTransport); const prompt = await client.getPrompt({ name: 'investigate_feature', arguments: { feature: 'offline search' } }); const text = prompt.messages[0].content.text; assert.ok(text.includes(`Objective: ${contract.objective}`)); const reply = await client.callTool({ name: 'binary_session', arguments: expected === undefined ? {} : { expected_catalog_digest: expected } }); assert.notEqual(reply.isError, true); const identity = reply.structuredContent.result.server_identity; console.log(JSON.stringify({ variant, objectiveChanged: contract.objective !== originalObjective, renderedObjective: text.split('\n').find(line => line.startsWith('Objective:')), digests: identity.catalog.digests, alignment: identity.alignment }, null, 2)); } finally { await client.close(); await server.close(); await session.close(); } JS ``` 3. Run three fresh Node.js processes with isolated configuration directories. The first saves the baseline digest; the other two supply it as `expected_catalog_digest`. ```sh python3 - <<'PYTHON' import os import subprocess from pathlib import Path repro_root = Path('.repro-catalog').resolve() repro_environment = dict(os.environ) repro_environment.update({ 'HOME': str(repro_root / 'home'), 'XDG_CONFIG_HOME': str(repro_root / 'home/config'), 'XDG_CACHE_HOME': str(repro_root / 'home/cache'), 'XDG_DATA_HOME': str(repro_root / 'home/data'), }) for variant in ['baseline', 'objective', 'title']: command = ['node', str(repro_root / 'repro.mjs'), variant] if variant != 'baseline': command.append(str(repro_root / 'baseline.json')) with (repro_root / f'{variant}.json').open('w') as output: subprocess.run(command, env=repro_environment, stdout=output, check=True) PYTHON ``` 4. Compare the returned objective, digests, and alignment state. ```sh cat .repro-catalog/baseline.json .repro-catalog/objective.json .repro-catalog/title.json ``` The objective variant contains `Objective-only verification marker.` in the retrieved prompt but retains the baseline digests and reports `aligned`. The title variant changes both prompt-related digests and reports a catalog mismatch. The baseline reports `unknown` because it intentionally supplies no expectation. The tool digest stays unchanged in all three variants. The verifier executed the three Node.js variants against prepared dependencies and the built baseline. The clean-checkout dependency installation and build steps above are setup instructions; the verifier did not rerun them during verification. ### Expected behavior An objective-only guided workflow change should change `prompts_sha256` and `combined_sha256`. Supplying the previous combined digest should return `mcp_server_restart_required` with `catalog_digest_mismatch`. `docs/mcp-contracts.md:366-375` documents comparison of the live session with the caller's `expected_catalog_digest` and the catalog identity summary. It describes schema digests and does not explicitly promise a hash of every rendered prompt byte. The applicable public workflow contract declares `objective` in `src/contracts/promptContracts.ts:28-36` and renders it at `src/contracts/promptContracts.ts:78-80`. The existing prompt projection already includes rendered workflow steps at `src/catalogIdentity.ts:46-52`. The objective should also participate in that guided workflow identity. ### Impact and scope This is a bounded catalog-comparison defect. An MCP caller cannot detect an objective-only guidance difference through the supported catalog expectation input and receives a false alignment result. The reproduction establishes no core binary-analysis failure, downstream agent harm, or crossed security boundary. A caller can retrieve the prompt with the same arguments and compare its text with the known expected prompt. The reproduced responses expose the changed objective directly, so that comparison distinguishes these variants. No real analysis provider, deployed historical package pair, or real MCP client application was tested. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node.js v24.21.0; macOS arm64. The checkout declares Node.js v24.18.0, but the verifier did not use that exact version. The draft does not state a macOS release number. ### Target and artifact context No binary target, analysis engine, external service, or network server is needed. The fixture uses the production `BinarySession`, an empty `AnalysisProviderRegistry`, production `createServer`, and the installed MCP SDK's `Client` and `InMemoryTransport`. Each fresh process changes at most one field of the exported `investigate_feature` contract before server and catalog initialization. ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence ```text baseline: prompts_sha256: 5296f0301566cadaf89243e774512011f11045af36d48405f76a3ff9a8947ac2 combined_sha256: e4eea9fc91fc4df1418c84cd9bcfd46bfe200494a8f5bac4fe1bcd58719553a9 alignment.state: unknown objective: objectiveChanged: true retrieved objective contains: Objective-only verification marker. prompts_sha256: 5296f0301566cadaf89243e774512011f11045af36d48405f76a3ff9a8947ac2 combined_sha256: e4eea9fc91fc4df1418c84cd9bcfd46bfe200494a8f5bac4fe1bcd58719553a9 alignment.state: aligned alignment.reasons: [] alignment.remediation: null title control: prompts_sha256: cbde01e0df40194cf8b54e8aa6ee9895d3c45ab209d464536686ecd6dc1d0f2f combined_sha256: 9df90512c3f7e79b2be56ba1d547f309023ae908d8d3e7d88b7e02bf5924e851 alignment.state: mcp_server_restart_required alignment.reasons: [catalog_digest_mismatch] All three Node.js processes exited 0. tools_sha256 stayed unchanged. ``` ### Typed outcome or recovery expected For the objective variant, `binary_session` should return `server_identity.alignment.state` as `mcp_server_restart_required`, include `catalog_digest_mismatch` in its reasons, and provide the existing restart-and-compare remediation. An unchanged contract should retain the same identity. The title-only control should continue to detect a mismatch, and these prompt-only changes should preserve the tool digest. ### Analysis or suspected cause Source-derived at baseline `58300885207133422af235e0f2b10658cf958b03`: `src/catalogIdentity.ts:46-52` projects `name`, `title`, `description`, `arguments`, and `steps` into the prompt catalog but omits `objective`. Lines 72-77 use that projection for both `prompts_sha256` and `combined_sha256`. `src/serverIdentity.ts:41-45` compares the caller's expectation only with the combined digest. The public path registers prompts through `src/server/createServer.ts:570-572` and `src/server/registerPrompts.ts:108-115`, renders the objective at `src/contracts/promptContracts.ts:80`, and forwards the status-tool expectation at `src/server/registerSessionStatusTool.ts:76-84`. These source locators explain the reproduced false alignment. Fresh-process initialization and the title-only control distinguish this field omission from a cached-catalog refresh problem. ### Additional context Related reports are #2578 and #115. #2578 concerns input examples omitted from the tool digest, while #115 introduced stale-server identity reporting; this report concerns objective coverage in the separate prompt projection and requires an objective-only change to alter the prompt and combined digests. The verifier assessed impact as P2 because the comparison returns an incorrect result for a bounded input difference and direct prompt-text comparison provides an available workaround.

bsakaguchi

2026-10-15 18:31:37 +00:00