[GH-ISSUE #5080] [Bug] find_changed_behavior description omits supported function comparison Evidence #3117

New issue

Closed

opened 2026-10-31 00:36:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:36:49 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5080

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

SOURCE-ONLY description mismatch: the canonical find_changed_behavior description enumerates process and artifact comparison Evidence but omits function comparison Evidence. The registered tool supports authenticated compare_functions Evidence, and its canonical example uses that input class.

A focused test also confirmed acceptance through the registered in-memory MCP boundary with a simulated session/provider. No real analysis provider or agent tool-selection behavior was exercised.

Reproduction

At source revision 3537e58c1ab29309f06eabf50bc52adba9817a0c:

Inspect the canonical description, which names process and artifact comparisons.
Inspect the canonical function-comparison input and its Evidence construction.
With locked dependencies available, run the existing focused test from the checkout root. The executed test was selected with the arguments below, under an isolated subprocess environment:

WORK=$(mktemp -d)
mkdir -p "$WORK/home" "$WORK/tmp" "$WORK/config" "$WORK/cache" "$WORK/data"
env -i PATH="$PATH" HOME="$WORK/home" TMPDIR="$WORK/tmp" \
  XDG_CONFIG_HOME="$WORK/config" XDG_CACHE_HOME="$WORK/cache" \
  XDG_DATA_HOME="$WORK/data" \
  node node_modules/vitest/vitest.mjs run --project=mcp-boundary \
  tests/boundary/mcp/investigationAnalysisMcp.test.ts \
  -t 'aggregates comparison Evidence and records an approved runtime gap' \
  --maxWorkers=1 --no-file-parallelism

The existing test retains both function observations and their comparison, calls find_changed_behavior with that comparison, and asserts successful changed-behavior Evidence. It passed in one selected run. The wording mismatch itself was established by source inspection, rather than a captured tools/list response.

Expected behavior

The advertised description should include function comparison Evidence alongside process and artifact comparisons. This matches the canonical example and the explicit supported operation set in the admission parser.

Function comparisons must continue to produce static candidates, separately from observed runtime changes.

Impact and scope

P3 wording defect in the agent-facing tool description. The description understates an accepted input class. Function comparison processing still works in the tested MCP boundary; no wrong computed result, data loss, blocked workflow, or actual agent avoidance was observed. Frequency and downstream tool-selection effects are NOT_ENOUGH_INFO.

REA version

Source revision 3537e58c1ab29309f06eabf50bc52adba9817a0c, package version 6.7.0.

Runtime and platform

Node v26.12.0; macOS 27.0.1; arm64; Vitest 5.0.3. This differs from the repository's Node 24.18.0 pin; pinned-runtime verification is NOT_ENOUGH_INFO. No native analysis engine was started.

Target and artifact context

Canonical synthetic function comparison Evidence and its retained source observations. No binary target is required for the description mismatch.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

Focused in-memory MCP test result:

Test Files  1 passed (1)
Tests       1 passed | 12 skipped (13)
Exit        0

The passing test verifies acceptance and successful Evidence delivery for the omitted input class. Its behavior_status: unknown assertion is expected because this fixture supplies static comparison Evidence without runtime observations.

Source-derived advertisement path: toolRegistrationOptions copies contract.description, and the registered workflow preserves those options and invokes findChangedBehavior. The workflow checks retained Evidence closure before delivery.

The function dispatcher and finding construction keep function changes under static_candidate.

Typed outcome or recovery expected

Keep the existing typed behavior. Include the supported function input class in the description while retaining the distinction between static candidates and runtime observations.

Analysis or suspected cause

SOURCE-ONLY: the description's enumeration is incomplete relative to the explicit operation allowlist, dispatch, and canonical example. The history or reason for the omission was not established.

Additional context

The relevant contract, example, domain implementation, registration options, and workflow registration files have no diff between rea-agents-6.7.0 and the tested main revision. This establishes source equality for those files; the published stable package was not run.

The complete open-and-closed issue snapshot was searched for the tool name, function comparisons, the quoted input enumeration, and description/catalog variants. Related comparison and investigation reports address different contracts; no matching description-omission report was found. Live issue freshness and real-client tool-selection consequences are NOT_ENOUGH_INFO.

Originally created by @coygeek on GitHub (Oct 18, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5080 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area MCP or agent integration ## Observed behavior SOURCE-ONLY description mismatch: the canonical `find_changed_behavior` description enumerates process and artifact comparison Evidence but omits function comparison Evidence. The registered tool supports authenticated `compare_functions` Evidence, and its canonical example uses that input class. A focused test also confirmed acceptance through the registered in-memory MCP boundary with a simulated session/provider. No real analysis provider or agent tool-selection behavior was exercised. ## Reproduction At source revision `3537e58c1ab29309f06eabf50bc52adba9817a0c`: 1. Inspect [the canonical description](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/contracts/sessionToolContracts.ts#L120-L124), which names process and artifact comparisons. 2. Inspect [the canonical function-comparison input](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/contracts/investigationExamples.ts#L92) and [its Evidence construction](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/contracts/investigationExamples.ts#L14-L37). 3. With locked dependencies available, run the existing focused test from the checkout root. The executed test was selected with the arguments below, under an isolated subprocess environment: ```sh WORK=$(mktemp -d) mkdir -p "$WORK/home" "$WORK/tmp" "$WORK/config" "$WORK/cache" "$WORK/data" env -i PATH="$PATH" HOME="$WORK/home" TMPDIR="$WORK/tmp" \ XDG_CONFIG_HOME="$WORK/config" XDG_CACHE_HOME="$WORK/cache" \ XDG_DATA_HOME="$WORK/data" \ node node_modules/vitest/vitest.mjs run --project=mcp-boundary \ tests/boundary/mcp/investigationAnalysisMcp.test.ts \ -t 'aggregates comparison Evidence and records an approved runtime gap' \ --maxWorkers=1 --no-file-parallelism ``` The [existing test](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/tests/boundary/mcp/investigationAnalysisMcp.test.ts#L33-L66) retains both function observations and their comparison, calls `find_changed_behavior` with that comparison, and asserts successful changed-behavior Evidence. It passed in one selected run. The wording mismatch itself was established by source inspection, rather than a captured `tools/list` response. ## Expected behavior The advertised description should include function comparison Evidence alongside process and artifact comparisons. This matches the canonical example and the explicit supported operation set in [the admission parser](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/domain/changedBehavior.ts#L203-L239). Function comparisons must continue to produce static candidates, separately from observed runtime changes. ## Impact and scope P3 wording defect in the agent-facing tool description. The description understates an accepted input class. Function comparison processing still works in the tested MCP boundary; no wrong computed result, data loss, blocked workflow, or actual agent avoidance was observed. Frequency and downstream tool-selection effects are NOT_ENOUGH_INFO. ## REA version Source revision `3537e58c1ab29309f06eabf50bc52adba9817a0c`, package version 6.7.0. ## Runtime and platform Node v26.12.0; macOS 27.0.1; arm64; Vitest 5.0.3. This differs from the repository's Node 24.18.0 pin; pinned-runtime verification is NOT_ENOUGH_INFO. No native analysis engine was started. ## Target and artifact context Canonical synthetic function comparison Evidence and its retained source observations. No binary target is required for the description mismatch. ## Interface MCP ## Earliest failure stage Transport or result projection ## Relevant output or evidence Focused in-memory MCP test result: ```text Test Files 1 passed (1) Tests 1 passed | 12 skipped (13) Exit 0 ``` The passing test verifies acceptance and successful Evidence delivery for the omitted input class. Its `behavior_status: unknown` assertion is expected because this fixture supplies static comparison Evidence without runtime observations. Source-derived advertisement path: [toolRegistrationOptions](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/server/toolRegistrationOptions.ts#L20-L33) copies `contract.description`, and [the registered workflow](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/server/registerInvestigationTools.ts#L65-L89) preserves those options and invokes `findChangedBehavior`. The workflow checks retained Evidence closure before delivery. The [function dispatcher and finding construction](https://github.com/morluto/rea/blob/3537e58c1ab29309f06eabf50bc52adba9817a0c/src/domain/changedBehavior.ts#L282-L347) keep function changes under `static_candidate`. ## Typed outcome or recovery expected Keep the existing typed behavior. Include the supported function input class in the description while retaining the distinction between static candidates and runtime observations. ## Analysis or suspected cause SOURCE-ONLY: the description's enumeration is incomplete relative to the explicit operation allowlist, dispatch, and canonical example. The history or reason for the omission was not established. ## Additional context The relevant contract, example, domain implementation, registration options, and workflow registration files have no diff between `rea-agents-6.7.0` and the tested main revision. This establishes source equality for those files; the published stable package was not run. The complete open-and-closed issue snapshot was searched for the tool name, function comparisons, the quoted input enumeration, and description/catalog variants. Related comparison and investigation reports address different contracts; no matching description-omission report was found. Live issue freshness and real-client tool-selection consequences are NOT_ENOUGH_INFO.

bsakaguchi closed this issue

2026-10-31 00:36:49 +00:00

bsakaguchi referenced this issue

2026-10-31 01:08:06 +00:00

[PR #3117] [CLOSED] fix: stabilize reference tests and patch development qs #5627