morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5507] [Bug] trace_feature promises an operation count absent from successful results #3353

New issue

Closed

opened 2026-10-31 00:39:28 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:39:28 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5507

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

The advertised trace_feature description promises an operation count, but its output schema and successful responses contain no count. An independent verifier executed real Hopper-backed MCP calls with {"query":"needle"} and {"query":"no_matching_literal_control_8319"}. Both returned the same six normalized-result keys without an operation count. The matching case returned five matches and eight reference rows; the no-match control returned none. Neither Evidence envelope exposed a count elsewhere.

Reproduction

The independent verifier compiled this fixture on macOS arm64:

cat > trace_fixture.c <<'C'
#include <stdio.h>
const char *needle_one = "needle feature alpha";
const char *needle_two = "needle feature beta";
static const char *dead_needle = "needle feature unreachable";
int needle_function(int x) { return x + 42; }
int unused_needle_function(int x) { return x + dead_needle[0]; }
int main(void) { printf("%s %s %d\n", needle_one, needle_two, needle_function(1)); return 0; }
C
clang -O0 -g -o trace_fixture trace_fixture.c

From the built source checkout, start node scripts/rea.mjs mcp through an MCP client. Use isolated HOME, XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_CACHE_HOME, XDG_STATE_HOME and TMPDIR directories, with the installed Hopper launcher.
Read listTools and inspect the trace_feature description and output schema.
Call open_binary with {"path":"<absolute path to trace_fixture>","provider_id":"hopper"}.
Call trace_feature with {"query":"needle"}, then with {"query":"no_matching_literal_control_8319"}. Inspect the complete Evidence and structuredContent.normalized_result.
Call close_binary with {}, then binary_session. The verified cleanup reported open: false and cleanup_pending: false; Hopper remained running.

The verifier ran this sequence with Node.js 26.12.0, then repeated it with the repository-pinned Node.js 24.18.0. Both sequences exited successfully and reproduced the omission. No CLI trace command was independently executed.

Expected behavior

The running catalog's description says the tool returns "observations, operation count, and residual unknowns", matching src/contracts/enhancedToolContracts.ts:70-72 at the tested revision. docs/mcp-contracts.md, under Tool results, identifies structuredContent.normalized_result as the operation result.

The description, schema and successful result should agree about operation-count availability. The current description does not define what an operation means. Either remove the unsupported promise or return a count with a defined, truthful meaning.

Impact and scope

This is a P3 wording/result-contract mismatch. Callers cannot obtain the advertised count from the tested responses. No consequential trace-analysis error, resource-accounting impact or security impact was established. Both the matching and no-match cases reproduced the omission in both runtime runs. Other providers were not tested. Deriving a count from result rows would require choosing an undocumented meaning of operation and would not measure provider-internal work.

REA version

6.7.0; source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8, built before verification.

Runtime and platform

Node.js 24.18.0, with an initial run on 26.12.0; macOS arm64; Hopper Disassembler 6.1.0-demo, build 42.6295552.

Target and artifact context

Freshly compiled native C executable from the fixture above, analyzed through the real Hopper provider.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

The verifier captured these exact keys from each successful structuredContent.normalized_result:

["query", "search_mode", "matches", "references", "truncated", "residual_unknowns"]

For needle, matches.length was 5 and references.length was 8. For the no-match control, both lengths were 0. Both results had truncated: false and residual_unknowns: []. Evidence parameters contained only query and case_sensitive; raw_result was null. The advertised output schema had no count member.

The existing focused MCP trace-feature coverage file passed all 10 tests through node scripts/run-exact-tests.mjs tests/boundary/mcp/enhancedTools.traceFeatureCoverage.test.ts under Node.js 24.18.0 using the already-built baseline. These tests establish current behavior, not restoration of the advertised count.

Typed outcome or recovery expected

Keep the trace observations and cleanup behavior while making the catalog, schema and successful result agree. A proposed restoration check is to repeat both MCP calls and inspect the complete Evidence. If the count promise remains, define its unit and verify it against recorded work for both cases; a constant count alone would not establish restoration.

Analysis or suspected cause

Source-derived corroboration: src/application/EnhancedLiteralTracing.ts, traceLiteralFeature, returns the six fields above; src/contracts/toolOutputSchemaGroups.ts:233-254, literalTraceOutput, declares them without a count. src/server/registerEnhancedTools.ts, executeEnhancedTool, wraps the composed result as Evidence without adding a count.

The upstream check fetched origin/main at c374c2fe7290b914d73d9e27af38b498753ec593. The advertised description and tracing implementation were unchanged from the tested revision. Later edits to the cited documentation, output-schema file and registration file concern other behavior and do not add a trace count. No later runtime reproduction is claimed.

Additional context

Prepared with AI assistance and independently reproduced by a separate verifier through the real Hopper MCP provider.

Related: #5503 reports dropped string encoding and termination qualifications in trace_feature. It has a separate restoration condition and is not a duplicate of this missing advertised operation count. This report does not assert that every trace-analysis fact is correct.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5507 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area MCP or agent integration ### Observed behavior The advertised `trace_feature` description promises an operation count, but its output schema and successful responses contain no count. An independent verifier executed real Hopper-backed MCP calls with `{"query":"needle"}` and `{"query":"no_matching_literal_control_8319"}`. Both returned the same six normalized-result keys without an operation count. The matching case returned five matches and eight reference rows; the no-match control returned none. Neither Evidence envelope exposed a count elsewhere. ### Reproduction The independent verifier compiled this fixture on macOS arm64: ```sh cat > trace_fixture.c <<'C' #include <stdio.h> const char *needle_one = "needle feature alpha"; const char *needle_two = "needle feature beta"; static const char *dead_needle = "needle feature unreachable"; int needle_function(int x) { return x + 42; } int unused_needle_function(int x) { return x + dead_needle[0]; } int main(void) { printf("%s %s %d\n", needle_one, needle_two, needle_function(1)); return 0; } C clang -O0 -g -o trace_fixture trace_fixture.c ``` 1. From the built source checkout, start `node scripts/rea.mjs mcp` through an MCP client. Use isolated `HOME`, `XDG_CONFIG_HOME`, `XDG_DATA_HOME`, `XDG_CACHE_HOME`, `XDG_STATE_HOME` and `TMPDIR` directories, with the installed Hopper launcher. 2. Read `listTools` and inspect the `trace_feature` description and output schema. 3. Call `open_binary` with `{"path":"<absolute path to trace_fixture>","provider_id":"hopper"}`. 4. Call `trace_feature` with `{"query":"needle"}`, then with `{"query":"no_matching_literal_control_8319"}`. Inspect the complete Evidence and `structuredContent.normalized_result`. 5. Call `close_binary` with `{}`, then `binary_session`. The verified cleanup reported `open: false` and `cleanup_pending: false`; Hopper remained running. The verifier ran this sequence with Node.js 26.12.0, then repeated it with the repository-pinned Node.js 24.18.0. Both sequences exited successfully and reproduced the omission. No CLI trace command was independently executed. ### Expected behavior The running catalog's description says the tool returns "observations, operation count, and residual unknowns", matching `src/contracts/enhancedToolContracts.ts:70-72` at the tested revision. `docs/mcp-contracts.md`, under Tool results, identifies `structuredContent.normalized_result` as the operation result. The description, schema and successful result should agree about operation-count availability. The current description does not define what an operation means. Either remove the unsupported promise or return a count with a defined, truthful meaning. ### Impact and scope This is a P3 wording/result-contract mismatch. Callers cannot obtain the advertised count from the tested responses. No consequential trace-analysis error, resource-accounting impact or security impact was established. Both the matching and no-match cases reproduced the omission in both runtime runs. Other providers were not tested. Deriving a count from result rows would require choosing an undocumented meaning of operation and would not measure provider-internal work. ### REA version 6.7.0; source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8`, built before verification. ### Runtime and platform Node.js 24.18.0, with an initial run on 26.12.0; macOS arm64; Hopper Disassembler 6.1.0-demo, build 42.6295552. ### Target and artifact context Freshly compiled native C executable from the fixture above, analyzed through the real Hopper provider. ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence The verifier captured these exact keys from each successful `structuredContent.normalized_result`: ```json ["query", "search_mode", "matches", "references", "truncated", "residual_unknowns"] ``` For `needle`, `matches.length` was 5 and `references.length` was 8. For the no-match control, both lengths were 0. Both results had `truncated: false` and `residual_unknowns: []`. Evidence `parameters` contained only `query` and `case_sensitive`; `raw_result` was null. The advertised output schema had no count member. The existing focused MCP trace-feature coverage file passed all 10 tests through `node scripts/run-exact-tests.mjs tests/boundary/mcp/enhancedTools.traceFeatureCoverage.test.ts` under Node.js 24.18.0 using the already-built baseline. These tests establish current behavior, not restoration of the advertised count. ### Typed outcome or recovery expected Keep the trace observations and cleanup behavior while making the catalog, schema and successful result agree. A proposed restoration check is to repeat both MCP calls and inspect the complete Evidence. If the count promise remains, define its unit and verify it against recorded work for both cases; a constant count alone would not establish restoration. ### Analysis or suspected cause Source-derived corroboration: `src/application/EnhancedLiteralTracing.ts`, `traceLiteralFeature`, returns the six fields above; `src/contracts/toolOutputSchemaGroups.ts:233-254`, `literalTraceOutput`, declares them without a count. `src/server/registerEnhancedTools.ts`, `executeEnhancedTool`, wraps the composed result as Evidence without adding a count. The upstream check fetched `origin/main` at `c374c2fe7290b914d73d9e27af38b498753ec593`. The advertised description and tracing implementation were unchanged from the tested revision. Later edits to the cited documentation, output-schema file and registration file concern other behavior and do not add a trace count. No later runtime reproduction is claimed. ### Additional context Prepared with AI assistance and independently reproduced by a separate verifier through the real Hopper MCP provider. Related: #5503 reports dropped string encoding and termination qualifications in `trace_feature`. It has a separate restoration condition and is not a duplicate of this missing advertised operation count. This report does not assert that every trace-analysis fact is correct.

bsakaguchi closed this issue

2026-10-31 00:39:28 +00:00

bsakaguchi commented

2026-10-31 00:39:28 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with sibling feature tracing reports in Cluster AM: #5449 ([Bug] trace-application-feature reports changed Evidence as a retryable input error), #5474, and #5503. All isolate contract discrepancies, string qualification omissions, and result schema gaps in trace_feature and trace_crash.

Consolidated under Cluster AM (Trace Feature Contract & Crash Diagnostic Inconsistencies) in tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with sibling feature tracing reports in **Cluster AM**: #5449 (`[Bug] trace-application-feature reports changed Evidence as a retryable input error`), #5474, and #5503. All isolate contract discrepancies, string qualification omissions, and result schema gaps in `trace_feature` and `trace_crash`. Consolidated under **Cluster AM (Trace Feature Contract & Crash Diagnostic Inconsistencies)** in tracker #4330.

bsakaguchi commented

2026-10-31 00:39:29 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The exact mismatch is repaired by the issue’s explicitly permitted option of removing the unsupported promise: canonical trace_feature description now says it returns observations and residual unknowns, without promising operation count. Implementation and schema continue to return query/search_mode/matches/references/truncated/residual_unknowns. Existing trace coverage tests assert exact result shape for complete and capped responses; no dedicated assertion guarding absence of the operation-count phrase was found.

Implementation: enhancedToolContracts.ts, EnhancedLiteralTracing.ts.

The relevant regression checks in enhancedTools.traceFeatureCoverage.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The exact mismatch is repaired by the issue’s explicitly permitted option of removing the unsupported promise: canonical trace_feature description now says it returns observations and residual unknowns, without promising operation count. Implementation and schema continue to return query/search_mode/matches/references/truncated/residual_unknowns. Existing trace coverage tests assert exact result shape for complete and capped responses; no dedicated assertion guarding absence of the operation-count phrase was found. Implementation: [enhancedToolContracts.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/contracts/enhancedToolContracts.ts#L76), [EnhancedLiteralTracing.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/EnhancedLiteralTracing.ts#L71). The relevant regression checks in [enhancedTools.traceFeatureCoverage.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/mcp/enhancedTools.traceFeatureCoverage.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:09:26 +00:00

[PR #3504] [CLOSED] fix(apple): classify launchd plists by owning directory #5787

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3353

No description provided.

Rows
Columns