morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5742] [Docs] Function-comparison error catalogue marks unshipped probe wording as verified #3488

New issue

Closed

opened 2026-10-31 00:41:03 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-31 00:41:03 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5742

Before submitting

I searched the existing issues for a related documentation report.
I removed credentials, tokens, private paths, and sensitive target data.

Documentation location

docs/user-visible-errors.csv:35, harm_rank 24.93, location src/server/registerFunctionComparisonTool.ts:probe rationale, trigger Function comparison is incomplete.

Problem

The row records this proposed_replacement:

Capture complete dossiers for both functions under the same target context and analysis limits.

It marks the replacement implemented and verified by focused function-comparison MCP test. A real stdio MCP comparison with an unknown comments dimension instead exposes a rationale ending in same target context. through list_unknowns. The cited test asserts that shorter sentence too.

This misstates which wording is implemented and verified. The observed discrepancy is P3 documentation accuracy. It does not demonstrate incorrect comparison results or a retry failure caused by unequal analysis limits.

Expected content

Reconcile the row's replacement, implementation status, and verification description with the served rationale and the cited assertion. Correcting the catalogue or adopting and verifying the longer wording can resolve the discrepancy. Historical current_copy differences alone are not the reported defect.

Preserve the incomplete comparison's unknown comments dimension and the unchanged control's zero residual unknown items when checking a correction.

Supporting evidence

The independent verifier reproduced this through the built source checkout's stdio MCP server, and I repeated the same reproduction successfully. Each run used isolated HOME/XDG/TMPDIR storage and started node scripts/rea.mjs mcp; no binary target or analysis provider was opened.

Executed procedure:

Import createEvidence from dist/domain/evidence.js and FUNCTION_COMPARISON_EXAMPLE from dist/contracts/functionComparisonExample.js.
Clone the example's left normalized_result and set comments to [{address: '0x0', kind: 'comment', text: 'outside'}]. Build left/right Evidence using the example's subject path/format, provider, predicate type, operation, parameters, confidence, authority, limitations, and relationshipCoverage. Use SHA-256 values of 64 zeroes and 64 ones respectively. Preserving relationship coverage avoids introducing a second unknown dimension.
Initialize MCP with protocol 2025-11-25, call compare_functions with that left/right Evidence, then call list_unknowns with {"domain":"function-comparison"}.
In a fresh server, repeat with the unchanged example's left Evidence as both inputs.

Both the incomplete case and control returned, in order:

{"status":"unknown","unknown_dimensions":["comments"]}
{"total":1,"probes":[{"operation":"analyze_function","rationale":"Capture complete dossiers for both functions under the same target context."}]}
{"status":"unchanged","unknown_dimensions":[]}
{"total":0,"probes":[]}

The reproduction exited 0 with empty stderr. Source-derived corroboration: registerFunctionComparisonTool.ts:193-197 produces the shorter rationale; functionComparisonMcp.test.ts:215-218 asserts it. The test assertion was inspected; no test-suite execution is claimed.

Tested rea-agents 6.7.0 at source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68, Node.js v26.12.0, macOS 27.0.1 arm64. Before submission, I fetched origin/main at d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f. No intervening commits changed the catalogue, producer, or cited test. Main was checked through source history; no separate build or runtime execution of that later revision is claimed.

Additional context

Prepared with AI assistance and independently reproduced. Callers can inspect the actual recommendation through list_unknowns; maintainers can inspect the cited assertion when auditing implemented copy. No security impact or consequential comparison failure was established.

Related #5473 and PR #5519 concern process-input catalogue rows; #5078 and PR #5278 concern host-support rows. Their corrections do not cover this function-comparison row. The separate capability-unavailable catalogue entry at harm_rank 24.5 concerns removed unknown tracking and is outside this report's scope.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5742 ### Before submitting - [x] I searched the existing issues for a related documentation report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Documentation location [`docs/user-visible-errors.csv:35`](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/user-visible-errors.csv#L35), `harm_rank` 24.93, location `src/server/registerFunctionComparisonTool.ts:probe rationale`, trigger `Function comparison is incomplete`. ### Problem The row records this `proposed_replacement`: > Capture complete dossiers for both functions under the same target context and analysis limits. It marks the replacement `implemented` and `verified by focused function-comparison MCP test`. A real stdio MCP comparison with an unknown comments dimension instead exposes a rationale ending in `same target context.` through `list_unknowns`. The cited test asserts that shorter sentence too. This misstates which wording is implemented and verified. The observed discrepancy is P3 documentation accuracy. It does not demonstrate incorrect comparison results or a retry failure caused by unequal analysis limits. ### Expected content Reconcile the row's replacement, implementation status, and verification description with the served rationale and the cited assertion. Correcting the catalogue or adopting and verifying the longer wording can resolve the discrepancy. Historical `current_copy` differences alone are not the reported defect. Preserve the incomplete comparison's unknown comments dimension and the unchanged control's zero residual unknown items when checking a correction. ### Supporting evidence The independent verifier reproduced this through the built source checkout's stdio MCP server, and I repeated the same reproduction successfully. Each run used isolated HOME/XDG/TMPDIR storage and started `node scripts/rea.mjs mcp`; no binary target or analysis provider was opened. Executed procedure: 1. Import `createEvidence` from `dist/domain/evidence.js` and `FUNCTION_COMPARISON_EXAMPLE` from `dist/contracts/functionComparisonExample.js`. 2. Clone the example's left `normalized_result` and set `comments` to `[{address: '0x0', kind: 'comment', text: 'outside'}]`. Build left/right Evidence using the example's subject path/format, provider, predicate type, operation, parameters, confidence, authority, limitations, and `relationshipCoverage`. Use SHA-256 values of 64 zeroes and 64 ones respectively. Preserving relationship coverage avoids introducing a second unknown dimension. 3. Initialize MCP with protocol `2025-11-25`, call `compare_functions` with that left/right Evidence, then call `list_unknowns` with `{"domain":"function-comparison"}`. 4. In a fresh server, repeat with the unchanged example's left Evidence as both inputs. Both the incomplete case and control returned, in order: ```json {"status":"unknown","unknown_dimensions":["comments"]} {"total":1,"probes":[{"operation":"analyze_function","rationale":"Capture complete dossiers for both functions under the same target context."}]} {"status":"unchanged","unknown_dimensions":[]} {"total":0,"probes":[]} ``` The reproduction exited 0 with empty stderr. Source-derived corroboration: [`registerFunctionComparisonTool.ts:193-197`](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/server/registerFunctionComparisonTool.ts#L193-L197) produces the shorter rationale; [`functionComparisonMcp.test.ts:215-218`](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/tests/boundary/mcp/functionComparisonMcp.test.ts#L215-L218) asserts it. The test assertion was inspected; no test-suite execution is claimed. Tested `rea-agents` 6.7.0 at source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`, Node.js v26.12.0, macOS 27.0.1 arm64. Before submission, I fetched `origin/main` at `d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f`. No intervening commits changed the catalogue, producer, or cited test. Main was checked through source history; no separate build or runtime execution of that later revision is claimed. ### Additional context Prepared with AI assistance and independently reproduced. Callers can inspect the actual recommendation through `list_unknowns`; maintainers can inspect the cited assertion when auditing implemented copy. No security impact or consequential comparison failure was established. Related #5473 and PR #5519 concern process-input catalogue rows; #5078 and PR #5278 concern host-support rows. Their corrections do not cover this function-comparison row. The separate capability-unavailable catalogue entry at `harm_rank` 24.5 concerns removed unknown tracking and is outside this report's scope.

bsakaguchi closed this issue

2026-10-31 00:41:03 +00:00

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Confirmed — the catalogue row documents a longer proposed_replacement as implemented/verified, while both the served rationale and the cited test actually use the shorter "same target context" wording. Since this CSV is the recovery-behavior source of truth for auditors, the cheap correct fix is to align the row's replacement text with what's actually shipped (the shorter sentence) rather than adopting the longer wording, which would require re-verification. Either way the row should not claim the long wording is live.

@XinwangTao commented on GitHub (Oct 21, 2026): Confirmed — the catalogue row documents a longer proposed_replacement as implemented/verified, while both the served rationale and the cited test actually use the shorter "same target context" wording. Since this CSV is the recovery-behavior source of truth for auditors, the cheap correct fix is to align the row's replacement text with what's actually shipped (the shorter sentence) rather than adopting the longer wording, which would require re-verification. Either way the row should not claim the long wording is live.

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@Iyanuoluwa007 commented on GitHub (Oct 21, 2026):

Fix is ready: docs/error-catalogue-stale-rows (commit 52b4834bf). Row 24.93's proposed_replacement now records the shipped sentence, "Capture complete dossiers for both functions under the same target context.", which registerFunctionComparisonTool.ts:196 serves and functionComparisonMcp.test.ts:217 asserts. Status and verification stay as they are. The same branch fixes #5743. I'll open the PR as soon as one of my open PR slots frees up (the 3-open-PR limit).

@Iyanuoluwa007 commented on GitHub (Oct 21, 2026): Fix is ready: [`docs/error-catalogue-stale-rows`](https://github.com/Iyanuoluwa007/rea/tree/docs/error-catalogue-stale-rows) (commit 52b4834bf). Row 24.93's `proposed_replacement` now records the shipped sentence, "Capture complete dossiers for both functions under the same target context.", which `registerFunctionComparisonTool.ts:196` serves and `functionComparisonMcp.test.ts:217` asserts. Status and verification stay as they are. The same branch fixes #5743. I'll open the PR as soon as one of my open PR slots frees up (the 3-open-PR limit).

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

CSV row 24.93 now records exactly the shipped shorter rationale ending same target context. The production rationale and functionComparisonMcp assertion match.

Implementation: user-visible-errors.csv, registerFunctionComparisonTool.ts.

The relevant regression checks in functionComparisonMcp.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. CSV row 24.93 now records exactly the shipped shorter rationale ending same target context. The production rationale and functionComparisonMcp assertion match. Implementation: [user-visible-errors.csv](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/docs/user-visible-errors.csv#L35), [registerFunctionComparisonTool.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/server/registerFunctionComparisonTool.ts#L180). The relevant regression checks in [functionComparisonMcp.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/mcp/functionComparisonMcp.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:09:33 +00:00

[PR #3527] [CLOSED] fix(android): stop native and abstract methods from taking neighbouri… #5804

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3488

No description provided.

Rows
Columns