morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5743] [Docs] Error catalogue marks removed capability-unknown tracking as implemented #3489

New issue

Closed

opened 2026-10-31 00:41:03 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-31 00:41:03 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5743

Before submitting

I searched the existing issues for a related documentation report.
I removed credentials, tokens, private paths, and sensitive target data.

Documentation location

docs/user-visible-errors.csv:28, harm_rank 24.5, location src/server/registerOfficialTools.ts:capability unknown.

Problem

The row describes Approved unknown tracking records an unavailable analysis, marks it implemented, and says verified by focused MCP integration test.

The current MCP test instead requires an unavailable analysis without supporting Evidence to leave the session unknown list empty. I ran that test on current main and it passed. Readers auditing supported recovery behavior therefore receive a stale implementation and verification claim.

This is P3 documentation accuracy. The passing unavailable-error case does not establish a missing runtime feature, lost Evidence, or incorrect analysis results.

Expected content

Describe the current supported behavior and what the focused test verifies. An unavailable analysis without supporting Evidence does not automatically create a session unknown. Mark the removed approval-based tracking path as historical or removed, or replace the entry with an accurate description of the current contract.

Keep the unavailable-error case and Evidence-backed unknown controls passing. This report requests a documentation correction, not restoration of automatic tracking or an approval input.

Supporting evidence

The independent verifier reproduced the discrepancy at source baseline 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Before submission I fetched origin/main at d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f. A cited contract file had changed, so I built that revision with npm run build:cached in a separate worktree and reran the following reproduction from its root:

work_dir="$(mktemp -d)"
mkdir -p "$work_dir"/{home,cfg,data,cache,state,tmp}
python3 - <<'CHECK'
import csv
import json
with open("docs/user-visible-errors.csv", newline="") as stream:
    row = next(r for r in csv.DictReader(stream) if r["harm_rank"] == "24.5")
print(json.dumps(row, indent=2))
CHECK
env -i PATH="$PATH" HOME="$work_dir/home" \
  XDG_CONFIG_HOME="$work_dir/cfg" XDG_DATA_HOME="$work_dir/data" \
  XDG_CACHE_HOME="$work_dir/cache" XDG_STATE_HOME="$work_dir/state" \
  TMPDIR="$work_dir/tmp" REA_LOG_LEVEL=silent \
  node scripts/run-exact-tests.mjs run \
  tests/boundary/mcp/serverUnknownsMcp.test.ts \
  --configLoader runner --cache false --maxWorkers 1

Observed on main: the row retains the trigger and completion metadata above; the test command exits 0 with one file and all three tests passing.

serverUnknownsMcp.test.ts:91-126 names the case does not record capability unavailability without supporting Evidence. It opens the Node executable through an in-memory fixture provider, invokes procedure_pseudo_code with { "procedure": "main" }, observes an error with category unavailable, and requires list_unknowns to return items: []. The capture-disagreement and Evidence-backed unknown controls also pass. No real decompiler was started.

Source-derived corroboration: registerEvidenceTools.ts:104-105 returns a failed execution before Evidence creation. The current pseudocode input has no approval field. Commit f55c2b92676e4a7eb06efbc8d9a908a7de62541a removed approval-based capability-unknown recording and changed the test to require an empty list. The catalogue retained its implemented claim.

Tested rea-agents 6.7.0 from source, Node.js v26.12.0, macOS 27.0.1 arm64. Verification covers the production MCP boundary with a fixture provider and session unknowns; it does not establish real-provider behavior or disk persistence.

Additional context

Prepared with AI assistance and independently verified. Readers can consult the current contract and focused MCP test for accurate behavior.

Related #5742 concerns the separate function-comparison probe-rationale row 24.93. Issues #5077 and #5473 concern process-input rows; #5078 and PR #5278 concern host-support rows, and PR #5519 concerns the process-input read diagnostic. Those reports and fixes do not cover the capability-unknown row.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5743 ### Before submitting - [x] I searched the existing issues for a related documentation report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Documentation location [`docs/user-visible-errors.csv:28`](https://github.com/morluto/rea/blob/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f/docs/user-visible-errors.csv#L28), `harm_rank` 24.5, location `src/server/registerOfficialTools.ts:capability unknown`. ### Problem The row describes `Approved unknown tracking records an unavailable analysis`, marks it `implemented`, and says `verified by focused MCP integration test`. The current MCP test instead requires an unavailable analysis without supporting Evidence to leave the session unknown list empty. I ran that test on current main and it passed. Readers auditing supported recovery behavior therefore receive a stale implementation and verification claim. This is P3 documentation accuracy. The passing unavailable-error case does not establish a missing runtime feature, lost Evidence, or incorrect analysis results. ### Expected content Describe the current supported behavior and what the focused test verifies. An unavailable analysis without supporting Evidence does not automatically create a session unknown. Mark the removed approval-based tracking path as historical or removed, or replace the entry with an accurate description of the current contract. Keep the unavailable-error case and Evidence-backed unknown controls passing. This report requests a documentation correction, not restoration of automatic tracking or an approval input. ### Supporting evidence The independent verifier reproduced the discrepancy at source baseline `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Before submission I fetched `origin/main` at `d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f`. A cited contract file had changed, so I built that revision with `npm run build:cached` in a separate worktree and reran the following reproduction from its root: ```sh work_dir="$(mktemp -d)" mkdir -p "$work_dir"/{home,cfg,data,cache,state,tmp} python3 - <<'CHECK' import csv import json with open("docs/user-visible-errors.csv", newline="") as stream: row = next(r for r in csv.DictReader(stream) if r["harm_rank"] == "24.5") print(json.dumps(row, indent=2)) CHECK env -i PATH="$PATH" HOME="$work_dir/home" \ XDG_CONFIG_HOME="$work_dir/cfg" XDG_DATA_HOME="$work_dir/data" \ XDG_CACHE_HOME="$work_dir/cache" XDG_STATE_HOME="$work_dir/state" \ TMPDIR="$work_dir/tmp" REA_LOG_LEVEL=silent \ node scripts/run-exact-tests.mjs run \ tests/boundary/mcp/serverUnknownsMcp.test.ts \ --configLoader runner --cache false --maxWorkers 1 ``` Observed on main: the row retains the trigger and completion metadata above; the test command exits 0 with one file and all three tests passing. [`serverUnknownsMcp.test.ts:91-126`](https://github.com/morluto/rea/blob/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f/tests/boundary/mcp/serverUnknownsMcp.test.ts#L91-L126) names the case `does not record capability unavailability without supporting Evidence`. It opens the Node executable through an in-memory fixture provider, invokes `procedure_pseudo_code` with `{ "procedure": "main" }`, observes an error with category `unavailable`, and requires `list_unknowns` to return `items: []`. The capture-disagreement and Evidence-backed unknown controls also pass. No real decompiler was started. Source-derived corroboration: [`registerEvidenceTools.ts:104-105`](https://github.com/morluto/rea/blob/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f/src/server/registerEvidenceTools.ts#L104-L105) returns a failed execution before Evidence creation. The current pseudocode input has no approval field. Commit `f55c2b92676e4a7eb06efbc8d9a908a7de62541a` removed approval-based capability-unknown recording and changed the test to require an empty list. The catalogue retained its implemented claim. Tested `rea-agents` 6.7.0 from source, Node.js v26.12.0, macOS 27.0.1 arm64. Verification covers the production MCP boundary with a fixture provider and session unknowns; it does not establish real-provider behavior or disk persistence. ### Additional context Prepared with AI assistance and independently verified. Readers can consult the current contract and focused MCP test for accurate behavior. Related #5742 concerns the separate function-comparison probe-rationale row 24.93. Issues #5077 and #5473 concern process-input rows; #5078 and PR #5278 concern host-support rows, and PR #5519 concerns the process-input read diagnostic. Those reports and fixes do not cover the capability-unknown row.

bsakaguchi closed this issue

2026-10-31 00:41:03 +00:00

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Confirmed docs bug — the CSV entry's "implemented + verified by test" claim is stale and directly contradicts what the focused MCP test actually asserts today (unavailable analysis without Evidence leaves the unknown list empty). For a recovery-behavior catalogue, a wrong implementation/verification claim is worse than a missing row because auditors will trust it. Agree the right fix is to rewrite the row to describe the current contract (no automatic session unknown without Evidence) and mark the approval-based tracking path as removed/historical; keep the unavailable-error case tests as-is.

@XinwangTao commented on GitHub (Oct 21, 2026): Confirmed docs bug — the CSV entry's "implemented + verified by test" claim is stale and directly contradicts what the focused MCP test actually asserts today (unavailable analysis without Evidence leaves the unknown list empty). For a recovery-behavior catalogue, a wrong implementation/verification claim is worse than a missing row because auditors will trust it. Agree the right fix is to rewrite the row to describe the current contract (no automatic session unknown without Evidence) and mark the approval-based tracking path as removed/historical; keep the unavailable-error case tests as-is.

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@Iyanuoluwa007 commented on GitHub (Oct 21, 2026):

Fix is ready: docs/error-catalogue-stale-rows (commit 8e2ad53cd). Row 24.5 is now marked removed (an existing catalogue status), and verification_result states what serverUnknownsMcp.test.ts actually checks: approval-based tracking was removed (f55c2b926), and an unavailable analysis without supporting Evidence leaves list_unknowns empty. The same branch fixes #5742, the other stale row in that file. I'll open the PR as soon as one of my open PR slots frees up (the 3-open-PR limit).

@Iyanuoluwa007 commented on GitHub (Oct 21, 2026): Fix is ready: [`docs/error-catalogue-stale-rows`](https://github.com/Iyanuoluwa007/rea/tree/docs/error-catalogue-stale-rows) (commit 8e2ad53cd). Row 24.5 is now marked `removed` (an existing catalogue status), and `verification_result` states what `serverUnknownsMcp.test.ts` actually checks: approval-based tracking was removed (f55c2b926), and an unavailable analysis without supporting Evidence leaves `list_unknowns` empty. The same branch fixes #5742, the other stale row in that file. I'll open the PR as soon as one of my open PR slots frees up (the 3-open-PR limit).

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

The fix in #6180 covers exactly what this report asked for: row 24.5 is now marked removed (an existing catalogue status), and verification_result states what serverUnknownsMcp.test.ts actually asserts — approval-based tracking was removed in f55c2b9 and an unavailable analysis without supporting Evidence leaves list_unknowns empty. Handling #5742 on the same branch is a sensible cleanup since both are stale rows in the same file. Looks good to merge.

@XinwangTao commented on GitHub (Oct 22, 2026): The fix in #6180 covers exactly what this report asked for: row 24.5 is now marked removed (an existing catalogue status), and verification_result states what serverUnknownsMcp.test.ts actually asserts — approval-based tracking was removed in f55c2b9 and an unavailable analysis without supporting Evidence leaves list_unknowns empty. Handling #5742 on the same branch is a sensible cleanup since both are stale rows in the same file. Looks good to merge.

bsakaguchi commented

2026-10-31 00:41:04 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The 24.5 CSV row now identifies an unavailable analysis without supporting Evidence, says approval-based tracking was removed, and explicitly says source/assertions were inspected rather than tests rerun for this documentation change. The linked MCP test still requires list_unknowns items: [] after unavailability. This meets the requested documentation correction without restoring automatic tracking.

Implementation: user-visible-errors.csv, serverUnknownsMcp.test.ts.

The relevant regression checks in serverUnknownsMcp.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The 24.5 CSV row now identifies an unavailable analysis without supporting Evidence, says approval-based tracking was removed, and explicitly says source/assertions were inspected rather than tests rerun for this documentation change. The linked MCP test still requires list_unknowns items: [] after unavailability. This meets the requested documentation correction without restoring automatic tracking. Implementation: [user-visible-errors.csv](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/docs/user-visible-errors.csv#L28), [serverUnknownsMcp.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/mcp/serverUnknownsMcp.test.ts#L92). The relevant regression checks in [serverUnknownsMcp.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/boundary/mcp/serverUnknownsMcp.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 00:44:30 +00:00

[GH-ISSUE #6403] [Feature] Link Android native methods to their JNI implementations (Java_* exports and RegisterNatives tables) #3836

bsakaguchi referenced this issue

2026-10-31 01:09:49 +00:00

[PR #3585] feat(android)!: expose method access flags and absent body reasons #5839

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471