morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6300] [Bug] Ledger reports missing residual-unknown Evidence as unreadable_output #3807

New issue

Closed

opened 2026-10-31 00:44:15 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:44:15 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 22, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6300

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

build_reconstruction_obligation_ledger misclassifies a caller bundle that omits a residual unknown's mutation Evidence record. The registered MCP tool returns unreadable_output / execution_failure, tells the caller to run rea doctor, and omits the missing-reference reason. The valid bundle succeeds, and a duplicate-record control receives the existing invalid_request / invalid_input diagnostic.

Reproduction

In a prepared checkout at the stated baseline, save the following as tests/boundary/mcp/residualGraphLedger.test.ts and run the command below. The verifier used this exact test content under a temporary filename, with HOME, DSH_HOME, XDG_CONFIG_HOME and XDG_CACHE_HOME directed to disposable directories. No runtime build or dependency installation was needed. It uses the registered production MCP handler and SDK in-memory transport; the session's inert provider is never invoked. Assertions describe the faulty baseline behavior.

node node_modules/vitest/vitest.mjs run tests/boundary/mcp/residualGraphLedger.test.ts --maxWorkers=1

import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { expect, it } from "vitest";
import { createEvidence } from "../../../src/domain/evidence.js";
import { createEvidenceBundle, parseEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
import { createResidualUnknown, recordUnknownInputSchema } from "../../../src/domain/residualUnknown.js";
import { resolveReconstructionObligationLedgerRequest } from "../../../src/application/ReconstructionObligationLedgerService.js";
import { createServer } from "../../../src/server/createServer.js";
import { ok } from "../../../src/domain/result.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { observed } from "../../fixtures/analysisExecution.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";

it("observes residual-graph caller input classification with a valid control", async () => {
  const input = recordUnknownInputSchema.parse({
    question: "Does the synthetic fixture retain state?", severity: "low", domain: "persistence",
    required_authority: null, required_confidence: "observed", required_environment: null,
    recommended_probes: [], relationships: [],
  });
  const mutation = createEvidence(undefined, { id: "fixture", name: "Fixture", version: "1" }, {
    predicateType: "rea.residual-unknown-mutation", operation: "record_unknown",
    parameters: { domain: input.domain, severity: input.severity },
    result: { action: "record", question: input.question },
  });
  const unknown = createResidualUnknown(input, mutation.evidence_id, null);
  const valid = createEvidenceBundle([mutation], [unknown]);
  const missing = createEvidenceBundle([], [unknown]);
  expect(parseEvidenceBundle(valid)).toEqual(valid);
  const expectedReason = `Residual unknown references missing evidence ${mutation.evidence_id}`;
  expect(() => parseEvidenceBundle(missing)).toThrow(expectedReason);
  const args = (bundle: unknown) => ({ evidence_bundle: bundle, reviewed_obligations: [], manifest: { bindings: [], contradictions: [] } });
  expect(resolveReconstructionObligationLedgerRequest(args(missing)).ok).toBe(true);
  const session = createTestBinarySession(() => ({ execute: () => Promise.resolve(observed(null)), close: () => Promise.resolve(ok(null)) }));
  const server = createServer({ kind: "session", session });
  const client = new Client({ name: "residual-graph-repro", version: "1" });
  const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair();
  try {
    await server.connect(serverTransport);
    await client.connect(clientTransport);
    const request = (bundle: unknown) => client.callTool({ name: "build_reconstruction_obligation_ledger", arguments: args(bundle) });
    const startValid = performance.now();
    const accepted = await request(valid);
    const validMs = performance.now() - startValid;
    expect(accepted.isError).not.toBe(true);
    expect(accepted.structuredContent).toMatchObject({ normalized_result: { status: "unknown", summary: { total: 0 } } });
    const startMissing = performance.now();
    const rejected = await request(missing);
    const missingMs = performance.now() - startMissing;
    const diagnostic = parseMcpToolError(rejected);
    console.log(JSON.stringify({ validMs, missingMs, expectedReason, diagnostic }));
    expect(diagnostic).toMatchObject({ error: { code: "unreadable_output", category: "execution_failure", retryable: false } });
    expect(diagnostic.error.details).toBeUndefined();
    const duplicate = await request({ ...valid, records: [mutation, mutation] });
    expect(parseMcpToolError(duplicate)).toMatchObject({ error: { code: "invalid_request", category: "invalid_input", details: { issues: [expect.objectContaining({ message: "Evidence bundle contains duplicate record IDs" })] } } });
    const recovered = await request(valid);
    expect(recovered.isError).not.toBe(true);
  } finally {
    await client.close();
    await server.close();
    await session.close();
  }
}, 30_000);

Two independent focused runs reproduced the missing-reference failure twice. Both runs accepted the complete bundle, classified the duplicate-record control as caller input, and accepted the complete bundle again after the rejected calls.

Expected behavior

Reject the bundle as invalid_request / invalid_input, retain the missing Evidence ID and failed reference constraint in details.issues, and direct the caller to correct or re-export the bundle. Keep rejection and Evidence authentication intact. A bundle containing the required authenticated record must still succeed. Preserve genuine unexpected internal failures as internal errors; this report does not ask for blanket TypeError remapping.

The documented recovery contract directs correction of invalid input. AnalysisInputError represents application-boundary caller parsing; workflow input diagnostics preserve late validation reasons. The executed duplicate-record control confirms that semantically invalid caller bundles receive this classification after structural parsing.

Impact and scope

P2. The machine-readable category and correction guidance are wrong for this bounded malformed bundle, and the actionable missing Evidence ID is hidden. The failure reproduced in both focused runs. Restoring the authenticated record and consistent manifests succeeded on the same connection, preserving the residual unknown. No data loss, general outage, or security-boundary crossing was demonstrated. Runtime evidence here covers one missing mutation Evidence reference over registered in-memory MCP. Other residual graph constraints and built CLI/stdio parity were not independently executed.

REA version

Source commit 4a4bbab3c3 (main)

Runtime and platform

Node.js v26.12.1; macOS 27.0.1 arm64; Vitest 5.0.3. The repository pins Node 24.18.0, which was not exercised. No Hopper, Ghidra, or other external engine was required.

Target and artifact context

A local synthetic Evidence bundle containing one revision-1 residual unknown and its authenticated mutation record. The failing variant omits that record and consistently rebuilds the bundle manifests, leaving the unknown's mutation_evidence_ids reference unchanged. Reviewed obligations and manifest bindings/contradictions are empty.

Interface

MCP

Earliest failure stage

Request parsing or validation

Relevant output or evidence

The public MCP failure in both runs was:

{"error":{"code":"unreadable_output","category":"execution_failure","message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","retryable":false,"remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}}}

The direct bundle-validation control reports Residual unknown references missing evidence ev_d9f120b4351b2335532d293757c39547638b5a17419a63e587ad4416acfc3cdd; structural ledger request validation succeeds. The valid request returns normalized_result.status: "unknown", summary.total: 0. The duplicate-record request returns invalid_request / invalid_input with Evidence bundle contains duplicate record IDs in details.issues. The valid request succeeds again after both failures.

Each focused run passed its one test because the assertions record current faulty behavior. Run 1: 4.26 s real, 4.25 s user, 0.82 s system; maximum resident set size 713818112 bytes. Run 2: 4.08 s real, 4.43 s user, 0.87 s system; maximum resident set size 661684224 bytes. These are command measurements from /usr/bin/time -l, not sampled whole-family peak memory. Valid-call spans: 7.833999999999833 ms and 8.626291999999921 ms; missing-reference spans: 1.2909170000002632 ms and 1.8202919999998812 ms. No agent-token or production-performance claim is made.

Typed outcome or recovery expected

A non-retryable invalid_request / invalid_input outcome naming the absent Evidence reference in details.issues and directing input correction or bundle re-export. Keep malformed bundles rejected and complete bundles accepted. Genuine internal failures must retain their internal-error outcome.

Analysis or suspected cause

Source-derived: candidate derivation authenticates the supplied bundle. Graph validation throws a plain TypeError for the missing reference. The ledger catch recognizes three named caller-input errors and ZodError; this failure falls through to AnalysisProtocolError. This explains the public result without establishing that every TypeError is caller input.

Additional context

This is the residual-graph follow-up explicitly excluded from merged #4452. Closed #4444 reproduced duplicate records and missing reviewed Evidence; its residual-graph mention was explicitly untested. Closed unmerged #4481 retains the exact residual-graph concern in its review, then closed because the two earlier fixes landed first. Those completed cases are controls here, rather than the reported defect.

Current open #6106 covers cross-field record_unknown / update_unknown validation being wrapped as a changed-state error, with separate producers and closure checks. Searches including validateUnknownGraph, ledger unreadable_output, "Residual unknown references missing evidence", "residual unknown" "unreadable_output", "mutation_evidence_ids" and "4444" found no exact current ledger residual-reference follow-up. This is a bounded search result, not an exhaustive proof that no matching report exists.

Fetched main 66aacc6fba7fb498302ca58894f179d77a847444 was inspected without checkout or execution. The bundle graph validator and ledger classification service are unchanged. Intervening changes replace integer-validator construction, share ordering/digest helpers, and add unrelated ASAR diagnostic details; none changes this reference check or classification branch. Persistence on that later revision is source-derived. No regression range was established.

Originally created by @coygeek on GitHub (Oct 22, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6300 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `build_reconstruction_obligation_ledger` misclassifies a caller bundle that omits a residual unknown's mutation Evidence record. The registered MCP tool returns `unreadable_output` / `execution_failure`, tells the caller to run `rea doctor`, and omits the missing-reference reason. The valid bundle succeeds, and a duplicate-record control receives the existing `invalid_request` / `invalid_input` diagnostic. ### Reproduction In a prepared checkout at the stated baseline, save the following as `tests/boundary/mcp/residualGraphLedger.test.ts` and run the command below. The verifier used this exact test content under a temporary filename, with HOME, DSH_HOME, XDG_CONFIG_HOME and XDG_CACHE_HOME directed to disposable directories. No runtime build or dependency installation was needed. It uses the registered production MCP handler and SDK in-memory transport; the session's inert provider is never invoked. Assertions describe the faulty baseline behavior. ```sh node node_modules/vitest/vitest.mjs run tests/boundary/mcp/residualGraphLedger.test.ts --maxWorkers=1 ``` ```ts import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { expect, it } from "vitest"; import { createEvidence } from "../../../src/domain/evidence.js"; import { createEvidenceBundle, parseEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; import { createResidualUnknown, recordUnknownInputSchema } from "../../../src/domain/residualUnknown.js"; import { resolveReconstructionObligationLedgerRequest } from "../../../src/application/ReconstructionObligationLedgerService.js"; import { createServer } from "../../../src/server/createServer.js"; import { ok } from "../../../src/domain/result.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { observed } from "../../fixtures/analysisExecution.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; it("observes residual-graph caller input classification with a valid control", async () => { const input = recordUnknownInputSchema.parse({ question: "Does the synthetic fixture retain state?", severity: "low", domain: "persistence", required_authority: null, required_confidence: "observed", required_environment: null, recommended_probes: [], relationships: [], }); const mutation = createEvidence(undefined, { id: "fixture", name: "Fixture", version: "1" }, { predicateType: "rea.residual-unknown-mutation", operation: "record_unknown", parameters: { domain: input.domain, severity: input.severity }, result: { action: "record", question: input.question }, }); const unknown = createResidualUnknown(input, mutation.evidence_id, null); const valid = createEvidenceBundle([mutation], [unknown]); const missing = createEvidenceBundle([], [unknown]); expect(parseEvidenceBundle(valid)).toEqual(valid); const expectedReason = `Residual unknown references missing evidence ${mutation.evidence_id}`; expect(() => parseEvidenceBundle(missing)).toThrow(expectedReason); const args = (bundle: unknown) => ({ evidence_bundle: bundle, reviewed_obligations: [], manifest: { bindings: [], contradictions: [] } }); expect(resolveReconstructionObligationLedgerRequest(args(missing)).ok).toBe(true); const session = createTestBinarySession(() => ({ execute: () => Promise.resolve(observed(null)), close: () => Promise.resolve(ok(null)) })); const server = createServer({ kind: "session", session }); const client = new Client({ name: "residual-graph-repro", version: "1" }); const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); try { await server.connect(serverTransport); await client.connect(clientTransport); const request = (bundle: unknown) => client.callTool({ name: "build_reconstruction_obligation_ledger", arguments: args(bundle) }); const startValid = performance.now(); const accepted = await request(valid); const validMs = performance.now() - startValid; expect(accepted.isError).not.toBe(true); expect(accepted.structuredContent).toMatchObject({ normalized_result: { status: "unknown", summary: { total: 0 } } }); const startMissing = performance.now(); const rejected = await request(missing); const missingMs = performance.now() - startMissing; const diagnostic = parseMcpToolError(rejected); console.log(JSON.stringify({ validMs, missingMs, expectedReason, diagnostic })); expect(diagnostic).toMatchObject({ error: { code: "unreadable_output", category: "execution_failure", retryable: false } }); expect(diagnostic.error.details).toBeUndefined(); const duplicate = await request({ ...valid, records: [mutation, mutation] }); expect(parseMcpToolError(duplicate)).toMatchObject({ error: { code: "invalid_request", category: "invalid_input", details: { issues: [expect.objectContaining({ message: "Evidence bundle contains duplicate record IDs" })] } } }); const recovered = await request(valid); expect(recovered.isError).not.toBe(true); } finally { await client.close(); await server.close(); await session.close(); } }, 30_000); ``` Two independent focused runs reproduced the missing-reference failure twice. Both runs accepted the complete bundle, classified the duplicate-record control as caller input, and accepted the complete bundle again after the rejected calls. ### Expected behavior Reject the bundle as `invalid_request` / `invalid_input`, retain the missing Evidence ID and failed reference constraint in `details.issues`, and direct the caller to correct or re-export the bundle. Keep rejection and Evidence authentication intact. A bundle containing the required authenticated record must still succeed. Preserve genuine unexpected internal failures as internal errors; this report does not ask for blanket `TypeError` remapping. The [documented recovery contract](https://github.com/morluto/rea/blob/4a4bbab3c337701febeb6b532ace49041c4876e0/docs/mcp-contracts.md#L246-L253) directs correction of invalid input. [AnalysisInputError](https://github.com/morluto/rea/blob/4a4bbab3c337701febeb6b532ace49041c4876e0/src/domain/analysisErrorCore.ts#L107-L119) represents application-boundary caller parsing; [workflow input diagnostics](https://github.com/morluto/rea/blob/4a4bbab3c337701febeb6b532ace49041c4876e0/src/application/workflowInputError.ts#L15-L34) preserve late validation reasons. The executed duplicate-record control confirms that semantically invalid caller bundles receive this classification after structural parsing. ### Impact and scope P2. The machine-readable category and correction guidance are wrong for this bounded malformed bundle, and the actionable missing Evidence ID is hidden. The failure reproduced in both focused runs. Restoring the authenticated record and consistent manifests succeeded on the same connection, preserving the residual unknown. No data loss, general outage, or security-boundary crossing was demonstrated. Runtime evidence here covers one missing mutation Evidence reference over registered in-memory MCP. Other residual graph constraints and built CLI/stdio parity were not independently executed. ### REA version Source commit 4a4bbab3c337701febeb6b532ace49041c4876e0 (main) ### Runtime and platform Node.js v26.12.1; macOS 27.0.1 arm64; Vitest 5.0.3. The repository pins Node 24.18.0, which was not exercised. No Hopper, Ghidra, or other external engine was required. ### Target and artifact context A local synthetic Evidence bundle containing one revision-1 residual unknown and its authenticated mutation record. The failing variant omits that record and consistently rebuilds the bundle manifests, leaving the unknown's `mutation_evidence_ids` reference unchanged. Reviewed obligations and manifest bindings/contradictions are empty. ### Interface MCP ### Earliest failure stage Request parsing or validation ### Relevant output or evidence The public MCP failure in both runs was: ```json {"error":{"code":"unreadable_output","category":"execution_failure","message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","retryable":false,"remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}}} ``` The direct bundle-validation control reports `Residual unknown references missing evidence ev_d9f120b4351b2335532d293757c39547638b5a17419a63e587ad4416acfc3cdd`; structural ledger request validation succeeds. The valid request returns `normalized_result.status: "unknown"`, `summary.total: 0`. The duplicate-record request returns `invalid_request` / `invalid_input` with `Evidence bundle contains duplicate record IDs` in `details.issues`. The valid request succeeds again after both failures. Each focused run passed its one test because the assertions record current faulty behavior. Run 1: 4.26 s real, 4.25 s user, 0.82 s system; maximum resident set size 713818112 bytes. Run 2: 4.08 s real, 4.43 s user, 0.87 s system; maximum resident set size 661684224 bytes. These are command measurements from `/usr/bin/time -l`, not sampled whole-family peak memory. Valid-call spans: 7.833999999999833 ms and 8.626291999999921 ms; missing-reference spans: 1.2909170000002632 ms and 1.8202919999998812 ms. No agent-token or production-performance claim is made. ### Typed outcome or recovery expected A non-retryable `invalid_request` / `invalid_input` outcome naming the absent Evidence reference in `details.issues` and directing input correction or bundle re-export. Keep malformed bundles rejected and complete bundles accepted. Genuine internal failures must retain their internal-error outcome. ### Analysis or suspected cause Source-derived: [candidate derivation](https://github.com/morluto/rea/blob/4a4bbab3c337701febeb6b532ace49041c4876e0/src/application/ReconstructionObligationCandidates.ts#L48-L56) authenticates the supplied bundle. [Graph validation](https://github.com/morluto/rea/blob/4a4bbab3c337701febeb6b532ace49041c4876e0/src/domain/evidenceBundle.ts#L370-L381) throws a plain `TypeError` for the missing reference. [The ledger catch](https://github.com/morluto/rea/blob/4a4bbab3c337701febeb6b532ace49041c4876e0/src/application/ReconstructionObligationLedgerService.ts#L54-L68) recognizes three named caller-input errors and ZodError; this failure falls through to AnalysisProtocolError. This explains the public result without establishing that every TypeError is caller input. ### Additional context This is the residual-graph follow-up explicitly excluded from merged [#4452](https://github.com/morluto/rea/pull/4452). Closed [#4444](https://github.com/morluto/rea/issues/4444) reproduced duplicate records and missing reviewed Evidence; its residual-graph mention was explicitly untested. Closed unmerged [#4481](https://github.com/morluto/rea/pull/4481) retains the exact residual-graph concern in its review, then closed because the two earlier fixes landed first. Those completed cases are controls here, rather than the reported defect. Current open [#6106](https://github.com/morluto/rea/issues/6106) covers cross-field `record_unknown` / `update_unknown` validation being wrapped as a changed-state error, with separate producers and closure checks. Searches including `validateUnknownGraph`, `ledger unreadable_output`, `"Residual unknown references missing evidence"`, `"residual unknown" "unreadable_output"`, `"mutation_evidence_ids"` and `"4444"` found no exact current ledger residual-reference follow-up. This is a bounded search result, not an exhaustive proof that no matching report exists. Fetched main `66aacc6fba7fb498302ca58894f179d77a847444` was inspected without checkout or execution. The bundle graph validator and ledger classification service are unchanged. Intervening changes replace integer-validator construction, share ordering/digest helpers, and add unrelated ASAR diagnostic details; none changes this reference check or classification branch. Persistence on that later revision is source-derived. No regression range was established.

bsakaguchi closed this issue

2026-10-31 00:44:15 +00:00

bsakaguchi referenced this issue

2026-10-31 01:10:24 +00:00

[PR #3807] [MERGED] feat(il2cpp): inventory Unity IL2CPP global-metadata images, types and methods #5913

bsakaguchi referenced this issue

2026-10-31 01:11:53 +00:00

[PR #4163] feat(analysis-view): page retained browser scenario steps and text #6103

bsakaguchi referenced this issue

2026-11-01 21:43:44 +00:00

[PR #5316] [MERGED] fix(ci): validate unindexed readiness heads with live PR metadata #6875