morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3219] [Bug] Non-canonical Evidence bundles are reported as unreadable ledger output #1157

New issue

Closed

opened 2026-10-15 18:31:35 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-15 18:31:35 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3219

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

build-reconstruction-obligation-ledger rejects a caller-supplied Evidence bundle with reversed record order as unreadable analysis output. The verified CLI request exited 1 with code: "unreadable_output", category: "execution_failure", and retryable: false. It directed the caller to run rea doctor and omitted the canonical-order constraint.

Changing only the order of the two records made the ledger command exit 0 with an unknown, zero-obligation ledger. Deleting the records field produced the ordinary invalid_request / invalid_input response. evaluate-reconstruction-coverage rejected the exact same reversed bundle as invalid_request / invalid_input, with retryable: true and the message "Evidence bundle manifests are not canonical".

Reproduction

Start from a clean checkout of morluto/rea at 58300885207133422af235e0f2b10658cf958b03. Install the locked dependencies and build the runtime from the repository root:
```
npm ci
npm run build:cached
```
With Python 3 available, run this block from the repository root. It creates the complete inputs inline and isolates the CLI's HOME and XDG directories:

fixture_dir=$(mktemp -d)
mkdir -p "$fixture_dir/home/config" "$fixture_dir/home/cache" "$fixture_dir/home/data"
python3 - "$fixture_dir" <<'PY'
import copy
import json
import pathlib
import sys

output = pathlib.Path(sys.argv[1])
provider = {"id": "fixture", "name": "Fixture", "version": "1"}

def record(evidence_id, digit):
    return {
        "evidence_id": evidence_id,
        "subject": None,
        "provider": provider,
        "analysis_profile": None,
        "predicate_type": "rea.analysis",
        "operation": "inventory_artifact",
        "parameters": {},
        "raw_result": None,
        "normalized_result": {"digit": digit},
        "confidence": "observed",
        "authority": "shipped-artifact",
        "environment": None,
        "limitations": ["Artifact identity is unavailable for this observation."],
        "relationship_coverage": None,
        "locations": [],
        "evidence_links": [],
    }

bundle = {
    "artifacts": [],
    "providers": [provider],
    "environments": [],
    "scenarios": [],
    "captures": [],
    "unknowns": [],
    "records": [
        record("ev_e27f44efa84a1893323219d98a7b305ee30ff7a0317f99c16556e6a96b282499", "2"),
        record("ev_d9f1d065cc34b0133878db108d84bf6219f34ee892b1e1daaa88cd77eadfe341", "1"),
    ],
}
ledger = {
    "evidence_bundle": bundle,
    "reviewed_obligations": [],
    "manifest": {"bindings": [], "contradictions": []},
}
canonical = copy.deepcopy(ledger)
canonical["evidence_bundle"]["records"].sort(key=lambda item: item["evidence_id"])
missing = copy.deepcopy(ledger)
del missing["evidence_bundle"]["records"]
coverage = {
    "boundary_id": "b",
    "coverage": {
        "evidence_bundle": bundle,
        "artifacts": [],
        "surfaces": [],
        "owners": [],
        "claims": [],
        "verifier_contracts": [],
        "verifier_results": [],
        "residual_unknown_ids": [],
        "contradictions": [],
        "package_proofs": [],
        "boundaries": [{
            "boundary_id": "b",
            "title": "Test boundary",
            "required_surface_ids": ["s"],
            "required_claim_ids": ["c"],
            "required_package_proof_kinds": ["build"],
            "allowed_dispositions": [],
            "allowed_unknown_ids": [],
        }],
    },
}
for name, value in {
    "ledger-reversed": ledger,
    "ledger-canonical": canonical,
    "ledger-missing-records": missing,
    "coverage-reversed": coverage,
}.items():
    (output / (name + ".json")).write_text(json.dumps(value) + "\n")
PY
rea_local() {
  env HOME="$fixture_dir/home" \
    XDG_CONFIG_HOME="$fixture_dir/home/config" \
    XDG_CACHE_HOME="$fixture_dir/home/cache" \
    XDG_DATA_HOME="$fixture_dir/home/data" \
    node scripts/rea.mjs "$@"
}
rea_local build-reconstruction-obligation-ledger "$fixture_dir/ledger-reversed.json" --json
printf 'reversed ledger exit: %s\n' "$?"
rea_local build-reconstruction-obligation-ledger "$fixture_dir/ledger-canonical.json" --json
printf 'canonical ledger exit: %s\n' "$?"
rea_local build-reconstruction-obligation-ledger "$fixture_dir/ledger-missing-records.json" --json
printf 'missing records exit: %s\n' "$?"
rea_local evaluate-reconstruction-coverage "$fixture_dir/coverage-reversed.json" --json
printf 'coverage exit: %s\n' "$?"
rm -r "$fixture_dir"

Observe the exit statuses 1, 0, 1, 1. The reversed ledger reports unreadable output, the canonical ledger succeeds with status: "unknown" and zero obligations, the missing-records ledger reports invalid input, and coverage reports invalid input with the canonical-order constraint.

The independent verifier replayed this exact fixture block against the prepared built baseline runtime. It did not rerun npm ci or npm run build:cached, because dependencies and built output were already present.

Expected behavior

Reject the descending bundle as caller input that violates Evidence authentication, with invalid_request, category invalid_input, and actionable canonical-order guidance. docs/cli.md:302-313 documents structured failure classification, including invalid input. AGENTS.md, Architecture and Cleanup, requires expected failures to use the tagged error algebra and Result. docs/reconstruction-obligation-ledgers.md:3-8 requires authenticated Evidence, so rejection of non-canonical bundles remains appropriate. The coverage command demonstrates the applicable classification for this exact constraint.

The canonical control should continue to succeed with an unknown, zero-obligation ledger and its lack-of-closure limitation. The missing-records control should retain its invalid-input classification. Evidence authentication must remain intact.

Impact and scope

This bounded malformed-input case gives callers the wrong machine-readable category and retryability and removes the constraint needed to correct the request. The response directs them to diagnose analysis output or provider setup instead of sorting the bundle. The verified workaround is to sort these two records by evidence_id, preserving each record and its Evidence ID.

The controls establish that this valid sorted request and a structural input error still work. No data loss or general workflow outage was observed. The verifier exercised CLI behavior; live MCP behavior was not exercised.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node v26.10.0; macOS arm64. The checkout pins Node 24.18.0, which the verifier did not exercise. The macOS version was not recorded.

Target and artifact context

A synthetic Evidence bundle with two authenticated records, complete matching manifests, and the records array in descending evidence_id order. The ledger request has no reviewed obligations and an empty binding/contradiction manifest. The coverage control supplies the same bundle in a complete coverage request. No external analysis engine or network service is required.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Reversed ledger, exit 1:
{"error":"Application workflow failed","code":"unreadable_output","category":"execution_failure","message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","retryable":false,"remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}}

Coverage with the same reversed bundle, exit 1:
{"error":"Application workflow failed","code":"invalid_request","category":"invalid_input","message":"Analysis input is invalid. Check the arguments and try again.","retryable":true,"remediation":{"action":"Correct the listed arguments and retry."},"details":{"operation":"evaluate_reconstruction_coverage","issues":[{"path":[],"reason":"invalid_value","message":"Evidence bundle manifests are not canonical"}]}}

Canonical ledger: exit 0, ReconstructionObligationLedger, status "unknown", summary.total 0.
Missing records: exit 1, invalid_request, invalid_input, retryable true, missing evidence_bundle.records array.

Typed outcome or recovery expected

Return invalid_request / invalid_input with an issue that identifies "Evidence bundle manifests are not canonical" and recovery guidance to correct the input and retry. Coverage currently returns retryable: true, details.operation: "evaluate_reconstruction_coverage", and a root-path invalid_value issue with that message. The ledger should report its own operation and preserve the failed constraint. The unchanged descending request should remain rejected.

Analysis or suspected cause

Source-derived reasoning at baseline 58300885207133422af235e0f2b10658cf958b03:

src/cli/applicationCommands.ts:150-166 reaches the ledger service after structural validation.
src/application/ReconstructionObligationCandidates.ts:40-44 authenticates the supplied bundle before deriving candidates.
src/domain/evidenceBundle.ts:279-310 authenticates the records and compares the canonical projection. Lines 309-310 throw a TypeError for non-canonical bundle order.
src/application/ReconstructionObligationLedgerService.ts:47-54 classifies only Zod errors as input errors and wraps the canonical-order TypeError in AnalysisProtocolError.
src/application/workflowInputError.ts:20-34 already accepts a ZodError or TypeError and preserves the failed constraint as an input issue. src/application/ReconstructionCoverageService.ts:32-36 uses that input-error contract for this constraint.
src/domain/analysisErrorCore.ts:54-55, src/domain/analysisErrorProjection.ts:192-196, and src/domain/analysisErrorPresentation.ts:198-202 define and project caller-input errors as invalid_request / invalid_input.

A separate built-runtime probe confirmed that the reversed request passes the structural ledger input schema, then parseEvidenceBundle throws TypeError: Evidence bundle manifests are not canonical. Sorting the same records authenticates both. Structural schema acceptance does not establish semantic bundle validity.

Additional context

Related PRs #1089 and #1073 improved diagnostics for already-classified workflow input errors and bundle import/comparison failures, respectively; this report requires correcting the ledger's canonical-order TypeError classification, which those changes left unchanged.

The verifier assessed P2 because the defect changes machine-readable failure behavior and correction guidance for a bounded malformed input, with a demonstrated safe sorting workaround and no observed data loss or general workflow outage.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3219 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `build-reconstruction-obligation-ledger` rejects a caller-supplied Evidence bundle with reversed record order as unreadable analysis output. The verified CLI request exited 1 with `code: "unreadable_output"`, `category: "execution_failure"`, and `retryable: false`. It directed the caller to run `rea doctor` and omitted the canonical-order constraint. Changing only the order of the two records made the ledger command exit 0 with an unknown, zero-obligation ledger. Deleting the records field produced the ordinary `invalid_request` / `invalid_input` response. `evaluate-reconstruction-coverage` rejected the exact same reversed bundle as `invalid_request` / `invalid_input`, with `retryable: true` and the message "Evidence bundle manifests are not canonical". ### Reproduction 1. Start from a clean checkout of `morluto/rea` at `58300885207133422af235e0f2b10658cf958b03`. Install the locked dependencies and build the runtime from the repository root: ```sh npm ci npm run build:cached ``` 2. With Python 3 available, run this block from the repository root. It creates the complete inputs inline and isolates the CLI's HOME and XDG directories: ```sh fixture_dir=$(mktemp -d) mkdir -p "$fixture_dir/home/config" "$fixture_dir/home/cache" "$fixture_dir/home/data" python3 - "$fixture_dir" <<'PY' import copy import json import pathlib import sys output = pathlib.Path(sys.argv[1]) provider = {"id": "fixture", "name": "Fixture", "version": "1"} def record(evidence_id, digit): return { "evidence_id": evidence_id, "subject": None, "provider": provider, "analysis_profile": None, "predicate_type": "rea.analysis", "operation": "inventory_artifact", "parameters": {}, "raw_result": None, "normalized_result": {"digit": digit}, "confidence": "observed", "authority": "shipped-artifact", "environment": None, "limitations": ["Artifact identity is unavailable for this observation."], "relationship_coverage": None, "locations": [], "evidence_links": [], } bundle = { "artifacts": [], "providers": [provider], "environments": [], "scenarios": [], "captures": [], "unknowns": [], "records": [ record("ev_e27f44efa84a1893323219d98a7b305ee30ff7a0317f99c16556e6a96b282499", "2"), record("ev_d9f1d065cc34b0133878db108d84bf6219f34ee892b1e1daaa88cd77eadfe341", "1"), ], } ledger = { "evidence_bundle": bundle, "reviewed_obligations": [], "manifest": {"bindings": [], "contradictions": []}, } canonical = copy.deepcopy(ledger) canonical["evidence_bundle"]["records"].sort(key=lambda item: item["evidence_id"]) missing = copy.deepcopy(ledger) del missing["evidence_bundle"]["records"] coverage = { "boundary_id": "b", "coverage": { "evidence_bundle": bundle, "artifacts": [], "surfaces": [], "owners": [], "claims": [], "verifier_contracts": [], "verifier_results": [], "residual_unknown_ids": [], "contradictions": [], "package_proofs": [], "boundaries": [{ "boundary_id": "b", "title": "Test boundary", "required_surface_ids": ["s"], "required_claim_ids": ["c"], "required_package_proof_kinds": ["build"], "allowed_dispositions": [], "allowed_unknown_ids": [], }], }, } for name, value in { "ledger-reversed": ledger, "ledger-canonical": canonical, "ledger-missing-records": missing, "coverage-reversed": coverage, }.items(): (output / (name + ".json")).write_text(json.dumps(value) + "\n") PY rea_local() { env HOME="$fixture_dir/home" \ XDG_CONFIG_HOME="$fixture_dir/home/config" \ XDG_CACHE_HOME="$fixture_dir/home/cache" \ XDG_DATA_HOME="$fixture_dir/home/data" \ node scripts/rea.mjs "$@" } rea_local build-reconstruction-obligation-ledger "$fixture_dir/ledger-reversed.json" --json printf 'reversed ledger exit: %s\n' "$?" rea_local build-reconstruction-obligation-ledger "$fixture_dir/ledger-canonical.json" --json printf 'canonical ledger exit: %s\n' "$?" rea_local build-reconstruction-obligation-ledger "$fixture_dir/ledger-missing-records.json" --json printf 'missing records exit: %s\n' "$?" rea_local evaluate-reconstruction-coverage "$fixture_dir/coverage-reversed.json" --json printf 'coverage exit: %s\n' "$?" rm -r "$fixture_dir" ``` 3. Observe the exit statuses `1`, `0`, `1`, `1`. The reversed ledger reports unreadable output, the canonical ledger succeeds with `status: "unknown"` and zero obligations, the missing-records ledger reports invalid input, and coverage reports invalid input with the canonical-order constraint. The independent verifier replayed this exact fixture block against the prepared built baseline runtime. It did not rerun `npm ci` or `npm run build:cached`, because dependencies and built output were already present. ### Expected behavior Reject the descending bundle as caller input that violates Evidence authentication, with `invalid_request`, category `invalid_input`, and actionable canonical-order guidance. `docs/cli.md:302-313` documents structured failure classification, including invalid input. `AGENTS.md`, Architecture and Cleanup, requires expected failures to use the tagged error algebra and Result. `docs/reconstruction-obligation-ledgers.md:3-8` requires authenticated Evidence, so rejection of non-canonical bundles remains appropriate. The coverage command demonstrates the applicable classification for this exact constraint. The canonical control should continue to succeed with an unknown, zero-obligation ledger and its lack-of-closure limitation. The missing-records control should retain its invalid-input classification. Evidence authentication must remain intact. ### Impact and scope This bounded malformed-input case gives callers the wrong machine-readable category and retryability and removes the constraint needed to correct the request. The response directs them to diagnose analysis output or provider setup instead of sorting the bundle. The verified workaround is to sort these two records by `evidence_id`, preserving each record and its Evidence ID. The controls establish that this valid sorted request and a structural input error still work. No data loss or general workflow outage was observed. The verifier exercised CLI behavior; live MCP behavior was not exercised. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node v26.10.0; macOS arm64. The checkout pins Node 24.18.0, which the verifier did not exercise. The macOS version was not recorded. ### Target and artifact context A synthetic Evidence bundle with two authenticated records, complete matching manifests, and the records array in descending `evidence_id` order. The ledger request has no reviewed obligations and an empty binding/contradiction manifest. The coverage control supplies the same bundle in a complete coverage request. No external analysis engine or network service is required. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence ```text Reversed ledger, exit 1: {"error":"Application workflow failed","code":"unreadable_output","category":"execution_failure","message":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.","retryable":false,"remediation":{"action":"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed."}} Coverage with the same reversed bundle, exit 1: {"error":"Application workflow failed","code":"invalid_request","category":"invalid_input","message":"Analysis input is invalid. Check the arguments and try again.","retryable":true,"remediation":{"action":"Correct the listed arguments and retry."},"details":{"operation":"evaluate_reconstruction_coverage","issues":[{"path":[],"reason":"invalid_value","message":"Evidence bundle manifests are not canonical"}]}} Canonical ledger: exit 0, ReconstructionObligationLedger, status "unknown", summary.total 0. Missing records: exit 1, invalid_request, invalid_input, retryable true, missing evidence_bundle.records array. ``` ### Typed outcome or recovery expected Return `invalid_request` / `invalid_input` with an issue that identifies "Evidence bundle manifests are not canonical" and recovery guidance to correct the input and retry. Coverage currently returns `retryable: true`, `details.operation: "evaluate_reconstruction_coverage"`, and a root-path `invalid_value` issue with that message. The ledger should report its own operation and preserve the failed constraint. The unchanged descending request should remain rejected. ### Analysis or suspected cause Source-derived reasoning at baseline `58300885207133422af235e0f2b10658cf958b03`: - `src/cli/applicationCommands.ts:150-166` reaches the ledger service after structural validation. - `src/application/ReconstructionObligationCandidates.ts:40-44` authenticates the supplied bundle before deriving candidates. - `src/domain/evidenceBundle.ts:279-310` authenticates the records and compares the canonical projection. Lines 309-310 throw a `TypeError` for non-canonical bundle order. - `src/application/ReconstructionObligationLedgerService.ts:47-54` classifies only Zod errors as input errors and wraps the canonical-order `TypeError` in `AnalysisProtocolError`. - `src/application/workflowInputError.ts:20-34` already accepts a ZodError or TypeError and preserves the failed constraint as an input issue. `src/application/ReconstructionCoverageService.ts:32-36` uses that input-error contract for this constraint. - `src/domain/analysisErrorCore.ts:54-55`, `src/domain/analysisErrorProjection.ts:192-196`, and `src/domain/analysisErrorPresentation.ts:198-202` define and project caller-input errors as `invalid_request` / `invalid_input`. A separate built-runtime probe confirmed that the reversed request passes the structural ledger input schema, then `parseEvidenceBundle` throws `TypeError: Evidence bundle manifests are not canonical`. Sorting the same records authenticates both. Structural schema acceptance does not establish semantic bundle validity. ### Additional context Related PRs #1089 and #1073 improved diagnostics for already-classified workflow input errors and bundle import/comparison failures, respectively; this report requires correcting the ledger's canonical-order TypeError classification, which those changes left unchanged. The verifier assessed P2 because the defect changes machine-readable failure behavior and correction guidance for a bounded malformed input, with a demonstrated safe sorting workaround and no observed data loss or general workflow outage.

bsakaguchi

2026-10-15 18:31:35 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:37 +00:00

Author

Owner

@aakash-env commented on GitHub (Oct 14, 2026):

@coygeek @monomyth kindly assign this issue to me

@aakash-env commented on GitHub (Oct 14, 2026): @coygeek @monomyth kindly assign this issue to me

bsakaguchi commented

2026-10-15 18:31:37 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 14, 2026):

@aakash-env Proceed

@coygeek commented on GitHub (Oct 14, 2026): @aakash-env Proceed

bsakaguchi commented

2026-10-15 18:31:37 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Resolved on main by 3a34ad37b.

Non-canonical authenticated bundles are classified as caller input errors with the canonical-order diagnostic; canonical bundles continue to succeed.

Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: tests/boundary/cli/reconstructionObligationLedgerCli.test.ts and tests/boundary/mcp/reconstructionObligationLedgerMcp.test.ts.

@morluto commented on GitHub (Oct 14, 2026): Resolved on main by 3a34ad37b. Non-canonical authenticated bundles are classified as caller input errors with the canonical-order diagnostic; canonical bundles continue to succeed. Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: tests/boundary/cli/reconstructionObligationLedgerCli.test.ts and tests/boundary/mcp/reconstructionObligationLedgerMcp.test.ts.

bsakaguchi referenced this issue

2026-10-16 21:02:34 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:09:08 +00:00

[PR #1157] [MERGED] fix(javascript): omit self-referential static references with disclosure #2690