morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7442] [Bug] analyze --snapshot replaces specific schema diagnostics with a constant failure reason #4449

New issue

Open

opened 2026-10-31 00:49:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:49:53 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7442

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Selecting a snapshot containing only empty entries and workflow_entries arrays makes analyze --snapshot return exit 1 with details.reason: "Analysis snapshot validation failed". Its public message and recovery action both say to recreate or re-import invalid Evidence. Neither identifies the missing snapshot fields. An independent call to the built snapshot reader on the same file showed an internal ZodError identifying missing target, binding and evidence_bundle fields, so the format validator had specific diagnostics available.

Reproduction

Use the prepared baseline checkout with its existing dependencies and built artifacts, accessible as ../rea from an otherwise empty working directory. Put Node 24.22.1 on PATH. Create these local fixtures:

mkdir home
python3 - <<'FIXTURE'
import json,zipfile
with zipfile.ZipFile('target.zip','w') as archive:
    archive.writestr('sample.txt','hello\n')
with open('snapshot.json','w') as snapshot:
    json.dump({'entries': [], 'workflow_entries': []},snapshot)
FIXTURE

Run the supported CLI with its home redirected to the fixture directory:

HOME=./home node ../rea/scripts/rea.mjs analyze ./target.zip --snapshot ./snapshot.json --format json

Observe exit 1, empty stderr and this stdout:

{"error":"Analysis failed","code":"evidence_integrity_mismatch","category":"integrity_mismatch","message":"Evidence is invalid or has changed. Recreate or re-import it, then try again.","retryable":false,"remediation":{"action":"Evidence is invalid or has changed. Recreate or re-import it, then try again."},"details":{"reason":"Analysis snapshot validation failed"}}

The missing-header case produced this result in all three observed invocations: two in an initial fixture matrix and one execution of the reduced commands above. No target executable or analysis-provider engine was started.

Expected behavior

Retain a useful description of the format validator's specific failure through the file wrapper and public projection while continuing to refuse the invalid snapshot. docs/tool-design.md, “Implement and evaluate” (lines 318-321), states: “File-error wrappers retain the format owner's specific diagnostic and recovery text.” Its strict-JSON example explicitly preserves that information through CLI/MCP projection. For this input, the existing missing-field diagnostics should not become a constant reason that describes none of them. This does not require exposing the raw exception, adding a new structured issue list, or including a selected-file-path field.

Impact and scope

The observed loss is diagnostic specificity for invalid saved snapshots: the caller cannot identify the missing fields from the returned failure. The invalid input remains refused. The initial matrix also observed the same constant reason twice each for an invalid provider entry and an incomplete embedded bundle, while a schema-valid wrong-target control returned its specific mismatch explanation twice. No incorrect replay, target mutation, data loss or provider failure was observed. MCP delivery, naturally occurring snapshot damage, other platforms and repair of the malformed files were not tested.

REA version

6.8.0 (source commit a905a71e31)

Runtime and platform

Node 24.22.1; macOS 27.0.1 arm64

Target and artifact context

An inline-generated ZIP containing one regular text member and a manually authored JSON snapshot with only two empty entry arrays; the ZIP supplies a local CLI target and is never executed.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Observed CLI output is included above. Independently executing the built readAnalysisSnapshot on the same header fixture returned an integrity failure with internal cause name ZodError and issue paths ["target"], ["binding"] and ["evidence_bundle"]; their messages were respectively Invalid input: expected object, received undefined, Invalid input: expected object, received undefined, and Invalid input: expected nonoptional, received undefined. This establishes that specific format diagnostics exist before projection; it does not establish that callers should receive the raw cause object.

Source-only (not executed as an instrumented source test): src/domain/analysisSnapshot.ts:232 defines the required header fields and src/domain/analysisSnapshot.ts:244 parses them. src/application/binary/AnalysisSnapshotFiles.ts:71 invokes that parser, and src/application/binary/AnalysisSnapshotFiles.ts:86 constructs the integrity failure using snapshotValidationMessage. That helper at src/application/binary/AnalysisSnapshotFiles.ts:143 retains TypeError messages but substitutes the constant for other exceptions. The built reader and CLI were executed separately; no source edits or injected failures were used.

Typed outcome or recovery expected

Preserve refusal and its integrity classification while returning the existing format-specific explanation, or an equivalent concise diagnostic, through the canonical public error projection.

Analysis or suspected cause

Source-only (not executed as an instrumented source test): the snapshot parser raises ZodError, whereas the message helper in src/application/binary/AnalysisSnapshotFiles.ts:143 only recognizes TypeError. The original exception survives as the typed error's internal cause, but the message passed onward is already the constant. Retaining internal causes and preserving useful public diagnostics are separate requirements; this report does not request automatic cause serialization.

Additional context

The latest published rea-agents version returned by npm view rea-agents version was 6.8.0. Open issue #7311 concerns saving a fresh archive session, and closed issue #5945 concerns a valid snapshot selected for different target bytes; both have different triggers and acceptance criteria. The wrong-target control already showed the specific recovery described by #5945's closure. No open exact duplicate was found. The “Recovery text” section in docs/design/limits-and-recovery.md concerns limits/settings and does not establish a missing-field contract here; the CLI snapshot section also makes no separate field-naming promise. The expectation above relies on the file-wrapper rule. The first affected revision is unknown.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7442 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Selecting a snapshot containing only empty `entries` and `workflow_entries` arrays makes `analyze --snapshot` return exit 1 with `details.reason: "Analysis snapshot validation failed"`. Its public message and recovery action both say to recreate or re-import invalid Evidence. Neither identifies the missing snapshot fields. An independent call to the built snapshot reader on the same file showed an internal `ZodError` identifying missing `target`, `binding` and `evidence_bundle` fields, so the format validator had specific diagnostics available. ### Reproduction 1. Use the prepared baseline checkout with its existing dependencies and built artifacts, accessible as `../rea` from an otherwise empty working directory. Put Node 24.22.1 on PATH. Create these local fixtures: ```sh mkdir home python3 - <<'FIXTURE' import json,zipfile with zipfile.ZipFile('target.zip','w') as archive: archive.writestr('sample.txt','hello\n') with open('snapshot.json','w') as snapshot: json.dump({'entries': [], 'workflow_entries': []},snapshot) FIXTURE ``` 2. Run the supported CLI with its home redirected to the fixture directory: ```sh HOME=./home node ../rea/scripts/rea.mjs analyze ./target.zip --snapshot ./snapshot.json --format json ``` 3. Observe exit 1, empty stderr and this stdout: ```json {"error":"Analysis failed","code":"evidence_integrity_mismatch","category":"integrity_mismatch","message":"Evidence is invalid or has changed. Recreate or re-import it, then try again.","retryable":false,"remediation":{"action":"Evidence is invalid or has changed. Recreate or re-import it, then try again."},"details":{"reason":"Analysis snapshot validation failed"}} ``` The missing-header case produced this result in all three observed invocations: two in an initial fixture matrix and one execution of the reduced commands above. No target executable or analysis-provider engine was started. ### Expected behavior Retain a useful description of the format validator's specific failure through the file wrapper and public projection while continuing to refuse the invalid snapshot. `docs/tool-design.md`, “Implement and evaluate” (lines 318-321), states: “File-error wrappers retain the format owner's specific diagnostic and recovery text.” Its strict-JSON example explicitly preserves that information through CLI/MCP projection. For this input, the existing missing-field diagnostics should not become a constant reason that describes none of them. This does not require exposing the raw exception, adding a new structured issue list, or including a selected-file-path field. ### Impact and scope The observed loss is diagnostic specificity for invalid saved snapshots: the caller cannot identify the missing fields from the returned failure. The invalid input remains refused. The initial matrix also observed the same constant reason twice each for an invalid provider entry and an incomplete embedded bundle, while a schema-valid wrong-target control returned its specific mismatch explanation twice. No incorrect replay, target mutation, data loss or provider failure was observed. MCP delivery, naturally occurring snapshot damage, other platforms and repair of the malformed files were not tested. ### REA version 6.8.0 (source commit a905a71e31f71457cb21fd056b234f872126974e) ### Runtime and platform Node 24.22.1; macOS 27.0.1 arm64 ### Target and artifact context An inline-generated ZIP containing one regular text member and a manually authored JSON snapshot with only two empty entry arrays; the ZIP supplies a local CLI target and is never executed. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence Observed CLI output is included above. Independently executing the built `readAnalysisSnapshot` on the same header fixture returned an integrity failure with internal cause name `ZodError` and issue paths `["target"]`, `["binding"]` and `["evidence_bundle"]`; their messages were respectively `Invalid input: expected object, received undefined`, `Invalid input: expected object, received undefined`, and `Invalid input: expected nonoptional, received undefined`. This establishes that specific format diagnostics exist before projection; it does not establish that callers should receive the raw cause object. Source-only (not executed as an instrumented source test): `src/domain/analysisSnapshot.ts:232` defines the required header fields and `src/domain/analysisSnapshot.ts:244` parses them. `src/application/binary/AnalysisSnapshotFiles.ts:71` invokes that parser, and `src/application/binary/AnalysisSnapshotFiles.ts:86` constructs the integrity failure using `snapshotValidationMessage`. That helper at `src/application/binary/AnalysisSnapshotFiles.ts:143` retains TypeError messages but substitutes the constant for other exceptions. The built reader and CLI were executed separately; no source edits or injected failures were used. ### Typed outcome or recovery expected Preserve refusal and its integrity classification while returning the existing format-specific explanation, or an equivalent concise diagnostic, through the canonical public error projection. ### Analysis or suspected cause Source-only (not executed as an instrumented source test): the snapshot parser raises `ZodError`, whereas the message helper in `src/application/binary/AnalysisSnapshotFiles.ts:143` only recognizes `TypeError`. The original exception survives as the typed error's internal cause, but the message passed onward is already the constant. Retaining internal causes and preserving useful public diagnostics are separate requirements; this report does not request automatic cause serialization. ### Additional context The latest published `rea-agents` version returned by `npm view rea-agents version` was 6.8.0. Open issue #7311 concerns saving a fresh archive session, and closed issue #5945 concerns a valid snapshot selected for different target bytes; both have different triggers and acceptance criteria. The wrong-target control already showed the specific recovery described by #5945's closure. No open exact duplicate was found. The “Recovery text” section in `docs/design/limits-and-recovery.md` concerns limits/settings and does not establish a missing-field contract here; the CLI snapshot section also makes no separate field-naming promise. The expectation above relies on the file-wrapper rule. The first affected revision is unknown.

bsakaguchi referenced this issue

2026-10-31 01:13:01 +00:00

[PR #4449] [CLOSED] docs: improve research and tool discovery #6250

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4449

No description provided.

Rows
Columns