[GH-ISSUE #155] Process compare command misclassifies malformed evidence as execution_failure #100

New issue

Closed

opened 2026-10-15 18:21:10 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:21:10 +00:00

Owner

Originally created by @morluto on GitHub (Jul 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/155

Symptom

compare-process-captures can classify user-provided, schema-invalid evidence files as execution_failure instead of invalid_input.

Evidence

compareProcessEvidenceFiles reads both files and passes them into parseCaptureEvidence (src/application/ProcessCli.ts:90-93).
parseCaptureEvidence calls parseEvidence and parseProcessCapture without any local ProcessCliFailure translation (src/application/ProcessCli.ts:114-136).
Failures from those parsers are not ProcessCliFailure or AnalysisError, so projectProcessCliError falls through to category execution_failure (src/application/ProcessCli.ts:178-193).

Reproduction

Create two valid JSON files where at least one evidence payload is malformed (for example {} or a file with an invalid evidence_id).
Run rea compare-process-captures <left.json> <right.json>.
Observe error object containing:

error: "Process command failed"
category: "execution_failure"

Expected

Malformed user input should be reported as category: "invalid_input", matching how other CLI argument/evidence parse failures are handled.

Why this matters

Users get an incorrect failure class for deterministic input validation failures, making automated triage and UX guidance inconsistent with other CLI process commands (invalid_input vs generic execution_failure).

Originally created by @morluto on GitHub (Jul 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/155 ## Symptom `compare-process-captures` can classify user-provided, schema-invalid evidence files as `execution_failure` instead of `invalid_input`. ## Evidence - `compareProcessEvidenceFiles` reads both files and passes them into `parseCaptureEvidence` (`src/application/ProcessCli.ts:90-93`). - `parseCaptureEvidence` calls `parseEvidence` and `parseProcessCapture` without any local `ProcessCliFailure` translation (`src/application/ProcessCli.ts:114-136`). - Failures from those parsers are not `ProcessCliFailure` or `AnalysisError`, so `projectProcessCliError` falls through to category `execution_failure` (`src/application/ProcessCli.ts:178-193`). ## Reproduction 1. Create two valid JSON files where at least one evidence payload is malformed (for example `{}` or a file with an invalid `evidence_id`). 2. Run `rea compare-process-captures <left.json> <right.json>`. 3. Observe error object containing: - `error: "Process command failed"` - `category: "execution_failure"` ## Expected Malformed user input should be reported as `category: "invalid_input"`, matching how other CLI argument/evidence parse failures are handled. ## Why this matters Users get an incorrect failure class for deterministic input validation failures, making automated triage and UX guidance inconsistent with other CLI process commands (`invalid_input` vs generic `execution_failure`).

bsakaguchi

2026-10-15 18:21:10 +00:00