morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5957] [Bug] Process capture parsing drops structural issues when a semantic issue is present #3626

New issue

Closed

opened 2026-10-31 00:42:30 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:42:30 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5957

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

A malformed process capture with both a missing manifest.completed_at and an invalid finalization signal reason produces two schema issues. parseProcessCapture and the built compare-process-captures CLI retain only the finalization issue. The required timestamp disappears, with no remaining-issue count.

The same capture with only the missing timestamp reports that field correctly. The mixed-invalid and semantic-only inputs produce the same CLI diagnostic. All invalid inputs are rejected; this concerns lost validation information.

Reproduction

Using the already installed dependencies and built runtime at the revision below, I executed this script from the repository root with Node 24.18.0. Save it as capture-validation-repro.mjs and pass an existing disposable directory for its JSON files:

node capture-validation-repro.mjs fixture-directory

import { writeFileSync } from 'node:fs';
import { resolve, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { spawnSync } from 'node:child_process';
const load = p => import(pathToFileURL(resolve(p)));
const { FINALIZED_PROCESS_CAPTURE_EXAMPLE } = await load('dist/domain/process/processCaptureExample.js');
const { processCaptureSchema } = await load('dist/domain/process/processCapture.js');
const { parseProcessCapture } = await load('dist/domain/process/processCaptureParsing.js');
const { createEvidence } = await load('dist/domain/evidence.js');
const { PROCESS_PROVIDER } = await load('dist/domain/process/processEvidenceProvider.js');
const directory = resolve(process.argv[2]); // Existing disposable directory.
for (const name of ['valid', 'structural', 'semantic', 'mixed']) {
  const capture = structuredClone(FINALIZED_PROCESS_CAPTURE_EXAMPLE);
  if (name === 'structural' || name === 'mixed') delete capture.manifest.completed_at;
  if (name === 'semantic' || name === 'mixed') capture.exit.finalization.signals[0].reason = 'not allowed here';
  const checked = processCaptureSchema.safeParse(capture);
  console.log(name, 'schema issues:', checked.success ? [] : checked.error.issues);
  try { parseProcessCapture(capture); }
  catch (error) { console.log(name, 'parser:', error.name, error.message); }
  const evidence = createEvidence(undefined, PROCESS_PROVIDER, {
    predicateType: 'rea.process-capture', operation: 'capture_process_scenario',
    parameters: {}, result: capture,
  });
  writeFileSync(join(directory, `${name}.json`), JSON.stringify(evidence));
}
for (const name of ['valid', 'structural', 'semantic', 'mixed']) {
  const run = spawnSync(process.execPath, ['scripts/rea.mjs', 'compare-process-captures',
    join(directory, 'valid.json'), join(directory, `${name}.json`), '--json'],
    {encoding: 'utf8', timeout: 30000});
  if (run.error) throw run.error;
  const result = JSON.parse(run.stdout);
  console.log(name, JSON.stringify({exit: run.status, code: result.code,
    message: result.message, issues: result.details?.issues}));
}

The script builds canonical Evidence envelopes around synthetic captures, so Evidence identifier validation succeeds and the normalized capture parser is reached. It runs four real CLI subprocesses, plus the schema/parser controls. No target process is captured or simulated provider used.

Expected behavior

Preserve structural validation issues when the same capture also produces a semantic issue. Returning the original schema error would allow the existing formatter to identify the first issue and report the number remaining; preserving every issue in equivalent error metadata would also satisfy this expectation.

The existing formatter deliberately retains the first schema path/message and additional issue count. The comparison diagnostic test expects invalid normalized capture results to retain validation details through both the CLI workflow and MCP. This report does not require every message to be printed in full.

Impact and scope

P2: a bounded validation-diagnostic defect for captures containing both structural and nested semantic failures. A caller correcting the displayed semantic problem still fails on the undisclosed structural problem. Automated callers cannot recover the complete rejected-field set or know another issue exists from the projected diagnostic.

A practical diagnostic workaround is to inspect processCaptureSchema.safeParse(...).error.issues; the script demonstrates that both issues remain there. Recreating an untouched capture is the CLI's existing recovery direction. Input rejection and error classification remain correct; no data loss or malformed-input acceptance was observed. Frequency in real captures was not measured.

REA version

rea-agents 6.7.0, source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node 24.18.0, the repository toolchain pin.

Target and artifact context

Synthetic finalized process-capture result from the public example, wrapped in canonical process-capture Evidence. No live target, PTY, debugger, or external provider ran.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

The mixed input's schema result contains:

invalid_type at manifest.completed_at:
  Invalid input: expected string, received undefined
custom at exit.finalization.signals.0.reason:
  finalization reason is only allowed for an unverified delivery

parseProcessCapture instead throws:

TypeError: Invalid process capture: exit.finalization.signals.0.reason: finalization reason is only allowed for an unverified delivery

CLI mixed-input output exits 1 and includes:

{
  "code": "invalid_request",
  "category": "invalid_input",
  "message": "Capture evidence is malformed (Invalid process capture: exit.finalization.signals.0.reason: finalization reason is only allowed for an unverified delivery). Create new capture evidence, then try again."
}

Its single details.issues entry has path: ["right"] and repeats that same incomplete message. The structural-only control names manifest.completed_at; the valid control exits 0. The behavior reproduced in the script above and in a separate assertion-based probe of the same inputs.

Typed outcome or recovery expected

Keep invalid_request / invalid_input and the existing recovery direction. Preserve both rejected fields in metadata, or at least retain the first schema issue and the count of additional issues in the summary. Fixing one reported constraint should not hide the existence of other constraints already found during that parse.

Analysis or suspected cause

Source-derived cause: parseProcessCapture filters the failed schema parse down to code === "custom", then throws that subset as a TypeError. The original schema error is thrown only when the subset is empty. A nested finalization refinement can produce a custom issue while an unrelated manifest field produces a structural issue. The CLI caller subsequently has only the filtered exception to describe.

The issue originates in capture parsing; changing only the caller's formatter cannot recover issues that have already been discarded.

Additional context

Upstream main was fetched at 25e1a24a3ed3671a0148757555f18d16c29d936a; the affected parser, schema, caller and formatter files had no changes from the tested revision. That is a source freshness check, not execution of the upstream revision.

Related #155 and PR #1080 concern classification and caller-side descriptions; PR #1073 concerns bundle diagnostics. Their fixes do not retain the mixed failures discarded here. #5923 concerns first-divergence localization for valid captures and is separate. MCP comparison uses the same parser, but MCP transport, bundle import, reconstruction verification, live capture, other platforms, and the full suite were not executed.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5957 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior A malformed process capture with both a missing `manifest.completed_at` and an invalid finalization signal `reason` produces two schema issues. `parseProcessCapture` and the built `compare-process-captures` CLI retain only the finalization issue. The required timestamp disappears, with no remaining-issue count. The same capture with only the missing timestamp reports that field correctly. The mixed-invalid and semantic-only inputs produce the same CLI diagnostic. All invalid inputs are rejected; this concerns lost validation information. ### Reproduction Using the already installed dependencies and built runtime at the revision below, I executed this script from the repository root with Node 24.18.0. Save it as `capture-validation-repro.mjs` and pass an existing disposable directory for its JSON files: ```sh node capture-validation-repro.mjs fixture-directory ``` ```js import { writeFileSync } from 'node:fs'; import { resolve, join } from 'node:path'; import { pathToFileURL } from 'node:url'; import { spawnSync } from 'node:child_process'; const load = p => import(pathToFileURL(resolve(p))); const { FINALIZED_PROCESS_CAPTURE_EXAMPLE } = await load('dist/domain/process/processCaptureExample.js'); const { processCaptureSchema } = await load('dist/domain/process/processCapture.js'); const { parseProcessCapture } = await load('dist/domain/process/processCaptureParsing.js'); const { createEvidence } = await load('dist/domain/evidence.js'); const { PROCESS_PROVIDER } = await load('dist/domain/process/processEvidenceProvider.js'); const directory = resolve(process.argv[2]); // Existing disposable directory. for (const name of ['valid', 'structural', 'semantic', 'mixed']) { const capture = structuredClone(FINALIZED_PROCESS_CAPTURE_EXAMPLE); if (name === 'structural' || name === 'mixed') delete capture.manifest.completed_at; if (name === 'semantic' || name === 'mixed') capture.exit.finalization.signals[0].reason = 'not allowed here'; const checked = processCaptureSchema.safeParse(capture); console.log(name, 'schema issues:', checked.success ? [] : checked.error.issues); try { parseProcessCapture(capture); } catch (error) { console.log(name, 'parser:', error.name, error.message); } const evidence = createEvidence(undefined, PROCESS_PROVIDER, { predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: {}, result: capture, }); writeFileSync(join(directory, `${name}.json`), JSON.stringify(evidence)); } for (const name of ['valid', 'structural', 'semantic', 'mixed']) { const run = spawnSync(process.execPath, ['scripts/rea.mjs', 'compare-process-captures', join(directory, 'valid.json'), join(directory, `${name}.json`), '--json'], {encoding: 'utf8', timeout: 30000}); if (run.error) throw run.error; const result = JSON.parse(run.stdout); console.log(name, JSON.stringify({exit: run.status, code: result.code, message: result.message, issues: result.details?.issues})); } ``` The script builds canonical Evidence envelopes around synthetic captures, so Evidence identifier validation succeeds and the normalized capture parser is reached. It runs four real CLI subprocesses, plus the schema/parser controls. No target process is captured or simulated provider used. ### Expected behavior Preserve structural validation issues when the same capture also produces a semantic issue. Returning the original schema error would allow the existing formatter to identify the first issue and report the number remaining; preserving every issue in equivalent error metadata would also satisfy this expectation. The [existing formatter](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/evidenceBundle.ts#L258-L267) deliberately retains the first schema path/message and additional issue count. The [comparison diagnostic test](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts#L499-L521) expects invalid normalized capture results to retain validation details through both the CLI workflow and MCP. This report does not require every message to be printed in full. ### Impact and scope P2: a bounded validation-diagnostic defect for captures containing both structural and nested semantic failures. A caller correcting the displayed semantic problem still fails on the undisclosed structural problem. Automated callers cannot recover the complete rejected-field set or know another issue exists from the projected diagnostic. A practical diagnostic workaround is to inspect `processCaptureSchema.safeParse(...).error.issues`; the script demonstrates that both issues remain there. Recreating an untouched capture is the CLI's existing recovery direction. Input rejection and error classification remain correct; no data loss or malformed-input acceptance was observed. Frequency in real captures was not measured. ### REA version rea-agents 6.7.0, source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node 24.18.0, the repository toolchain pin. ### Target and artifact context Synthetic finalized process-capture result from the public example, wrapped in canonical process-capture Evidence. No live target, PTY, debugger, or external provider ran. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence The mixed input's schema result contains: ```text invalid_type at manifest.completed_at: Invalid input: expected string, received undefined custom at exit.finalization.signals.0.reason: finalization reason is only allowed for an unverified delivery ``` `parseProcessCapture` instead throws: ```text TypeError: Invalid process capture: exit.finalization.signals.0.reason: finalization reason is only allowed for an unverified delivery ``` CLI mixed-input output exits 1 and includes: ```json { "code": "invalid_request", "category": "invalid_input", "message": "Capture evidence is malformed (Invalid process capture: exit.finalization.signals.0.reason: finalization reason is only allowed for an unverified delivery). Create new capture evidence, then try again." } ``` Its single `details.issues` entry has `path: ["right"]` and repeats that same incomplete message. The structural-only control names `manifest.completed_at`; the valid control exits 0. The behavior reproduced in the script above and in a separate assertion-based probe of the same inputs. ### Typed outcome or recovery expected Keep `invalid_request` / `invalid_input` and the existing recovery direction. Preserve both rejected fields in metadata, or at least retain the first schema issue and the count of additional issues in the summary. Fixing one reported constraint should not hide the existence of other constraints already found during that parse. ### Analysis or suspected cause Source-derived cause: [parseProcessCapture](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/process/processCaptureParsing.ts#L26-L38) filters the failed schema parse down to `code === "custom"`, then throws that subset as a `TypeError`. The original schema error is thrown only when the subset is empty. A [nested finalization refinement](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/process/processCapture.ts#L413-L449) can produce a custom issue while an unrelated manifest field produces a structural issue. The [CLI caller](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/process/ProcessCli.ts#L201-L237) subsequently has only the filtered exception to describe. The issue originates in capture parsing; changing only the caller's formatter cannot recover issues that have already been discarded. ### Additional context Upstream `main` was fetched at `25e1a24a3ed3671a0148757555f18d16c29d936a`; the affected parser, schema, caller and formatter files had no changes from the tested revision. That is a source freshness check, not execution of the upstream revision. Related #155 and PR #1080 concern classification and caller-side descriptions; PR #1073 concerns bundle diagnostics. Their fixes do not retain the mixed failures discarded here. #5923 concerns first-divergence localization for valid captures and is separate. MCP comparison uses the same parser, but MCP transport, bundle import, reconstruction verification, live capture, other platforms, and the full suite were not executed.

bsakaguchi closed this issue

2026-10-31 00:42:30 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471