[GH-ISSUE #5966] [Bug] Late process scenario events lose their indexes in validation issues #3636

New issue

Closed

opened 2026-10-31 00:42:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:42:35 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5966

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

Submitting a process scenario with two events after timeout_ms returns two identical validation issues, both at path: ["events"]. Neither structured issue identifies which array entry is invalid. The message also omits the event time.

This reproduced through the built capture-process CLI. It rejects the request with the correct invalid_request / invalid_input classification before launching the selected executable.

Reproduction

With the already installed dependencies and built runtime at the revision below, I executed this script from the repository root using Node 24.18.0. Save it as scenario-timeout-repro.mjs and pass an existing disposable directory:

node scenario-timeout-repro.mjs fixture-directory

import { writeFileSync } from 'node:fs';
import { resolve, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { spawnSync } from 'node:child_process';
const { processScenarioSchema } = await import(pathToFileURL(resolve('dist/domain/process/processScenario.js')));
const scenario = {executable: '/bin/true', timeout_ms: 1000, events: [
  {type: 'input', at_ms: 1500, data: 'a'},
  {type: 'signal', at_ms: 2000, signal: 'SIGINT'},
]};
const file = join(resolve(process.argv[2]), 'scenario.json');
writeFileSync(file, JSON.stringify(scenario));
const parsed = processScenarioSchema.safeParse(scenario);
console.log('schema issues:', parsed.success ? [] : parsed.error.issues);
const run = spawnSync(process.execPath, ['scripts/rea.mjs', 'capture-process', file, '--json'],
  {encoding: 'utf8', timeout: 30000});
if (run.error) throw run.error;
console.log('exit:', run.status, 'result:', run.stdout);
const control = structuredClone(scenario);
control.events = [{type: 'input', at_ms: 1000, data: 'a'}];
console.log('event exactly at timeout accepted by schema:', processScenarioSchema.safeParse(control).success);

The real command invoked is node scripts/rea.mjs capture-process scenario.json --json. The schema-only control moves one event exactly to the timeout; it is accepted without launching a process. No provider simulation is needed.

Expected behavior

Identify each rejected event by its array index, for example ["events", 0] and ["events", 1]. Including the offending at_ms in the message would add context, but the index is the essential missing machine-readable information.

Process capture documentation requires event times to fall within timeout_ms. Sibling per-event checks already report indexed paths for ordering and oversized resize events, and the resize test asserts that indexed path. The MCP diagnostic contract preserves request-specific rejected fields and reasons.

Impact and scope

P2: bounded loss of event identity in structured input diagnostics. A caller consuming details.issues cannot map either of the two issues to a particular event without reimplementing the timeout check over the original array. This affects correction/retry tooling and makes the returned duplicate issues less actionable.

A practical workaround is to inspect the original scenario and find events whose at_ms exceeds timeout_ms. The rejection itself remains correct. No capture starts, no artifact is corrupted, and no security boundary crossing was observed. Frequency in user scenarios and downstream automation effects were not measured.

REA version

rea-agents 6.7.0, source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node 24.18.0, the repository toolchain pin.

Target and artifact context

Process scenario JSON with timeout_ms=1000 and events at 1500 and 2000. Input validation rejects it before any target is launched.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

CLI exits 1 with code: "invalid_request", category: "invalid_input", and details.operation: "capture_process_scenario". Its details.issues is:

[
  {
    "path": ["events"],
    "reason": "invalid_value",
    "message": "event occurs after the scenario timeout"
  },
  {
    "path": ["events"],
    "reason": "invalid_value",
    "message": "event occurs after the scenario timeout"
  }
]

The direct schema parse reports the same two coarse paths. The event exactly at 1000 is accepted by the schema. These results reproduced in two script executions, including the initial assertion-based probe.

Typed outcome or recovery expected

Keep the existing classification and inspect_input_contract recovery. Return one indexed issue for each late event so callers can locate the entries to correct while retaining the same validation rule.

Analysis or suspected cause

Source-derived: the per-event validation loop has both index and event, but the timeout branch passes path: ["events"]. The adjacent resize branch uses ["events", index]. The CLI application workflow projects the schema issues before capture begins, so it faithfully propagates the missing event identity. Correcting the path at the schema boundary would address the producing source of the diagnostic.

Additional context

Fetched upstream main at 25e1a24a3ed3671a0148757555f18d16c29d936a has no changes to the affected scenario schema, CLI workflow, input projection or shared contract files. This checks source freshness; that upstream runtime was not built or executed.

#5923 concerns comparison divergence localization and #5916 concerns finite-trace duplicate admission; neither shares this contract or fix. #3423 concerns browser timeout documentation and #1253 concerns child finalization after timeout. MCP advertises the same process scenario schema, but MCP transport was not executed. Live capture, other platforms and the full suite were not run.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5966 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior Submitting a process scenario with two events after `timeout_ms` returns two identical validation issues, both at `path: ["events"]`. Neither structured issue identifies which array entry is invalid. The message also omits the event time. This reproduced through the built `capture-process` CLI. It rejects the request with the correct `invalid_request` / `invalid_input` classification before launching the selected executable. ### Reproduction With the already installed dependencies and built runtime at the revision below, I executed this script from the repository root using Node 24.18.0. Save it as `scenario-timeout-repro.mjs` and pass an existing disposable directory: ```sh node scenario-timeout-repro.mjs fixture-directory ``` ```js import { writeFileSync } from 'node:fs'; import { resolve, join } from 'node:path'; import { pathToFileURL } from 'node:url'; import { spawnSync } from 'node:child_process'; const { processScenarioSchema } = await import(pathToFileURL(resolve('dist/domain/process/processScenario.js'))); const scenario = {executable: '/bin/true', timeout_ms: 1000, events: [ {type: 'input', at_ms: 1500, data: 'a'}, {type: 'signal', at_ms: 2000, signal: 'SIGINT'}, ]}; const file = join(resolve(process.argv[2]), 'scenario.json'); writeFileSync(file, JSON.stringify(scenario)); const parsed = processScenarioSchema.safeParse(scenario); console.log('schema issues:', parsed.success ? [] : parsed.error.issues); const run = spawnSync(process.execPath, ['scripts/rea.mjs', 'capture-process', file, '--json'], {encoding: 'utf8', timeout: 30000}); if (run.error) throw run.error; console.log('exit:', run.status, 'result:', run.stdout); const control = structuredClone(scenario); control.events = [{type: 'input', at_ms: 1000, data: 'a'}]; console.log('event exactly at timeout accepted by schema:', processScenarioSchema.safeParse(control).success); ``` The real command invoked is `node scripts/rea.mjs capture-process scenario.json --json`. The schema-only control moves one event exactly to the timeout; it is accepted without launching a process. No provider simulation is needed. ### Expected behavior Identify each rejected event by its array index, for example `["events", 0]` and `["events", 1]`. Including the offending `at_ms` in the message would add context, but the index is the essential missing machine-readable information. [Process capture documentation](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/process-capture.md#L220-L221) requires event times to fall within `timeout_ms`. [Sibling per-event checks](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/process/processScenario.ts#L273-L297) already report indexed paths for ordering and oversized resize events, and the [resize test](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/process/processScenario.test.ts#L174-L188) asserts that indexed path. The [MCP diagnostic contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/mcp-contracts.md#L109-L113) preserves request-specific rejected fields and reasons. ### Impact and scope P2: bounded loss of event identity in structured input diagnostics. A caller consuming `details.issues` cannot map either of the two issues to a particular event without reimplementing the timeout check over the original array. This affects correction/retry tooling and makes the returned duplicate issues less actionable. A practical workaround is to inspect the original scenario and find events whose `at_ms` exceeds `timeout_ms`. The rejection itself remains correct. No capture starts, no artifact is corrupted, and no security boundary crossing was observed. Frequency in user scenarios and downstream automation effects were not measured. ### REA version rea-agents 6.7.0, source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node 24.18.0, the repository toolchain pin. ### Target and artifact context Process scenario JSON with timeout_ms=1000 and events at 1500 and 2000. Input validation rejects it before any target is launched. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence CLI exits 1 with `code: "invalid_request"`, `category: "invalid_input"`, and `details.operation: "capture_process_scenario"`. Its `details.issues` is: ```json [ { "path": ["events"], "reason": "invalid_value", "message": "event occurs after the scenario timeout" }, { "path": ["events"], "reason": "invalid_value", "message": "event occurs after the scenario timeout" } ] ``` The direct schema parse reports the same two coarse paths. The event exactly at 1000 is accepted by the schema. These results reproduced in two script executions, including the initial assertion-based probe. ### Typed outcome or recovery expected Keep the existing classification and `inspect_input_contract` recovery. Return one indexed issue for each late event so callers can locate the entries to correct while retaining the same validation rule. ### Analysis or suspected cause Source-derived: the [per-event validation loop](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/process/processScenario.ts#L284-L304) has both `index` and `event`, but the timeout branch passes `path: ["events"]`. The adjacent resize branch uses `["events", index]`. The [CLI application workflow](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/process/ProcessCli.ts#L63-L77) projects the schema issues before capture begins, so it faithfully propagates the missing event identity. Correcting the path at the schema boundary would address the producing source of the diagnostic. ### Additional context Fetched upstream `main` at `25e1a24a3ed3671a0148757555f18d16c29d936a` has no changes to the affected scenario schema, CLI workflow, input projection or shared contract files. This checks source freshness; that upstream runtime was not built or executed. #5923 concerns comparison divergence localization and #5916 concerns finite-trace duplicate admission; neither shares this contract or fix. #3423 concerns browser timeout documentation and #1253 concerns child finalization after timeout. MCP advertises the same process scenario schema, but MCP transport was not executed. Live capture, other platforms and the full suite were not run.

bsakaguchi closed this issue

2026-10-31 00:42:35 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471