morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7598] [Bug] Environment recording instructions require a deadline that pause steps reject #4529

New issue

Closed

opened 2026-10-31 00:50:31 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:50:31 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7598

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Documentation

Observed behavior

Following the environment recording guide's rule to add timeout_ms to every action conflicts with its recommended 1100 ms pause. With both duration_ms: 1100 and timeout_ms: 5000 on that pause, node scripts/rea.mjs validate-browser-scenario <scenario.json> --json exits 1 and reports invalid_request at actions[0].timeout_ms, with reason unknown_argument.

I independently observed that deleting only timeout_ms makes the same scenario pass validation and exit 0. The validator correctly enforces its schema; the guide gives an instruction that cannot be applied to the pause it recommends.

Reproduction

Use the already-built baseline identified below, with Node.js 24.22.1 on PATH. From the checkout root, run this inline fixture builder and public CLI comparison. It gives REA isolated HOME/XDG directories and removes them and the JSON fixture afterward. It requires no browser or running application.

python3 -I <<'PYTHON'
import json, os, pathlib, subprocess, tempfile

with tempfile.TemporaryDirectory() as directory:
    root = pathlib.Path(directory)
    environment = {"PATH": os.environ["PATH"], "NO_COLOR": "1"}
    for variable in ("HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "XDG_STATE_HOME", "TMPDIR"):
        folder = root / variable.lower()
        folder.mkdir()
        environment[variable] = str(folder)
    scenario = {
        "browser": {"mode": "launch", "executable_path": "/usr/bin/chromium"},
        "start_url": {"url": "http://127.0.0.1:3000/"},
        "actions": [{"step_id": "settle", "action": "wait_for_timeout", "duration_ms": 1100, "timeout_ms": 5000}],
    }
    fixture = root / "scenario.json"
    for label in ("with timeout_ms", "without timeout_ms"):
        fixture.write_text(json.dumps(scenario))
        result = subprocess.run(
            ["node", "scripts/rea.mjs", "validate-browser-scenario", str(fixture), "--json"],
            env=environment, capture_output=True, text=True, timeout=30,
        )
        print(label, "exit", result.returncode)
        print(result.stdout.strip())
        if result.stderr:
            print(result.stderr.strip())
        scenario["actions"][0].pop("timeout_ms", None)
PYTHON

Observe with timeout_ms exit 1, followed by an invalid-input diagnostic whose issue path is ["actions",0,"timeout_ms"] and reason is unknown_argument.
Observe without timeout_ms exit 0, followed by {"valid":true,"actions":1,"secrets":0}.

Both variants were independently run three times each: two initial comparisons and one execution of the exact inline script above. All three pairs produced these outcomes.

Expected behavior

The environment guide should give instructions that form a valid scenario when combined: action deadlines where the action accepts them, and the declared duration for a fixed pause. A caller adding the recommended pause between creating and editing an object should not need to discover an undocumented exception to the guide's universal deadline rule.

The canonical schema deliberately gives pause actions a duration without an action-deadline field; the corresponding source and documentation are cited below. This report requests a correction to the guide, with no demonstrated need to change the validator.

Impact and scope

The demonstrated result is rejection of a scenario assembled by following both instructions. Removing the pause's timeout_ms restores validation while preserving duration_ms: 1100. No browser capture, application write, or data-loss behavior was tested. The frequency with which agents encounter this instruction conflict is unknown.

Impact level: P2
Assessment status: supported
Impact basis: The documentation directs callers to construct input that the public validator rejects, blocking admission of the intended scenario.
Workaround status: available
Workaround basis: Omitting only the pause deadline passed the same validator in all three control runs; the finite requested duration remains present.

REA version

Exact output of node scripts/rea.mjs --version:

6.8.0

Source baseline: 65f1c447344a423fa2785847dbcea1e9a9e2454a.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1; Darwin arm64. The reproduction invokes only offline validation and uses no browser or native analysis provider.

Target and artifact context

A locally generated scenario JSON file containing a single fixed-duration pause. The browser path and loopback URL are declarative inputs; validation does not need either resource to exist.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

The failing public CLI invocation exited 1 with empty stderr. Selected fields from its JSON result:

{"error":"Browser scenario validation failed","code":"invalid_request","category":"invalid_input","details":{"operation":"capture_browser_scenario","issues":[{"path":["actions",0,"timeout_ms"],"reason":"unknown_argument"}]}}

Deleting only the pause deadline yielded exit 0, empty stderr, and the complete result:

{"valid":true,"actions":1,"secrets":0}

Source checked at the stated baseline:

.agents/skills/reverse-engineer-anything/references/environments.md:119-121 says “Give every step a timeout_ms.” Its numbered workflow at lines 146-147 recommends wait_for_timeout with duration_ms: 1100 before the first edit.
src/domain/browserScenarioValues.ts:452-462 defines the strict pause shape with only step_id, action, and duration_ms. Other actions use the optional deadline defined at lines 355-367.
docs/browser-scenario-contract.md:82-93 shows a pause with only a duration. Lines 158-161 distinguish action deadlines and pause durations.
docs/browser-scenario-contract.md:54-75 documents the offline validator's shared schema, unknown-field rejection, exit codes, and absence of provider acquisition. src/cliBrowserScenarioCommands.ts:104-165 implements that supported path through the canonical scenario parser.

Typed outcome or recovery expected

Narrow the environment guide's deadline instruction so it expressly excludes fixed pauses, which use duration_ms. The guide's combined instructions should produce a scenario that passes the public validator.

Additional context

Related issue #3423 concerns older timeout wording and other documentation contracts; it does not report this guide instruction or the pause's unknown-field rejection. Closed #3392 concerns timer overflow, which is separate from the small-duration validation case here. Open/closed issue searches, PR searches including merged changes, and comment searches found no report of this same contradiction.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7598 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Documentation ### Observed behavior Following the environment recording guide's rule to add `timeout_ms` to every action conflicts with its recommended 1100 ms pause. With both `duration_ms: 1100` and `timeout_ms: 5000` on that pause, `node scripts/rea.mjs validate-browser-scenario <scenario.json> --json` exits 1 and reports `invalid_request` at `actions[0].timeout_ms`, with reason `unknown_argument`. I independently observed that deleting only `timeout_ms` makes the same scenario pass validation and exit 0. The validator correctly enforces its schema; the guide gives an instruction that cannot be applied to the pause it recommends. ### Reproduction 1. Use the already-built baseline identified below, with Node.js 24.22.1 on PATH. From the checkout root, run this inline fixture builder and public CLI comparison. It gives REA isolated HOME/XDG directories and removes them and the JSON fixture afterward. It requires no browser or running application. ```sh python3 -I <<'PYTHON' import json, os, pathlib, subprocess, tempfile with tempfile.TemporaryDirectory() as directory: root = pathlib.Path(directory) environment = {"PATH": os.environ["PATH"], "NO_COLOR": "1"} for variable in ("HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "XDG_STATE_HOME", "TMPDIR"): folder = root / variable.lower() folder.mkdir() environment[variable] = str(folder) scenario = { "browser": {"mode": "launch", "executable_path": "/usr/bin/chromium"}, "start_url": {"url": "http://127.0.0.1:3000/"}, "actions": [{"step_id": "settle", "action": "wait_for_timeout", "duration_ms": 1100, "timeout_ms": 5000}], } fixture = root / "scenario.json" for label in ("with timeout_ms", "without timeout_ms"): fixture.write_text(json.dumps(scenario)) result = subprocess.run( ["node", "scripts/rea.mjs", "validate-browser-scenario", str(fixture), "--json"], env=environment, capture_output=True, text=True, timeout=30, ) print(label, "exit", result.returncode) print(result.stdout.strip()) if result.stderr: print(result.stderr.strip()) scenario["actions"][0].pop("timeout_ms", None) PYTHON ``` 2. Observe `with timeout_ms exit 1`, followed by an invalid-input diagnostic whose issue path is `["actions",0,"timeout_ms"]` and reason is `unknown_argument`. 3. Observe `without timeout_ms exit 0`, followed by `{"valid":true,"actions":1,"secrets":0}`. Both variants were independently run three times each: two initial comparisons and one execution of the exact inline script above. All three pairs produced these outcomes. ### Expected behavior The environment guide should give instructions that form a valid scenario when combined: action deadlines where the action accepts them, and the declared duration for a fixed pause. A caller adding the recommended pause between creating and editing an object should not need to discover an undocumented exception to the guide's universal deadline rule. The canonical schema deliberately gives pause actions a duration without an action-deadline field; the corresponding source and documentation are cited below. This report requests a correction to the guide, with no demonstrated need to change the validator. ### Impact and scope The demonstrated result is rejection of a scenario assembled by following both instructions. Removing the pause's `timeout_ms` restores validation while preserving `duration_ms: 1100`. No browser capture, application write, or data-loss behavior was tested. The frequency with which agents encounter this instruction conflict is unknown. - Impact level: P2 - Assessment status: supported - Impact basis: The documentation directs callers to construct input that the public validator rejects, blocking admission of the intended scenario. - Workaround status: available - Workaround basis: Omitting only the pause deadline passed the same validator in all three control runs; the finite requested duration remains present. ### REA version Exact output of `node scripts/rea.mjs --version`: ```text 6.8.0 ``` Source baseline: `65f1c447344a423fa2785847dbcea1e9a9e2454a`. ### Runtime and platform Node.js `v24.22.1`; macOS 27.0.1; Darwin arm64. The reproduction invokes only offline validation and uses no browser or native analysis provider. ### Target and artifact context A locally generated scenario JSON file containing a single fixed-duration pause. The browser path and loopback URL are declarative inputs; validation does not need either resource to exist. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence The failing public CLI invocation exited 1 with empty stderr. Selected fields from its JSON result: ```json {"error":"Browser scenario validation failed","code":"invalid_request","category":"invalid_input","details":{"operation":"capture_browser_scenario","issues":[{"path":["actions",0,"timeout_ms"],"reason":"unknown_argument"}]}} ``` Deleting only the pause deadline yielded exit 0, empty stderr, and the complete result: ```json {"valid":true,"actions":1,"secrets":0} ``` Source checked at the stated baseline: - `.agents/skills/reverse-engineer-anything/references/environments.md:119-121` says “Give every step a `timeout_ms`.” Its numbered workflow at lines 146-147 recommends `wait_for_timeout` with `duration_ms: 1100` before the first edit. - `src/domain/browserScenarioValues.ts:452-462` defines the strict pause shape with only `step_id`, `action`, and `duration_ms`. Other actions use the optional deadline defined at lines 355-367. - `docs/browser-scenario-contract.md:82-93` shows a pause with only a duration. Lines 158-161 distinguish action deadlines and pause durations. - `docs/browser-scenario-contract.md:54-75` documents the offline validator's shared schema, unknown-field rejection, exit codes, and absence of provider acquisition. `src/cliBrowserScenarioCommands.ts:104-165` implements that supported path through the canonical scenario parser. ### Typed outcome or recovery expected Narrow the environment guide's deadline instruction so it expressly excludes fixed pauses, which use `duration_ms`. The guide's combined instructions should produce a scenario that passes the public validator. ### Additional context Related issue [#3423](https://github.com/morluto/rea/issues/3423) concerns older timeout wording and other documentation contracts; it does not report this guide instruction or the pause's unknown-field rejection. Closed [#3392](https://github.com/morluto/rea/issues/3392) concerns timer overflow, which is separate from the small-duration validation case here. Open/closed issue searches, PR searches including merged changes, and comment searches found no report of this same contradiction.

bsakaguchi closed this issue

2026-10-31 00:50:31 +00:00

bsakaguchi referenced this issue

2026-10-31 01:09:28 +00:00

[PR #3513] fix(browser): keep an undecodable Electron worker target uncertain #5792

bsakaguchi referenced this issue

2026-10-31 01:13:19 +00:00

[PR #4529] [CLOSED] fix(electron): exclude scripts from unauthorized frames #6289

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425