morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #7116] [CLOSED] feat(browser): validate browser scenarios with --dry-run before launching #7652

New issue

Closed

opened 2026-11-03 13:39:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:23 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7116
Author: @kaoru0822-kitauji
Created: 10/25/2026
Status: ❌ Closed

Base: main ← Head: feat/browser-scenario-dry-run

📝 Commits (2)

c3b79ce feat(browser): validate browser scenarios with --dry-run before launching
4eaf0aa docs(browser): fence the dry-run plan example as output text

📊 Changes

6 files changed (+381 additions, -34 deletions)

View changed files

📝 docs/browser-scenario-contract.md (+45 -0)
📝 src/application/BrowserScenarioCaptureService.ts (+62 -2)
📝 src/browser/BrowserScenarioSecrets.ts (+11 -28)
📝 src/cliBrowserScenarioCommands.ts (+24 -4)
📝 src/domain/browserScenario.ts (+38 -0)
➕ tests/boundary/cli/browserScenarioDryRun.test.ts (+201 -0)

📄 Description

What broke: The only way to check a browser scenario file was to run it. A run discovers, launches or attaches to a browser and may change page state before a typo, a duplicate step ID or a missing secret variable is reported.
Fix: rea capture-browser-scenario <json> --dry-run applies the same JSON and path resolution, the same canonical schema and semantic rules, and the same secret-availability rule as a capture, all through shared owners. It then stops before any provider is created and returns either the same projected input error or a compact plan.
Risk: Low. The flag is opt-in. Without it the command is unchanged. The secret-availability rule moved verbatim from BrowserScenarioSecrets.resolve into the domain, and capture still uses it.

Summary

Fixes #5488

A dry run reports schema and semantic errors with field paths, using the exact projection a capture returns. On success it returns the browser mode, the start origin, every step in capture order, each declared secret with only environment_value: "present", and an explicit not_checked list. It creates no browser provider, profile, file or connection, and it never reads a secret value into its output.

Problem and expected behavior

Before: invalid input was reported only by running the command. A valid scenario immediately started browser acquisition.
Expected (#5488): an explicit validation-only workflow using the same rules as execution, reporting errors with field paths, catching duplicate step IDs and undeclared secret references, exiting successfully without a browser, never printing secret values, and with boundary tests showing that no browser or provider factory is invoked.

Change and scope

src/domain/browserScenario.ts: new browserScenarioSecretAvailability. This is the presence rule capture applied at session open: the variable must be an own property with a string value, and an empty string counts as set. It is moved from BrowserScenarioSecrets.resolve, which now calls it and then reads the values, so the rule has one owner and both paths return the same error.
src/application/BrowserScenarioCaptureService.ts: planBrowserScenario(scenario, environment) builds the plan from an already parsed scenario.
src/cliBrowserScenarioCommands.ts: adds the --dry-run option. Argument parsing, resolveCliJsonPaths and browserScenarioSchema are the same code path as a capture. The branch returns before the Playwright composition is imported or createProvider is called. Failures are labelled Browser scenario validation failed, with the same projected body.
Docs: a "Validate without a browser" section in docs/browser-scenario-contract.md, with an example plan.
Not included: an MCP tool. It would add catalog tokens for every agent, and MCP capture_browser_scenario already rejects schema-invalid input and missing secrets before acquiring a browser. Executable, endpoint and locator checks are also excluded, because they need discovery or a page; they are listed in not_checked instead.

Contribution scope: not applicable. This is an opt-in, read-only mode of an existing command that adds no target interaction.

Contract and boundary impact

Semantic owner and earliest changed stage: the scenario schema (unchanged) and the secret-availability rule, now owned by the domain. BrowserScenarioSecrets is its consumer.
Contribution scope fit and linked maintainer decision: not applicable.
CLI and MCP/tool-catalog contract: the CLI gains --dry-run. MCP is unchanged.
Provider, bridge, target-format, or platform compatibility: none. The dry run never loads Playwright.
Evidence, artifact, provenance, or reconstruction contract: a dry run produces no Evidence. Its plan is not Evidence and claims nothing about the target.
Target selection, authority, upstream behavior, and cleanup details: no browser, profile, process or network access, and nothing to clean up.
Generated metadata, package, or installation impact: none. metadata:check passes.
Documentation or usability only; no target interaction.

Evidence and regression coverage

Tests added (tests/boundary/cli/browserScenarioDryRun.test.ts):
- In process, with a createProvider spy that throws: a valid dry run returns the exact plan, the stdout never contains the secret value, and the spy is never called.
- A missing secret variable returns exactly projectAnalysisError(BrowserScenarioSecrets.resolve(...).error), which is the capture rule's error, at ["secrets", 0, "environment_variable"]. An inherited (prototype) variable also counts as missing.
- Duplicate step IDs plus an undeclared secret reference produce a projection identical to the non-dry-run command; only the error label differs. No provider is created in either mode.
- Without the flag, the provider factory is called once, so execution is unchanged.
- Through the compiled CLI process, with a nonexistent browser executable, a valid dry run exits 0 and never prints the secret value, and a missing secret exits non-zero with the field path.
Existing scenario suites still pass, including the BrowserScenarioSecrets producer tests.

User-visible CLI output (built CLI, on this branch):

$ REA_DEMO_TOKEN=… rea capture-browser-scenario scenario.json --dry-run --json   # exit 0
{"operation":"capture_browser_scenario","dry_run":true,"browser_mode":"launch","start_origin":"https://example.com","steps":[{"step_id":"scenario_start","action":"goto_start"},{"step_id":"open_account","action":"goto"}],"secrets":[{"secret_id":"api_token","environment_variable":"REA_DEMO_TOKEN","environment_value":"present"}],"not_checked":["browser executable, CDP endpoint and target discovery","browser launch or attachment","navigation, network reachability and page state","locator resolution, action outcomes and capture limits"]}
$ rea capture-browser-scenario scenario.json --dry-run --json   # exit 1
{"error":"Browser scenario validation failed","code":"invalid_request",…,"details":{"operation":"capture_browser_scenario","issues":[{"path":["secrets",0,"environment_variable"],"reason":"missing_argument",…}]}}

Remaining proof gaps: no real browser capture was run after the BrowserScenarioSecrets.resolve refactor, because no browser is installed on this host. The refactor is covered by the existing producer tests and by the equality assertion above.

Boundary regressions: input parsing and projection parity between modes, secret-environment admission (missing, inherited, present) and provider non-creation. There is no lifecycle stage, because no resource is acquired.

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npm run test:focused -- tests/boundary/cli/browserScenarioDryRun.test.ts tests/boundary/cli/scenarioCliInputDiagnostics.test.ts tests/boundary/cli/cliJsonCallerPath.test.ts tests/process-global/browserScenarioCleanup.test.ts tests/boundary/mcp/browserScenarioInputDiagnosticsMcp.test.ts: 23 passed.
npm run test:local -- src/browser/BrowserScenario.test.ts src/browser/PlaywrightScenarioRequest.test.ts: 51 passed.
npm run check:fast: passed.
npm run format:check: passed.
npm run docs:check: passed.
npm run metadata:check: passed.
Manual run of the built CLI: see the output above.

Compatibility, safety, and release

Breaking changes or migration steps: none.
Real Hopper/Ghidra, browser, or OS coverage: no browser by design; macOS arm64.
Package or release metadata impact: none.
Security, privacy, process, or containment review: the dry run checks only whether secret variables are present. Values are never copied into the plan, and tests assert they never reach stdout.

Author readiness

Human owner: @kaoru0822-kitauji

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7116 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/25/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/browser-scenario-dry-run` --- ### 📝 Commits (2) - [`c3b79ce`](https://github.com/morluto/rea/commit/c3b79cebe62f19f9aade36333772c45512314bc2) feat(browser): validate browser scenarios with --dry-run before launching - [`4eaf0aa`](https://github.com/morluto/rea/commit/4eaf0aae7312728baba0774859bf7fcc3316365e) docs(browser): fence the dry-run plan example as output text ### 📊 Changes **6 files changed** (+381 additions, -34 deletions) <details> <summary>View changed files</summary> 📝 `docs/browser-scenario-contract.md` (+45 -0) 📝 `src/application/BrowserScenarioCaptureService.ts` (+62 -2) 📝 `src/browser/BrowserScenarioSecrets.ts` (+11 -28) 📝 `src/cliBrowserScenarioCommands.ts` (+24 -4) 📝 `src/domain/browserScenario.ts` (+38 -0) ➕ `tests/boundary/cli/browserScenarioDryRun.test.ts` (+201 -0) </details> ### 📄 Description **What broke:** The only way to check a browser scenario file was to run it. A run discovers, launches or attaches to a browser and may change page state before a typo, a duplicate step ID or a missing secret variable is reported. **Fix:** `rea capture-browser-scenario <json> --dry-run` applies the same JSON and path resolution, the same canonical schema and semantic rules, and the same secret-availability rule as a capture, all through shared owners. It then stops before any provider is created and returns either the same projected input error or a compact plan. **Risk:** Low. The flag is opt-in. Without it the command is unchanged. The secret-availability rule moved verbatim from `BrowserScenarioSecrets.resolve` into the domain, and capture still uses it. ## Summary Fixes #5488 A dry run reports schema and semantic errors with field paths, using the exact projection a capture returns. On success it returns the browser mode, the start origin, every step in capture order, each declared secret with only `environment_value: "present"`, and an explicit `not_checked` list. It creates no browser provider, profile, file or connection, and it never reads a secret value into its output. ## Problem and expected behavior - Before: invalid input was reported only by running the command. A valid scenario immediately started browser acquisition. - Expected (#5488): an explicit validation-only workflow using the same rules as execution, reporting errors with field paths, catching duplicate step IDs and undeclared secret references, exiting successfully without a browser, never printing secret values, and with boundary tests showing that no browser or provider factory is invoked. ## Change and scope - `src/domain/browserScenario.ts`: new `browserScenarioSecretAvailability`. This is the presence rule capture applied at session open: the variable must be an own property with a string value, and an empty string counts as set. It is moved from `BrowserScenarioSecrets.resolve`, which now calls it and then reads the values, so the rule has one owner and both paths return the same error. - `src/application/BrowserScenarioCaptureService.ts`: `planBrowserScenario(scenario, environment)` builds the plan from an already parsed scenario. - `src/cliBrowserScenarioCommands.ts`: adds the `--dry-run` option. Argument parsing, `resolveCliJsonPaths` and `browserScenarioSchema` are the same code path as a capture. The branch returns before the Playwright composition is imported or `createProvider` is called. Failures are labelled `Browser scenario validation failed`, with the same projected body. - Docs: a "Validate without a browser" section in `docs/browser-scenario-contract.md`, with an example plan. - Not included: an MCP tool. It would add catalog tokens for every agent, and MCP `capture_browser_scenario` already rejects schema-invalid input and missing secrets before acquiring a browser. Executable, endpoint and locator checks are also excluded, because they need discovery or a page; they are listed in `not_checked` instead. Contribution scope: not applicable. This is an opt-in, read-only mode of an existing command that adds no target interaction. ## Contract and boundary impact - Semantic owner and earliest changed stage: the scenario schema (unchanged) and the secret-availability rule, now owned by the domain. `BrowserScenarioSecrets` is its consumer. - Contribution scope fit and linked maintainer decision: not applicable. - CLI and MCP/tool-catalog contract: the CLI gains `--dry-run`. MCP is unchanged. - Provider, bridge, target-format, or platform compatibility: none. The dry run never loads Playwright. - Evidence, artifact, provenance, or reconstruction contract: a dry run produces no Evidence. Its plan is not Evidence and claims nothing about the target. - Target selection, authority, upstream behavior, and cleanup details: no browser, profile, process or network access, and nothing to clean up. - Generated metadata, package, or installation impact: none. `metadata:check` passes. - [x] Documentation or usability only; no target interaction. ## Evidence and regression coverage - Tests added (`tests/boundary/cli/browserScenarioDryRun.test.ts`): - In process, with a `createProvider` spy that throws: a valid dry run returns the exact plan, the stdout never contains the secret value, and the spy is never called. - A missing secret variable returns exactly `projectAnalysisError(BrowserScenarioSecrets.resolve(...).error)`, which is the capture rule's error, at `["secrets", 0, "environment_variable"]`. An inherited (prototype) variable also counts as missing. - Duplicate step IDs plus an undeclared secret reference produce a projection identical to the non-dry-run command; only the `error` label differs. No provider is created in either mode. - Without the flag, the provider factory is called once, so execution is unchanged. - Through the compiled CLI process, with a nonexistent browser executable, a valid dry run exits 0 and never prints the secret value, and a missing secret exits non-zero with the field path. - Existing scenario suites still pass, including the `BrowserScenarioSecrets` producer tests. - User-visible CLI output (built CLI, on this branch): ```text $ REA_DEMO_TOKEN=… rea capture-browser-scenario scenario.json --dry-run --json # exit 0 {"operation":"capture_browser_scenario","dry_run":true,"browser_mode":"launch","start_origin":"https://example.com","steps":[{"step_id":"scenario_start","action":"goto_start"},{"step_id":"open_account","action":"goto"}],"secrets":[{"secret_id":"api_token","environment_variable":"REA_DEMO_TOKEN","environment_value":"present"}],"not_checked":["browser executable, CDP endpoint and target discovery","browser launch or attachment","navigation, network reachability and page state","locator resolution, action outcomes and capture limits"]} $ rea capture-browser-scenario scenario.json --dry-run --json # exit 1 {"error":"Browser scenario validation failed","code":"invalid_request",…,"details":{"operation":"capture_browser_scenario","issues":[{"path":["secrets",0,"environment_variable"],"reason":"missing_argument",…}]}} ``` - Remaining proof gaps: no real browser capture was run after the `BrowserScenarioSecrets.resolve` refactor, because no browser is installed on this host. The refactor is covered by the existing producer tests and by the equality assertion above. Boundary regressions: input parsing and projection parity between modes, secret-environment admission (missing, inherited, present) and provider non-creation. There is no lifecycle stage, because no resource is acquired. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:focused -- tests/boundary/cli/browserScenarioDryRun.test.ts tests/boundary/cli/scenarioCliInputDiagnostics.test.ts tests/boundary/cli/cliJsonCallerPath.test.ts tests/process-global/browserScenarioCleanup.test.ts tests/boundary/mcp/browserScenarioInputDiagnosticsMcp.test.ts`: 23 passed. - `npm run test:local -- src/browser/BrowserScenario.test.ts src/browser/PlaywrightScenarioRequest.test.ts`: 51 passed. - `npm run check:fast`: passed. - `npm run format:check`: passed. - `npm run docs:check`: passed. - `npm run metadata:check`: passed. - Manual run of the built CLI: see the output above. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: no browser by design; macOS arm64. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the dry run checks only whether secret variables are present. Values are never copied into the plan, and tests assert they never reach stdout. ## Author readiness Human owner: @kaoru0822-kitauji - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:39:23 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7652

No description provided.

Rows
Columns