morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #7162] [CLOSED] feat(browser): save values in browser scenarios and send them in later steps #7687

New issue

Closed

opened 2026-11-03 13:39:39 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:39 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7162
Author: @earakely-scale
Created: 10/25/2026
Status: ❌ Closed

Base: main ← Head: feat/recorder-saved-values

📝 Commits (10+)

c570016 feat(browser): save request values and send Basic credentials
51cf0c3 Merge main into the recorder saved-values branch
d2e9b68 feat(browser): save values from UI steps and send them as {saved:name}
6984068 docs(skill): show a UI-shaped create in the recording example
583e104 Merge remote-tracking branch 'origin/main' into feat/recorder-saved-values
48a4543 docs(spec): record saved values in browser scenarios
31d6e1d fix(environment): learn addresses a UI-made create spells, and check templates on later creates
cf40db8 docs: record the write-model fixes and fresh recordings; ask agents to clear earlier attempts
8218b0b Merge remote-tracking branch 'origin/main' into feat/recorder-saved-values
812f9ff docs(spec): record the gates after merging main

📊 Changes

37 files changed (+3941 additions, -215 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+1 -1)
📝 .agents/skills/reverse-engineer-anything/references/environments.md (+68 -26)
📝 docs/agent-environments.md (+4 -1)
📝 docs/browser-scenario-contract.md (+118 -1)
➕ docs/specs/agent-environments/scenario-saved-values/plan.md (+22 -0)
➕ docs/specs/agent-environments/scenario-saved-values/spec.md (+81 -0)
➕ docs/specs/agent-environments/scenario-saved-values/tasks.md (+15 -0)
➕ docs/specs/agent-environments/scenario-saved-values/verification.md (+132 -0)
📝 docs/web-network-captures.md (+28 -5)
➕ src/application/BrowserScenarioEvidence.test.ts (+63 -0)
📝 src/application/BrowserScenarioEvidence.ts (+19 -4)
📝 src/application/environment/environmentBuild.ts (+5 -1)
📝 src/application/environment/environmentBuildReport.test.ts (+52 -0)
📝 src/application/environment/environmentBuildReport.ts (+52 -0)
📝 src/application/environment/environmentStateAssignedCoverage.test.ts (+37 -16)
📝 src/browser/BrowserScenarioSecrets.ts (+122 -6)
📝 src/browser/PlaywrightScenarioActions.ts (+62 -16)
📝 src/browser/PlaywrightScenarioRequest.test.ts (+1181 -2)
📝 src/browser/PlaywrightScenarioRequest.ts (+507 -23)
📝 src/domain/analysisView/EnvironmentStateWriteSimulation.ts (+1 -0)

...and 17 more files

📄 Description

Summary

An agent that builds an environment from a prompt records the application with one browser scenario. Until now a
scenario could not carry a value from one step to a later one:

No ids from responses. After creating an object, the agent could not open, read, edit or delete it by the id the
application assigned. Agents chose their own ids where the API allows it, or read lists back instead.
No derived auth headers. An API that takes Basic auth, or a header a frontend builds from a login token, could not
be sent. Miniflux's agent stopped and asked for an API key.

This PR lets steps save values and later steps send them:

A request step saves [{name, json_pointer, secret}] from its JSON response. A click or press step saves from
the response to the request it triggers, with save_response: {method, path, save}, so a create made through the
UI yields its id.
{saved:name} sends a saved value in a request path (and its query), a goto URL, and a JSON body, where a string that
is only the placeholder keeps the saved JSON type. Header values and Basic credential parts take
{"source": "saved", "name"}.
A header can send Basic credentials, {"source": "basic_credentials", "parts": [username, password]}, each part a
literal, declared secret, saved value, cookie or local storage entry.
The skill records writes the way the UI does, saving ids from the responses its clicks trigger.
The write model serves UI-made creates as faithfully as API-made ones. Recording through the UI let Grafana generate
folder uids, which exposed three gaps; this PR closes them (below).

Results, agents building seven applications from the same prompts, with main and with this branch:

Served writes go from 9 to 24, and write tasks from 4/20 to 8/20 and 9/20 with the two models. Miniflux goes from no
environment to 3/3, and Vikunja from 0/5 to 3/5.
Rebuilding the branch's own recordings with the write-model fixes changes only Grafana's plans, whose tasks go from
0/2 to 2/2.
The fresh Grafana and EspoCRM recordings scored below main's, for reasons in what was recorded (below).

Problem and expected behavior

A step should save a string or number from a JSON response, by name, and a later step should send it in a path, a
URL, a JSON body or a header.
A UI step that submits a form should save from the response to the request it triggers.
A header should carry Basic credentials built from any value kind.
A scenario that uses none of this should record and hash as before, and earlier REA versions should read captures that
use it.

Change and scope

Schema (browserScenarioValues.ts, browserScenario.ts):
- save on request steps, save_response on click and press, the saved value source in headers and credential
  parts, basic_credentials with two parts, and {saved:name} (a parsed URL's percent-encoded form included).
- Validation refuses a saved value no earlier step saves. Other braces, such as {id}, are sent as written.
Runtime (PlaywrightScenarioRequest.ts, PlaywrightScenarioActions.ts, BrowserScenarioSecrets.ts):
- A saving request step reads its response text, up to 4,000,000 characters, in the page.
- A UI step's save_response starts waiting before the step acts and fails the step unless a matching 2xx JSON
  response arrives before its timeout.
- Saved values keep their JSON type and fill placeholders percent-encoded in paths and URLs.
- Basic credentials are base64-encoded and redacted, as are parts read from the page. A value saved with
  secret: true is redacted from later records.
Evidence (BrowserScenarioEvidence.ts): an empty save is left out of the recorded step, so parameters and
scenario_sha256 are unchanged for scenarios that save nothing.
Environment build: warns about saved values named like credentials that are not marked secret
(credential-named-saved-values).
Skill (version 50):
- Writes the way the UI does and saves ids instead of guessing them.
- Falls back to request steps when the UI cannot reach an operation or coverage withholds the UI-shaped write.
- Its recording example creates through the UI.
Write model (src/domain/analysisView):
- A composed string that reads a field besides the generated value a template would rebuild it around wins over
  the template, so Grafana's /dashboards/f/{uid}/{slug(title)} is composed when the uid is generated.
- Later creates check template fields against their answers, as they check composed ones, so coverage no longer
  serves a template a later create contradicts.
- A slug learns how the application spells &: ampersand: "and" on a slug rule or composed part, learned from the
  call that shows it. Calls that slugged no & agree with either spelling.
- Where one text could come from two sources (an id equal to the title's slug), the entities the type holds pick the
  one reading they share. When they cannot, a later create that tells the fields apart lends its composed rule to
  the first create, whose body contract is learned again with it.
Skill: also asks agents to delete what an earlier attempt created before recording again, since the environment
holds a leftover as seeded data.
Docs and spec: the browser scenario contract's saved values section, the environment and web capture references,
and docs/specs/agent-environments/scenario-saved-values/.

Not included:

general value transforms or script steps;
saved values in fill and select_option;
multi-factor sign-in;
lists sorted by a field an update changes, which REA keeps in place (pre-existing; seen in Grafana's fresh run).

Contract and boundary impact

Semantic owner and earliest changed stage: the browser scenario contract (src/domain/browserScenarioValues.ts)
and its Playwright runtime.
Contribution scope fit and linked maintainer decision: within the scenario request steps of #6946. Part of #4933.
CLI and MCP/tool-catalog contract:
- capture_browser_scenario accepts save, save_response, the saved source, basic_credentials and
  {saved:name}.
- The compact catalog grows by 3,664 bytes, from 1,038,975 on main 2b84efe0 to 1,042,639, under the 1 MiB frame
  outputSchemaProfile.test.ts measures. A first design with path_params and json_values fields failed that test.
- save_response.path uses the request path's portable pattern, now shared as START_ORIGIN_PATH.
Provider, bridge, target-format, or platform compatibility: unchanged.
Evidence, artifact, provenance, or reconstruction contract:
- Scenarios that save nothing record the same parameters and scenario_sha256.
- Captures that save values parse on earlier REA versions.
- environment-state views may carry ampersand on slug rules and parts; earlier REA versions refuse views that do,
  as the schemas are strict. Earlier views parse unchanged.
Target selection, authority, upstream behavior, and cleanup details:
- A UI step with save_response waits for a response on the start origin.
- Request steps stay same-origin, as before.
Generated metadata, package, or installation impact: the skill version goes from 49 to 50.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Agent-built environments. Claude Code (claude-sonnet-5-5) built each environment from the same prompt with main
ff6d96bc (skill 49) and with this branch, on freshly seeded instances. Each bundle's write tasks ran with
fireworks_ai/gpt-oss-120b and fireworks_ai/glm-5p3-flash, each checking the result from the environment's state.

The branch's recordings, rebuilt before and after the write-model fixes (gpt-oss, GLM). Only Grafana's plans change:

Application	Before	After
Miniflux	3/3 3/3	3/3 3/3
Paperless	1/3 2/3	1/3 1/3
Vikunja	1/3 1/3 + 0/2	1/3 1/3 + 0/2
Linkding	0/2 0/2	0/2 0/2
Homebox	0/2 0/2	0/2 0/2
Grafana	0/2 0/2	2/2 2/2
EspoCRM	1/3 1/3	1/3 1/3

Paperless's plans are identical in both builds; its GLM difference is the eval model's.

Fresh recordings, main and the final branch:

Application	Write tasks	Main: served, tasks	Branch: served, tasks
Miniflux	3	no bundle	4, 3/3 3/3
Paperless	3	3, 1/3 1/3; 6, 2/3 2/3	6, 1/3 2/3
Vikunja	3 + 2	1, 0/3 0/3 + 0/2 0/2	3, 1/3 1/3 + 2/2 2/2
Linkding	2	0, 0/2 0/2	3, 0/2 0/2
Homebox	2	1, 0/2 0/2	5, 0/2 0/2
Grafana	2	2, 2/2 2/2; 3, 2/2 2/2	1, 1/2 1/2
EspoCRM	3	2, 1/3 1/3	2, 0/3 0/3

"Served" counts the write tools the final capture serves; tasks are gpt-oss then GLM. Paperless and Grafana were
recorded twice on main.

Miniflux: main's agent stopped for an API key; with Basic credentials the branch serves category create, rename and
delete, and every category task passed in all three branch recordings.
Saved values in use: every branch agent saved ids, with save_response on UI writes.
How the branch got here. The first branch recording showed that EspoCRM's agent copied the skill's request-step
example and wrote through the API, which EspoCRM answers in a shape the write model withholds (0/3); the example now
creates through the UI. The second showed Grafana's UI-made creates served with wrong addresses (0/2); the write-model
fixes above close that.
Grafana, fresh: the create is served with the right address and passes. The rename is withheld: Grafana lists
folders sorted by title, the recording renamed a folder to a title that sorts first, and REA keeps an updated entity
where it was in a list. Main's REA reads this recording the same way, and would serve its create with the template
that keeps the first folder's slug. Main's two recordings passed because their titles kept their order.
EspoCRM, fresh: the recording started with a contact an earlier attempt created and left, so the environment
holds it as seeded data and the "nothing else changed" checks fail. The skill now asks agents to delete such objects
before recording again; that was not measured.
Paperless: the two main runs differ as much as main and the branch do.
Linkding and Homebox serve more writes, but their create tasks still fail. Linkding's served create requires every
member its recorded calls sent, so a task sending a URL and a title is refused.
Census: write plans for 109 captures (the 50 published captures, 29 earlier agent recordings and the 30 above),
built by main and by this branch: 524 plans and 247 served on each. Three plans differ, all Grafana folder creates,
whose url the branch composes from the uid and the title's slug.

Real Chrome, a local application with a cookie-session UI and an API that takes only Basic auth:

An 11-step scenario saved an id from a Save click and opened /items/{saved:item1}.
It sent Basic credentials from local storage and a secret, and saved a secret token.
It created a child with {"name": "Child of {saved:item1}", "ref": "{saved:item1}"}, which was sent as
{"name":"Child of 101","ref":101}.
The password appears nowhere in the capture, and the token header is recorded redacted.

Tests added:

schema:
- a saved value sent before it is saved, in a path, query, goto URL (percent-encoded too), JSON body, header or
  credential part;
- literal braces sent as written;
- credential parts from page values, an undeclared secret part, and a wrong part count;
- save_response paths and an empty save;
runtime with a fake page:
- ids sent in a path and a JSON body with their type, and a saved secret redacted;
- a missing pointer and a non-JSON response;
- Basic credentials from local storage and a cookie, encoded and redacted;
- a goto filled percent-encoded;
- save_response waiting before the click and matching method, origin and {*} segments, and failing without a
  matching or 2xx response;
Evidence: a scenario that saves nothing records and hashes as before; saved values and placeholders are recorded;
the credential-named-saved-values warning;
write model:
- a generated uid and the title's slug compose a folder's address;
- the & spelling is learned from a later create, a dropped & stays dropped and a create spelling it otherwise is
  withheld;
- an id equal to the title's slug is told apart by a later create, or by held entities;
- a template a later create answers otherwise is withheld;
- a copied member a composition rebuilds is accepted with any value, and one a template would keep stays fixed.
  Each fails on main.
Review round (each fails on 812f9ff2 unless noted):
- Basic credentials from one-letter secrets and a two-character secret save are redacted;
- a literal Basic password is refused, an empty one accepted;
- save_response ignores a request already running when the step began, matches My List against My%20List,
  refuses a response whose received size is over the cap without reading it, and stops listening when the step
  fails (passes on 812f9ff2, which never listened);
- a saving request step stops reading after the chunk that passes the cap;
- a goto keeps a structured query value that spells a placeholder;
- a slug field keeps a dropped & an accepted create showed, takes and from held tags, and learns no slug rule
  they contradict; a composed address takes and from held folders;
- many fields sharing one text leave an address unread (passes on 812f9ff2, which reached the same result with
  unbounded work).
Second review round (each fails on af3d5626, the merge of main before it):
- a saved integer past 2^53 fails the step instead of saving a rounded id;
- a JSON string spelling %7Bsaved:item%7D needs no save and is sent as written;
- a held slug neither & spelling gives keeps a create that slugs & from learning a slug rule, and leaves a create
  without & its plain rule.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Fifth review round at 9f5191ab0 (main ca3fd7ba merged), on macOS with Node v24.22.1:

6f6c61613 fixes Greptile's three findings on 22a93fadc: a saved secret is redacted as a URL's query spells it once
parsed (' as %27); a step's saved-value fills are counted before each is built against a 4,000,000-character budget
(percent-encoded in a path or goto URL), so repeated references fail the step before it sends; and a create whose
slug the held & spellings refuse accepts the slugged member only as recorded. Its six tests fail on 22a93fadc;
9f5191ab0 makes one of them check the plan's body is present first. 3de21ef5d merges main ca3fd7ba.
typecheck, check:fast, format:check, knip, docs:check, mcp-catalog:check, verify:test-discovery and
scan:todos: passed. domain + services + adapters: 18,758 passed, with the same two failures as main. Browser
boundary tests and webArtifactReadOwnership: 1,223 passed, with the two sazWebCapture failures main has.
Census on main ca3fd7ba and this head: 524 plans and 246 served on each (main's anchorless-clock change withholds
one Vikunja label update on both), with the same three Grafana creates differing.

Fourth review round at 22a93fadc (main dd7a2da5 merged), on macOS with Node v24.22.1:

The red CI on 81a275b13 came from main as merged for that run: 662ec2e45 left scripts/verify-apk-signature.mjs
unformatted (main fixed it in 741f2de75), and webArtifactReadOwnership.test.ts, which passes on this branch,
failed only in that merge. 723fe02c8 merges main dd7a2da5.
22a93fadc decodes a saving response as fatal UTF-8 in both save paths, so a malformed byte fails the step instead of
saving U+FFFD, and bounds a secret save's spellings from the value before building any (Codex follow-ups). Its four
tests fail on 723fe02c8.
typecheck, check:fast, format:check, knip, docs:check, mcp-catalog:check, verify:test-discovery and
scan:todos: passed. domain + services + adapters: 18,673 passed; the Inspector stat test and
SetupRegistrationPrewarm's detached-descendant test fail, as they do on main dd7a2da5. Browser boundary tests and
webArtifactReadOwnership: 1,223 passed; two sazWebCapture tests fail, as they do on main.
Census on main dd7a2da5 and this head: 524 plans and 247 served on each, with the same three Grafana creates
differing.

Third review round at 69f38a78b (main a30ffd27 merged), on macOS with Node v24.22.1:

7b2e78077 then checks each filled path segment after substitution, so a saved . that the text beside it turns into a
dot segment fails the step (Greptile's follow-up); its tests fail on 69f38a78b, check:fast passes and src/browser/
passes 1,043 tests.
a712f52e2 saves no number that is not finite (1e400), and d0d7c248b lets a later update refine an ambiguous slug
spelling as creates do (Codex follow-ups); their tests fail on 7b2e78077. check:fast, docs:check and 2,852 browser,
write-model and environment tests pass, and the 109-capture census stays at 524 plans and 247 served.
22ac315c3 keeps an update's word spelling of & for a later name without one and withholds a later create that omits
a composed or templated field; d39f386d2 caps the saved values a scenario keeps at 4,000,000 characters. Their tests
fail on d0d7c248b; check:fast, docs:check and 2,859 browser, write-model and environment tests pass, and the census
is unchanged at 524 plans and 247 served.
059322048 reads a saved response only once its decoded size is bounded (gzip/deflate by DEFLATE's maximum expansion;
br, zstd or an unknown size fail the step) and counts only the spellings a secret save adds; 81a275b13 saves no number
JSON parsing changes (1e-400, 1.0000000000000001) and spells an update's & as held entities do. Their tests fail on
the head each thread was reported on; check:fast, docs:check and 2,880 browser, write-model and environment tests
pass, and the census is unchanged at 524 plans and 247 served.
Fixes the five Codex findings on 98f18cc5: saved-response reads bounded by the step's deadline and abort signal; the
4,000,000-character cap kept, with a 12,000,000-byte bound before reading; Basic usernames with a colon refused;
dot-only saved path segments refused; mixed held & spellings leave no slug rule. It also makes the saved-values
example a whole scenario, as the product catalog test requires. Each new test fails on 98f18cc52.
tsc, check:fast, format:check, docs:check, mcp-catalog:check, verify:test-discovery, scan:todos and
jscpd: passed. knip reports only isToolFamilyId, as main does. domain + services + adapters: 18,201
passed, with the Inspector stat test failing as on main. Product catalog, output schema profile, MCP schema
validation, tool schema validity, the environment build and serve CLI, served-writes MCP and analysis-view CLI
boundaries: 8 files, 75 passed.
Census on main a30ffd27 and this head: 524 plans and 247 served on each, with the same three Grafana creates
differing; none of this round's fixes changes a plan.

Earlier rounds:

Run on macOS with Node v24.22.1 at e42362c9 (main 2b84efe0). Agent recordings ran on Ubuntu 24.04 VMs with snap
Chromium.

npm run typecheck, lint:built, format:check, metadata:check, verify:architecture-guards, knip:production,
lint:typed, docs:check, mcp-catalog:check, verify:test-discovery, scan:todos and jscpd: passed. knip
reports one unused export, isToolFamilyId in src/contracts/toolFamilies.ts, which this branch does not touch and
main 2b84efe0 reports too.
npx vitest run --project domain --project services --project adapters: 18,261 passed and 1 failed, "stops Inspector
target authorization after the current stat settles", which fails the same way on main in this environment.
npm run test:focused -- the browser scenario CLI and MCP boundaries, scenario input diagnostics, scenario
cancellation and cleanup, the analysis-view CLI and MCP boundaries, the environment build, export and serve CLI
boundaries, the environment serve, served-writes and bundle MCP boundaries, tool schema validity, MCP schema
validation, and output schema constraints and profile: 17 files, 128 passed.
Census on main 2b84efe0 and this branch: 524 plans and 247 served on each, with the same three Grafana creates
differing.
Scenarios that save nothing: identical parameters and scenario_sha256 on main and on this branch.
Main reads the real-Chrome capture: environment-build --dry-run gives the same manifest, state, coverage and file plan
on both.
After the first review round, the real-Chrome scenario, with its save_response path written as /ui/{*}, completed
all 12 steps with no password or raw token recorded.

Compatibility, safety, and release

Breaking changes or migration steps: none for callers.
Real Hopper/Ghidra, browser, or OS coverage: Google Chrome on macOS for the local check; snap Chromium on Ubuntu
24.04 for the agent recordings.
Package or release metadata impact: skill version 50.
Security, privacy, process, or containment review:
- Saved values are kept in memory for the scenario only.
- A value saved with secret: true, Basic credentials, and parts read from the page are redacted from later records.
- The response a secret was saved from may keep it, since the recorder reads bodies on their own schedule; the
  contract says so.
- The build warns about credential-named saved values not marked secret.

Author readiness

Human owner: @earakely-scale

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

Part of #4933.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7162 **Author:** [@earakely-scale](https://github.com/earakely-scale) **Created:** 10/25/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/recorder-saved-values` --- ### 📝 Commits (10+) - [`c570016`](https://github.com/morluto/rea/commit/c5700163845c0ab71d002fde0a1191f1f7ad53e2) feat(browser): save request values and send Basic credentials - [`51cf0c3`](https://github.com/morluto/rea/commit/51cf0c3ad211ca7535e9c177c57bec5b1380c7db) Merge main into the recorder saved-values branch - [`d2e9b68`](https://github.com/morluto/rea/commit/d2e9b6804380e87e58fcc30cd02cee03d35a52b1) feat(browser): save values from UI steps and send them as {saved:name} - [`6984068`](https://github.com/morluto/rea/commit/6984068bcb3cfa33a02b908c52174616ccfb77d9) docs(skill): show a UI-shaped create in the recording example - [`583e104`](https://github.com/morluto/rea/commit/583e10483739c4e526ba55786f6437cad47521b2) Merge remote-tracking branch 'origin/main' into feat/recorder-saved-values - [`48a4543`](https://github.com/morluto/rea/commit/48a4543b39ce7f8531653046c69eb6badc3fe39c) docs(spec): record saved values in browser scenarios - [`31d6e1d`](https://github.com/morluto/rea/commit/31d6e1d97844d7e84d56ab3220a312af21cdcd3a) fix(environment): learn addresses a UI-made create spells, and check templates on later creates - [`cf40db8`](https://github.com/morluto/rea/commit/cf40db8c5b16790a2f679990f5353b714e1c4644) docs: record the write-model fixes and fresh recordings; ask agents to clear earlier attempts - [`8218b0b`](https://github.com/morluto/rea/commit/8218b0b246dd987150af2a4a23955532e01ea19e) Merge remote-tracking branch 'origin/main' into feat/recorder-saved-values - [`812f9ff`](https://github.com/morluto/rea/commit/812f9ff2fa9dab8551172afc43660f9da07aa92d) docs(spec): record the gates after merging main ### 📊 Changes **37 files changed** (+3941 additions, -215 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+1 -1) 📝 `.agents/skills/reverse-engineer-anything/references/environments.md` (+68 -26) 📝 `docs/agent-environments.md` (+4 -1) 📝 `docs/browser-scenario-contract.md` (+118 -1) ➕ `docs/specs/agent-environments/scenario-saved-values/plan.md` (+22 -0) ➕ `docs/specs/agent-environments/scenario-saved-values/spec.md` (+81 -0) ➕ `docs/specs/agent-environments/scenario-saved-values/tasks.md` (+15 -0) ➕ `docs/specs/agent-environments/scenario-saved-values/verification.md` (+132 -0) 📝 `docs/web-network-captures.md` (+28 -5) ➕ `src/application/BrowserScenarioEvidence.test.ts` (+63 -0) 📝 `src/application/BrowserScenarioEvidence.ts` (+19 -4) 📝 `src/application/environment/environmentBuild.ts` (+5 -1) 📝 `src/application/environment/environmentBuildReport.test.ts` (+52 -0) 📝 `src/application/environment/environmentBuildReport.ts` (+52 -0) 📝 `src/application/environment/environmentStateAssignedCoverage.test.ts` (+37 -16) 📝 `src/browser/BrowserScenarioSecrets.ts` (+122 -6) 📝 `src/browser/PlaywrightScenarioActions.ts` (+62 -16) 📝 `src/browser/PlaywrightScenarioRequest.test.ts` (+1181 -2) 📝 `src/browser/PlaywrightScenarioRequest.ts` (+507 -23) 📝 `src/domain/analysisView/EnvironmentStateWriteSimulation.ts` (+1 -0) _...and 17 more files_ </details> ### 📄 Description ## Summary An agent that builds an environment from a prompt records the application with one browser scenario. Until now a scenario could not carry a value from one step to a later one: - **No ids from responses.** After creating an object, the agent could not open, read, edit or delete it by the id the application assigned. Agents chose their own ids where the API allows it, or read lists back instead. - **No derived auth headers.** An API that takes Basic auth, or a header a frontend builds from a login token, could not be sent. Miniflux's agent stopped and asked for an API key. This PR lets steps save values and later steps send them: 1. A `request` step saves `[{name, json_pointer, secret}]` from its JSON response. A `click` or `press` step saves from the response to the request it triggers, with `save_response: {method, path, save}`, so a create made through the UI yields its id. 2. `{saved:name}` sends a saved value in a request path (and its query), a goto URL, and a JSON body, where a string that is only the placeholder keeps the saved JSON type. Header values and Basic credential parts take `{"source": "saved", "name"}`. 3. A header can send Basic credentials, `{"source": "basic_credentials", "parts": [username, password]}`, each part a literal, declared secret, saved value, cookie or local storage entry. 4. The skill records writes the way the UI does, saving ids from the responses its clicks trigger. 5. The write model serves UI-made creates as faithfully as API-made ones. Recording through the UI let Grafana generate folder uids, which exposed three gaps; this PR closes them (below). **Results**, agents building seven applications from the same prompts, with main and with this branch: - Served writes go from 9 to 24, and write tasks from 4/20 to 8/20 and 9/20 with the two models. Miniflux goes from no environment to 3/3, and Vikunja from 0/5 to 3/5. - Rebuilding the branch's own recordings with the write-model fixes changes only Grafana's plans, whose tasks go from 0/2 to 2/2. - The fresh Grafana and EspoCRM recordings scored below main's, for reasons in what was recorded (below). ## Problem and expected behavior - A step should save a string or number from a JSON response, by name, and a later step should send it in a path, a URL, a JSON body or a header. - A UI step that submits a form should save from the response to the request it triggers. - A header should carry Basic credentials built from any value kind. - A scenario that uses none of this should record and hash as before, and earlier REA versions should read captures that use it. ## Change and scope - **Schema** (`browserScenarioValues.ts`, `browserScenario.ts`): - `save` on request steps, `save_response` on click and press, the `saved` value source in headers and credential parts, `basic_credentials` with two parts, and `{saved:name}` (a parsed URL's percent-encoded form included). - Validation refuses a saved value no earlier step saves. Other braces, such as `{id}`, are sent as written. - **Runtime** (`PlaywrightScenarioRequest.ts`, `PlaywrightScenarioActions.ts`, `BrowserScenarioSecrets.ts`): - A saving request step reads its response text, up to 4,000,000 characters, in the page. - A UI step's `save_response` starts waiting before the step acts and fails the step unless a matching 2xx JSON response arrives before its timeout. - Saved values keep their JSON type and fill placeholders percent-encoded in paths and URLs. - Basic credentials are base64-encoded and redacted, as are parts read from the page. A value saved with `secret: true` is redacted from later records. - **Evidence** (`BrowserScenarioEvidence.ts`): an empty `save` is left out of the recorded step, so parameters and `scenario_sha256` are unchanged for scenarios that save nothing. - **Environment build**: warns about saved values named like credentials that are not marked secret (`credential-named-saved-values`). - **Skill** (version 50): - Writes the way the UI does and saves ids instead of guessing them. - Falls back to request steps when the UI cannot reach an operation or coverage withholds the UI-shaped write. - Its recording example creates through the UI. - **Write model** (`src/domain/analysisView`): - A `composed` string that reads a field besides the generated value a `template` would rebuild it around wins over the template, so Grafana's `/dashboards/f/{uid}/{slug(title)}` is composed when the uid is generated. - Later creates check `template` fields against their answers, as they check `composed` ones, so coverage no longer serves a template a later create contradicts. - A slug learns how the application spells `&`: `ampersand: "and"` on a `slug` rule or composed part, learned from the call that shows it. Calls that slugged no `&` agree with either spelling. - Where one text could come from two sources (an id equal to the title's slug), the entities the type holds pick the one reading they share. When they cannot, a later create that tells the fields apart lends its composed rule to the first create, whose body contract is learned again with it. - **Skill:** also asks agents to delete what an earlier attempt created before recording again, since the environment holds a leftover as seeded data. - **Docs and spec:** the browser scenario contract's saved values section, the environment and web capture references, and `docs/specs/agent-environments/scenario-saved-values/`. **Not included:** - general value transforms or script steps; - saved values in `fill` and `select_option`; - multi-factor sign-in; - lists sorted by a field an update changes, which REA keeps in place (pre-existing; seen in Grafana's fresh run). ## Contract and boundary impact - **Semantic owner and earliest changed stage:** the browser scenario contract (`src/domain/browserScenarioValues.ts`) and its Playwright runtime. - **Contribution scope fit and linked maintainer decision:** within the scenario request steps of #6946. Part of #4933. - **CLI and MCP/tool-catalog contract:** - `capture_browser_scenario` accepts `save`, `save_response`, the `saved` source, `basic_credentials` and `{saved:name}`. - The compact catalog grows by 3,664 bytes, from 1,038,975 on main 2b84efe0 to 1,042,639, under the 1 MiB frame `outputSchemaProfile.test.ts` measures. A first design with `path_params` and `json_values` fields failed that test. - `save_response.path` uses the request path's portable pattern, now shared as `START_ORIGIN_PATH`. - **Provider, bridge, target-format, or platform compatibility:** unchanged. - **Evidence, artifact, provenance, or reconstruction contract:** - Scenarios that save nothing record the same parameters and `scenario_sha256`. - Captures that save values parse on earlier REA versions. - `environment-state` views may carry `ampersand` on slug rules and parts; earlier REA versions refuse views that do, as the schemas are strict. Earlier views parse unchanged. - **Target selection, authority, upstream behavior, and cleanup details:** - A UI step with `save_response` waits for a response on the start origin. - Request steps stay same-origin, as before. - **Generated metadata, package, or installation impact:** the skill version goes from 49 to 50. Expected effects: - [ ] Read existing files or captured data. - [x] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [x] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage **Agent-built environments.** Claude Code (`claude-sonnet-5-5`) built each environment from the same prompt with main ff6d96bc (skill 49) and with this branch, on freshly seeded instances. Each bundle's write tasks ran with `fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash`, each checking the result from the environment's state. **The branch's recordings, rebuilt before and after the write-model fixes** (gpt-oss, GLM). Only Grafana's plans change: | Application | Before | After | | ----------- | ------------- | ------------- | | Miniflux | 3/3 3/3 | 3/3 3/3 | | Paperless | 1/3 2/3 | 1/3 1/3 | | Vikunja | 1/3 1/3 + 0/2 | 1/3 1/3 + 0/2 | | Linkding | 0/2 0/2 | 0/2 0/2 | | Homebox | 0/2 0/2 | 0/2 0/2 | | Grafana | 0/2 0/2 | 2/2 2/2 | | EspoCRM | 1/3 1/3 | 1/3 1/3 | Paperless's plans are identical in both builds; its GLM difference is the eval model's. **Fresh recordings, main and the final branch:** | Application | Write tasks | Main: served, tasks | Branch: served, tasks | | ----------- | ----------- | ---------------------- | --------------------- | | Miniflux | 3 | no bundle | 4, 3/3 3/3 | | Paperless | 3 | 3, 1/3 1/3; 6, 2/3 2/3 | 6, 1/3 2/3 | | Vikunja | 3 + 2 | 1, 0/3 0/3 + 0/2 0/2 | 3, 1/3 1/3 + 2/2 2/2 | | Linkding | 2 | 0, 0/2 0/2 | 3, 0/2 0/2 | | Homebox | 2 | 1, 0/2 0/2 | 5, 0/2 0/2 | | Grafana | 2 | 2, 2/2 2/2; 3, 2/2 2/2 | 1, 1/2 1/2 | | EspoCRM | 3 | 2, 1/3 1/3 | 2, 0/3 0/3 | "Served" counts the write tools the final capture serves; tasks are gpt-oss then GLM. Paperless and Grafana were recorded twice on main. - **Miniflux:** main's agent stopped for an API key; with Basic credentials the branch serves category create, rename and delete, and every category task passed in all three branch recordings. - **Saved values in use:** every branch agent saved ids, with `save_response` on UI writes. - **How the branch got here.** The first branch recording showed that EspoCRM's agent copied the skill's request-step example and wrote through the API, which EspoCRM answers in a shape the write model withholds (0/3); the example now creates through the UI. The second showed Grafana's UI-made creates served with wrong addresses (0/2); the write-model fixes above close that. - **Grafana, fresh:** the create is served with the right address and passes. The rename is withheld: Grafana lists folders sorted by title, the recording renamed a folder to a title that sorts first, and REA keeps an updated entity where it was in a list. Main's REA reads this recording the same way, and would serve its create with the template that keeps the first folder's slug. Main's two recordings passed because their titles kept their order. - **EspoCRM, fresh:** the recording started with a contact an earlier attempt created and left, so the environment holds it as seeded data and the "nothing else changed" checks fail. The skill now asks agents to delete such objects before recording again; that was not measured. - **Paperless:** the two main runs differ as much as main and the branch do. - **Linkding and Homebox** serve more writes, but their create tasks still fail. Linkding's served create requires every member its recorded calls sent, so a task sending a URL and a title is refused. - **Census:** write plans for 109 captures (the 50 published captures, 29 earlier agent recordings and the 30 above), built by main and by this branch: 524 plans and 247 served on each. Three plans differ, all Grafana folder creates, whose `url` the branch composes from the uid and the title's slug. **Real Chrome**, a local application with a cookie-session UI and an API that takes only Basic auth: - An 11-step scenario saved an id from a Save click and opened `/items/{saved:item1}`. - It sent Basic credentials from local storage and a secret, and saved a secret token. - It created a child with `{"name": "Child of {saved:item1}", "ref": "{saved:item1}"}`, which was sent as `{"name":"Child of 101","ref":101}`. - The password appears nowhere in the capture, and the token header is recorded redacted. **Tests added:** - schema: - a saved value sent before it is saved, in a path, query, goto URL (percent-encoded too), JSON body, header or credential part; - literal braces sent as written; - credential parts from page values, an undeclared secret part, and a wrong part count; - `save_response` paths and an empty save; - runtime with a fake page: - ids sent in a path and a JSON body with their type, and a saved secret redacted; - a missing pointer and a non-JSON response; - Basic credentials from local storage and a cookie, encoded and redacted; - a goto filled percent-encoded; - `save_response` waiting before the click and matching method, origin and `{*}` segments, and failing without a matching or 2xx response; - Evidence: a scenario that saves nothing records and hashes as before; saved values and placeholders are recorded; - the `credential-named-saved-values` warning; - write model: - a generated uid and the title's slug compose a folder's address; - the `&` spelling is learned from a later create, a dropped `&` stays dropped and a create spelling it otherwise is withheld; - an id equal to the title's slug is told apart by a later create, or by held entities; - a template a later create answers otherwise is withheld; - a copied member a composition rebuilds is accepted with any value, and one a template would keep stays fixed. Each fails on main. - **Review round** (each fails on 812f9ff2 unless noted): - Basic credentials from one-letter secrets and a two-character secret save are redacted; - a literal Basic password is refused, an empty one accepted; - `save_response` ignores a request already running when the step began, matches `My List` against `My%20List`, refuses a response whose received size is over the cap without reading it, and stops listening when the step fails (passes on 812f9ff2, which never listened); - a saving request step stops reading after the chunk that passes the cap; - a goto keeps a structured query value that spells a placeholder; - a slug field keeps a dropped `&` an accepted create showed, takes `and` from held tags, and learns no slug rule they contradict; a composed address takes `and` from held folders; - many fields sharing one text leave an address unread (passes on 812f9ff2, which reached the same result with unbounded work). - **Second review round** (each fails on af3d5626, the merge of main before it): - a saved integer past 2^53 fails the step instead of saving a rounded id; - a JSON string spelling `%7Bsaved:item%7D` needs no save and is sent as written; - a held slug neither `&` spelling gives keeps a create that slugs `&` from learning a slug rule, and leaves a create without `&` its plain rule. For evidence-bearing changes: - [ ] Observed, derived, and inferred claims remain distinguishable. - [ ] Artifact identity, source provenance, and failed attempts remain preserved. - [ ] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Fifth review round at `9f5191ab0` (main `ca3fd7ba` merged), on macOS with Node v24.22.1: - `6f6c61613` fixes Greptile's three findings on `22a93fadc`: a saved secret is redacted as a URL's query spells it once parsed (`'` as `%27`); a step's saved-value fills are counted before each is built against a 4,000,000-character budget (percent-encoded in a path or `goto` URL), so repeated references fail the step before it sends; and a create whose slug the held `&` spellings refuse accepts the slugged member only as recorded. Its six tests fail on `22a93fadc`; `9f5191ab0` makes one of them check the plan's body is present first. `3de21ef5d` merges main `ca3fd7ba`. - `typecheck`, `check:fast`, `format:check`, `knip`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery` and `scan:todos`: passed. `domain` + `services` + `adapters`: 18,758 passed, with the same two failures as main. Browser boundary tests and `webArtifactReadOwnership`: 1,223 passed, with the two `sazWebCapture` failures main has. - Census on main `ca3fd7ba` and this head: 524 plans and 246 served on each (main's anchorless-clock change withholds one Vikunja label update on both), with the same three Grafana creates differing. Fourth review round at `22a93fadc` (main `dd7a2da5` merged), on macOS with Node v24.22.1: - The red CI on `81a275b13` came from main as merged for that run: `662ec2e45` left `scripts/verify-apk-signature.mjs` unformatted (main fixed it in `741f2de75`), and `webArtifactReadOwnership.test.ts`, which passes on this branch, failed only in that merge. `723fe02c8` merges main `dd7a2da5`. - `22a93fadc` decodes a saving response as fatal UTF-8 in both save paths, so a malformed byte fails the step instead of saving U+FFFD, and bounds a secret save's spellings from the value before building any (Codex follow-ups). Its four tests fail on `723fe02c8`. - `typecheck`, `check:fast`, `format:check`, `knip`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery` and `scan:todos`: passed. `domain` + `services` + `adapters`: 18,673 passed; the Inspector stat test and `SetupRegistrationPrewarm`'s detached-descendant test fail, as they do on main `dd7a2da5`. Browser boundary tests and `webArtifactReadOwnership`: 1,223 passed; two `sazWebCapture` tests fail, as they do on main. - Census on main `dd7a2da5` and this head: 524 plans and 247 served on each, with the same three Grafana creates differing. Third review round at `69f38a78b` (main `a30ffd27` merged), on macOS with Node v24.22.1: - `7b2e78077` then checks each filled path segment after substitution, so a saved `.` that the text beside it turns into a dot segment fails the step (Greptile's follow-up); its tests fail on `69f38a78b`, `check:fast` passes and `src/browser/` passes 1,043 tests. - `a712f52e2` saves no number that is not finite (`1e400`), and `d0d7c248b` lets a later update refine an ambiguous slug spelling as creates do (Codex follow-ups); their tests fail on `7b2e78077`. `check:fast`, `docs:check` and 2,852 browser, write-model and environment tests pass, and the 109-capture census stays at 524 plans and 247 served. - `22ac315c3` keeps an update's word spelling of `&` for a later name without one and withholds a later create that omits a composed or templated field; `d39f386d2` caps the saved values a scenario keeps at 4,000,000 characters. Their tests fail on `d0d7c248b`; `check:fast`, `docs:check` and 2,859 browser, write-model and environment tests pass, and the census is unchanged at 524 plans and 247 served. - `059322048` reads a saved response only once its decoded size is bounded (gzip/deflate by DEFLATE's maximum expansion; br, zstd or an unknown size fail the step) and counts only the spellings a secret save adds; `81a275b13` saves no number JSON parsing changes (`1e-400`, `1.0000000000000001`) and spells an update's `&` as held entities do. Their tests fail on the head each thread was reported on; `check:fast`, `docs:check` and 2,880 browser, write-model and environment tests pass, and the census is unchanged at 524 plans and 247 served. - Fixes the five Codex findings on `98f18cc5`: saved-response reads bounded by the step's deadline and abort signal; the 4,000,000-character cap kept, with a 12,000,000-byte bound before reading; Basic usernames with a colon refused; dot-only saved path segments refused; mixed held `&` spellings leave no slug rule. It also makes the saved-values example a whole scenario, as the product catalog test requires. Each new test fails on `98f18cc52`. - `tsc`, `check:fast`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`, `scan:todos` and `jscpd`: passed. `knip` reports only `isToolFamilyId`, as main does. `domain` + `services` + `adapters`: 18,201 passed, with the Inspector stat test failing as on main. Product catalog, output schema profile, MCP schema validation, tool schema validity, the environment build and serve CLI, served-writes MCP and analysis-view CLI boundaries: 8 files, 75 passed. - Census on main `a30ffd27` and this head: 524 plans and 247 served on each, with the same three Grafana creates differing; none of this round's fixes changes a plan. Earlier rounds: Run on macOS with Node v24.22.1 at e42362c9 (main 2b84efe0). Agent recordings ran on Ubuntu 24.04 VMs with snap Chromium. - `npm run typecheck`, `lint:built`, `format:check`, `metadata:check`, `verify:architecture-guards`, `knip:production`, `lint:typed`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`, `scan:todos` and `jscpd`: passed. `knip` reports one unused export, `isToolFamilyId` in `src/contracts/toolFamilies.ts`, which this branch does not touch and main 2b84efe0 reports too. - `npx vitest run --project domain --project services --project adapters`: 18,261 passed and 1 failed, "stops Inspector target authorization after the current stat settles", which fails the same way on main in this environment. - `npm run test:focused --` the browser scenario CLI and MCP boundaries, scenario input diagnostics, scenario cancellation and cleanup, the analysis-view CLI and MCP boundaries, the environment build, export and serve CLI boundaries, the environment serve, served-writes and bundle MCP boundaries, tool schema validity, MCP schema validation, and output schema constraints and profile: 17 files, 128 passed. - Census on main 2b84efe0 and this branch: 524 plans and 247 served on each, with the same three Grafana creates differing. - Scenarios that save nothing: identical parameters and `scenario_sha256` on main and on this branch. - Main reads the real-Chrome capture: `environment-build --dry-run` gives the same manifest, state, coverage and file plan on both. - After the first review round, the real-Chrome scenario, with its `save_response` path written as `/ui/{*}`, completed all 12 steps with no password or raw token recorded. ## Compatibility, safety, and release - **Breaking changes or migration steps:** none for callers. - **Real Hopper/Ghidra, browser, or OS coverage:** Google Chrome on macOS for the local check; snap Chromium on Ubuntu 24.04 for the agent recordings. - **Package or release metadata impact:** skill version 50. - **Security, privacy, process, or containment review:** - Saved values are kept in memory for the scenario only. - A value saved with `secret: true`, Basic credentials, and parts read from the page are redacted from later records. - The response a secret was saved from may keep it, since the recorder reads bodies on their own schedule; the contract says so. - The build warns about credential-named saved values not marked secret. ## Author readiness Human owner: @earakely-scale - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. Part of #4933. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:39:39 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7687

No description provided.

Rows
Columns