[PR #6385] [MERGED] feat(environment): give more write tools a state target #7326

Closed
opened 2026-11-01 21:47:05 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6385
Author: @earakely-scale
Created: 10/22/2026
Status: ✅ Merged
Merged: 10/24/2026
Merged by: @morluto

Base: main ← Head: feat/environment-state-targets


📝 Commits (10+)

  • b49177e fix(environment): keep read-only POSTs out of the write list
  • d5a1fa9 fix(environment): take an entity an unplanned write made when a read first shows it
  • 004b064 fix(environment): plan a create into the type its returned id is read under
  • d8f9c05 fix(environment): end a type's initial reads at the write its reads disagree across
  • 64e472a fix(environment): replay request lines that show entities in another form
  • 28c3ae9 docs(environment): specify and record the state-target changes
  • 48d7fae fix(environment): keep a read-only POST a write when its evidence is incomplete
  • 5d48df7 fix(environment): answer a field selection only when the state shows it exactly
  • ce2301d fix(environment): hold adopted entities and linked creates to what the recording shows
  • 48106e1 Merge remote-tracking branch 'origin/main' into feat/environment-state-targets

📊 Changes

41 files changed (+5404 additions, -236 deletions)

View changed files

📝 docs/agent-environments.md (+12 -9)
📝 docs/specs/agent-environments/list-queries/spec.md (+3 -1)
➕ docs/specs/agent-environments/state-targets/plan.md (+14 -0)
➕ docs/specs/agent-environments/state-targets/spec.md (+66 -0)
➕ docs/specs/agent-environments/state-targets/tasks.md (+10 -0)
➕ docs/specs/agent-environments/state-targets/verification.md (+98 -0)
📝 docs/specs/agent-environments/state-view/spec.md (+50 -8)
📝 docs/specs/agent-environments/state-writes/spec.md (+80 -1)
📝 docs/specs/agent-environments/write-coverage/spec.md (+7 -2)
📝 docs/web-network-captures.md (+17 -5)
📝 src/application/environment/environmentState.ts (+12 -6)
📝 src/application/environment/environmentStateAnswer.ts (+52 -20)
📝 src/application/environment/environmentStateStore.ts (+9 -0)
📝 src/application/environment/environmentStateWriteAnswer.test.ts (+372 -0)
📝 src/application/environment/environmentStateWriteAnswer.ts (+88 -8)
📝 src/domain/analysisView/EnvironmentStateAccumulator.ts (+1 -0)
📝 src/domain/analysisView/EnvironmentStateExtraction.ts (+225 -59)
📝 src/domain/analysisView/EnvironmentStateWriteSimulation.test.ts (+167 -5)
📝 src/domain/analysisView/EnvironmentStateWriteSimulation.ts (+59 -15)
📝 src/domain/analysisView/WebNetworkProjectionAccumulator.ts (+8 -0)

...and 21 more files

📄 Description

Summary

Across the 50 published capture-built environments, 91 of 195 write tools got no state target. The gate refused them before weighing any evidence, because their path named no state type, item or document. We traced each of the 91 to its cause in the recordings and the code. This PR fixes the five largest shared causes:

  1. Read-only POSTs (lookups, read-only RPC tunnels, session refreshes) were planned as writes.
  2. Entities created by a write the gate doesn't model failed the first read that showed them.
  3. Creates whose path is not the read path (POST /Playlists, read as /Items/{Id}) had no target, and their entity leaked into the initial state.
  4. A type whose reads disagree across an unrelated write stayed replayed.
  5. Variant request lines (?counts=true, ?expand=, ?fields=), which show the same entities in another form, failed the whole view.

Over the same captures, served write tools go from 46 to 61 and no-state-target from 91 to 61. No served write and no state-backed read view is lost, and every served write tool applies.

Refs #4933. One commit per cause; each commit's tests fail on the commit before it. The spec is docs/specs/agent-environments/state-targets/.

Problem and expected behavior

  • Read-only calls. A POST stops counting as a state write, and is replayed, when any of these holds:

    • its response repeats the read of its path that finished last before it began, with no write between;
    • it tunnels only read operations;
    • its path's last meaningful word names a lookup;
    • it renews a session.

    It stays a write when a related read line changes across one of its calls, or finds its entity gone after it; a read overlapping the call counts on neither side. For example, Grafana's query-history stays a write.

  • Adopted entities. A write that no built plan models, or one redirected or unanswered, can create an entity the gate's copy never held. When a successful item read later shows that entity, the gate puts it into the copy, and later writes to it are checked as usual. An entity a modelled write touched is never taken back.

  • Linked creates. A POST whose path names no state type becomes a create into the type whose item read first names an id its response returned, before any other tool's write, as long as the id is new. Its observation is a read of an id its response returned, in an envelope too. Serving creates in that read's collection, for the recorded path arguments in either spelling. If a type's reads only show such an id once the POST began, the POST ends that type's initial window; a list read shows its listed entities' own ids, not those they nest.

  • Rewrite cuts. When a type's reads disagree across a write under another root, its initial window ends at the earliest write within the change, and the state is extracted again. When both cuts apply, the earlier wins.

  • Variant lines. A read line that shows the same entities in another form is a variant.

    • It is either an extra query parameter that adds or drops members, or it conflicts with a base line on a shared entity.
    • It leaves the view's checks and the entity observations.
    • Item and list views name it with the query keys that pick it out (projected_query_keys).
    • A call matching those keys replays its recording, so it does not show served writes.
    • A 400, 404 or 410 on such a line is still evidence.

    A learned filter role whose name matches no field needs two agreeing values.

Change and scope

  • Self-contained:
    • new modules: environmentStateQueryCalls.ts, environmentStateWriteAdoption.ts, environmentStateWriteLinks.ts, environmentStateRewrites.ts and environmentStateVariants.ts;
    • small hooks in the extraction, the write simulation, the gate, list queries, the view schema and serving.
  • Not included:
    • item actions (/{type}/{id}/archive);
    • bulk writes at a collection path;
    • templating literal ids by value;
    • envelope masking;
    • making a value that differs between two request lines volatile. This contradicts the spec's distinct query strings, and it made 11 recorded reads answer otherwise than replay.
  • Scope: analysis of existing Evidence, and serving from it, inside the existing boundary. No target execution.

Contract and boundary impact

  • Semantic owner and earliest changed stage: the environment-state analysis view (src/domain/analysisView/), then serving.
  • Contribution scope fit and linked maintainer decision: extends the existing state and write-gate model; not applicable.
  • CLI and MCP/tool-catalog contract:
    • item and list views gain an optional projected_query_keys, and list views may now carry projected_record_ordinals;
    • a replayed answer can carry the provenance reason variant;
    • read-only POSTs leave the write plans and stay in the served catalog as reads;
    • the catalog check passes.
  • Provider, bridge, target-format, or platform compatibility: none.
  • Evidence, artifact, provenance, or reconstruction contract:
    • States whose views, plans or initial windows change get new Evidence ids.
    • Supplied states keep loading.
    • The Grafana and Mailpit initial states no longer hold the dashboard and message their captures created, so those environments should be rebuilt.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Measurements. All 50 published captures, one commit at a time on main.

  • The served smoke rebuilds each manifest with the tree under test and applies every served write tool.
  • The serving check answers every recorded read made before the capture's first write through state serving, and compares it with the recording.
  • The serving check reads no records from the five browser-scenario captures (Art Institute collection, LoC photos, Lychee, PhotoPrism, Pinry), so its read counts cover 45 apps; their 41 views count among the state-backed views.
Step Write tools Served No state target State-backed read views Recorded reads matching
main 195 46 91 1,098 1,833 of 1,886
read-only POSTs 189 46 84 1,104 1,911 of 1,964
adoption 189 47 84 1,104 1,911 of 1,964
linked creates 189 48 80 1,105 1,911 of 1,964
rewrite cuts 189 51 77 1,110 1,922 of 1,975
variant lines 189 61 61 1,150 2,068 of 2,126
review fixes 189 61 61 1,151 2,075 of 2,133
third round 189 61 62 1,151 2,075 of 2,133

Newly served write tools:

  • Grocy product delete;
  • Jellyfin item delete;
  • Tandoor recipe create, update and delete;
  • Baserow row create, update and delete;
  • Keycloak user delete;
  • Miniflux category create and update;
  • PocketBase create, update and delete;
  • Vikunja task delete.

Of the 40 views that variant lines make state-backed, 146 of 151 recorded reads made before the first write match. The other five are Sentry projects, which replay the first of several differing recordings, as before.

Review round. The 10 Greptile and Codex threads are fixed in three commits, each finding with a test that fails without it:

  • read-only POSTs: body-level batch wrappers, the last path segment's write word, redirected or unanswered calls, and a veto resolved over all candidates;
  • adoption and linked creates: adopted entities are charged to the budget, a linked create holds only for its exemplar's path arguments, and short identity-named ids link with a guard;
  • variant lines: a field selection replays unless the state shows it exactly, and a list's fields line is set aside before the identity proof.

No write verdict changes; one NocoDB records list becomes state-backed (7 of 7 recorded reads match).

A second round fixed 8 more threads in three commits, again each with a test that fails without it: the write gate now skips exactly the reads serving replays; an earlier list or collection read showing a returned id vetoes a linked create; an RPC batch reads only when every entry names a read operation; a planning pass a rewrite cut discards is released from the budget; and no unbounded array is spread into call arguments. No measurement changes.

A third round fixed 9 more Codex and Greptile threads in one commit, each with a test that fails without it:

  • read-only POSTs: a repeated read must have finished before the call began, the veto compares reads that finished before and began after the call and counts a recorded absence as a change, and findAndModify-style operations stay writes;
  • adoption and links: a redirected or unanswered write lets a later read adopt what it made, a read still running when a POST began counts as after it, a list read's ids are its listed entities' own, and a linked create's read must come before any other tool's write;
  • budget and selections: line shapes are charged as they are built, and a fields=* list line is whole.

Mailpit's send, a linked create that was never observed, loses its target, because other writes ran before the read naming its id. No served write, read view or matching read changes.

A fourth round fixed the 6 threads the bots raised on that push, in one commit, each with a test that fails without it:

  • a linked create is observed only in a read of an id its response returned, envelopes included, and serving takes its recorded path values in either spelling;
  • a read-only POST compares the read that finished last before it began;
  • a list read also shows a suffix-named identity (projectId) distinct across its entities;
  • an entity's first sighting comes from the lines left after variant lines are dropped, so a plain list in another order keeps whole membership;
  • the pairwise variant pass stops at a work limit, leaving every line checked.

No verdict, read view or recorded read changes. A purge of temporary files removed 32 of the local captures, so this round was measured on captures restored from the published datasets. Where a local copy survived they are byte-identical; Baserow's, Grocy's and Keycloak's published captures predate the ones in the table. On that set, the previous head and this one both give 187 write tools, 62 served (smoke 62 of 62), 1,151 views and 2,085 of 2,155 recorded reads matching.

A fifth round, after main 6a34be41 was merged, fixed Greptile's 3 threads in one commit, each with a test that fails without it:

  • a list read counts a suffix-named field such as projectId as its entities' ids only when no id-named field identifies them, so a foreign key beside id cuts no window;
  • a read-only POST is compared with the read that finished last among those no other write follows, so a consuming call stays a write;
  • a linked create's source path values compare as spelled on both sides, so no spelling is decoded twice.

Nothing changes on the restored captures.

A sixth round fixed 5 more Greptile and Codex threads in one commit, each with a test that fails without it:

  • a linked create's source values compare segment by segment, so a template naming two parameters alike refuses a changed first one;
  • serving fills a linked create's path fields with its arguments decoded once, as the gate learned them;
  • the read-only check reads each related line's records once per call;
  • member signatures are JSON arrays, so a member name holding the separator still conflicts, and the variant work limit counts line pairs as well as entities.

Nothing changes on the restored captures.

A seventh round, after main 96ea32b7 was merged, fixed 4 more Greptile and Codex threads in one commit, each with a test that fails without it:

  • an item line first recorded after a write that names a new shaping parameter and shows other members is a replayed variant, like an initial one;
  • an item only a replayed line showed, which the state never started with, answers with its own recording;
  • a linked create fills its path fields from its exemplar's path decoded once, whichever accepted spelling the call sends;
  • each membership change span is charged against the budget.

WordPress's post create moves from side-effect to unconfirmed, still unserved: its only later read asks for context=edit, which shows the edit representation, so it is now replayed rather than checked.

An eighth round, after main 3e7e102a was merged, fixed 6 more Greptile and Codex threads in two commits, each with a test that fails without it:

  • item lines first recorded after a write are decided once variants are dropped, against the lines left, so a later ?include=shelf beside a dropped ?expand=shelf replays, and a later ?fields=id,name that keeps the identity replays too;
  • a linked create's earlier-read checks count only reads finished before the call began, so a list still running when it began no longer refuses the link;
  • each replay key is charged against the budget, and a rewrite span's earliest write is found by bisection.

Nothing changes on the restored captures.

A ninth round, after main b7ebccf8 was merged, fixed 5 more Greptile and Codex threads in one commit, each with a test that fails without it:

  • an item only a replayed line showed replays only while no served create has added to its type, so a created and deleted item stays missing;
  • a linked create's earlier item-read veto checks only the candidate's type or collection, and an item read answered with JSON null shows no id;
  • late lines' member names are charged before they are kept, and the read-only check passes once over each write-free stretch.

Nothing changes on the restored captures, session replay included (2,764 of 3,022 session reads matching).

A tenth round, after main a2235b09 was merged, fixed 3 more Greptile threads in one commit, each with a test that fails without it: a linked create passes over an item read naming its id that shows no item; repeated reads for read-only POSTs are found in one sweep; and a creation cut keeps and charges only the ids some creation returned. Nothing changes on the restored captures.

Three serving cases an independent review flagged were fixed before this PR:

  • a paged Paperless list now answers all five documents;
  • a PocketBase item without expand now answers without it;
  • a NetBox filter learned from one line is not adopted.

End-to-end rebuilds. Five published environments last built on 10-18 were re-recorded on disposable Linux VMs and rebuilt with this branch.

  • Recording: each write tool got two scratch objects, and every write was read back at once by item and by whole list. Updates and deletes also ran on a seeded entity.
  • Results: each passed smoke, retrieval and write evals, and was published.
Environment Served write tools Retrieval (gpt-oss / GLM) Write tasks (gpt-oss / GLM)
Grocy product create, update, delete 12/12 / 12/12 3/3 / 3/3
Mattermost post create, update, delete 12/12 / 12/12 3/3 / 3/3
Baserow row update, delete 12/12 / 12/12 3/3 / 3/3
Keycloak user delete 12/12 / 12/12 1/1 / 1/1
Grafana folder delete 12/12 / 12/12 1/2 / 2/2
  • Grafana: its initial state holds no dashboard or folder the capture created. The gpt-oss write miss is a provider rejection of the model's tool call.
  • Mattermost: POST /channels/{id}/members/ids replays as a lookup, and its channel, members and pinned reads answer from state.
  • Gaps these rebuilds expose, still unserved:
    • Keycloak's user create: its list shows access as a summary while its item shows the full map, and a created entity carries only the item's values;
    • Baserow's row create: a position sequence written as a decimal string, 5.0000…;
    • Grafana's folder create and rename: server-generated ids, URLs and versions.

Tests, each failing on the commit before:

  • read-only rules and their guard;
  • adopting an entity a write with no plan made, and not one a modelled write touched;
  • linked creates, and the creation cut with its two guards;
  • rewrite cuts and their composition with the creation cut;
  • variant list and item lines, by added, dropped and conflicting members;
  • a variant call that replays while the plain call answers from state;
  • a 404 on a variant line still confirming a delete;
  • single-line filter roles.

Nine existing view tests that asserted a rewritten type stays fields-differ now assert the cut and re-plan.

Remaining proof gaps:

  • After a write that is not served, a state-backed read can disagree with a recording made after that write (for example, a 404 right after a replayed create). This follows the existing rule.
  • Variant-line calls replay, and do not show served writes.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.

Validation performed

Run on macOS with Node v22.23.1, on the branch with main a2235b09 merged.

  • npm run -s check:fast (no new oxlint warnings in the changed files), npm run -s format:check, npx knip, npm run -s scan:todos, npm run -s docs:check, npm run -s verify:test-discovery, npm run -s verify:module-boundaries, npm run -s mcp-catalog:check and npm run jscpd (3.59%): passed.
  • npx vitest run --project domain --project services: 9393 passed, 6 pre-existing skips, and 10 failures that main has on its own (javascriptStaticAnalysis.mapEndpoints, javascriptStaticAnalysisReturnedInvocation, FridaInstrumentationService).
  • npm run -s test:prepare, then the 8 focused boundary files (environment serve and export CLI, serve and served-writes MCP, analysis-view CLI and MCP, compact input schema, tool schema validity): 90 passed.
  • The census on the restored captures, with the serving check passing a repeated query value as an array and a full session replay: unchanged by this round.
  • CI tests that fail on main itself are listed in the earlier rounds above; the agentenv bundle job failed once on a Docker Hub 502 and passed on the next run.

After merging main 3d4f5565 and then 2968d1e9, both without conflicts, into c166a694:

  • The static gates above: passed; npm run jscpd: 3.52%, main's own level.
  • npx vitest run --project domain --project services at 3d4f5565: the same 148 failing tests as main 3d4f5565 on this machine (Node v22.23.1, below main's new ^22.23.4 floor), none new. The boundary failures (clientConfigurationEnvironment and one analysisViewCli inventory test) fail on main too.
  • At c166a694: typecheck and format passed, the environment tests passed (1013), and the 8 focused boundary files passed (91). The census, served smoke, initial serve check and session replay are unchanged: 187 write tools, 62 served (smoke 62 of 62), 59 without a state target, 1,151 views, 2,085 of 2,155 initial reads and 2,764 of 3,022 session reads matching.
  • CI on c166a694: every build and test job passed.

After the four performance review threads on c166a694 (9614465f, checks recorded in 53b2966c):

  • The variant pass indexes lines by the entities they show and compares only lines sharing one; read-only decisions keep each body's canonical digest, taken once inside its charged parse; the creation cut skips reads at or after the write that ends the initial reads; adoption looks up a target only after an unplanned write.
  • Each of the 7 new tests failed on c166a694 with the old accessors wired as production wired them (work counts, a budget charge, and body access outside a lease); the test I/O now hands bodies through withBody, so the files' other tests no longer run against the old interface.
  • The static gates above passed (npm run jscpd: 3.53%); npx vitest run --project domain --project services: 9814 passed, 6 skipped, none failed; the environment boundary projects: 100 passed; the 8 focused boundary files: 91 passed.
  • Every census, serving-check and session-replay row equals the previous head's (62 served, 2,085 of 2,155 initial reads, 2,764 of 3,022 session reads). On a synthetic list plus 1,000 items read plainly and with ?expand=, the variant pass falls from 401 ms to 2 ms and the state build from 659 ms to 216 ms.

After the reviewer's second performance suggestion (3dd85f21), main d87e73f2 (f55c7315, and the same merge pushed by the reviewer, f151988a) and the round's checks (dee4dd75), at 3608c5aa, on macOS with Node v24.22.1 and main's locked dependencies:

  • The write gate lists a type's collections only when the path's own collection holds no id, and the variant pass parses base lines once (and not at all when it finds no variant). Both new tests fail on 53b2966c; the other 180 tests in their files pass there.
  • The static gates above passed (npm run jscpd: 3.58%); npx vitest run --project domain --project services: 9852 passed, 6 skipped, none failed; the environment boundary projects: 101 passed; the 8 focused boundary files: 93 passed. Every census, serving-check and session-replay row equals the previous head's.
  • The reviewer's synthetic capture (a list, then 1,000 items read plainly and with ?expand=): 259–264 ms against main's 244–247 ms, from 378–397 ms (Node v22.23.1, medians of seven warm runs).
  • The 50 captures' environment-state and environment-manifest views in one process: 2.96 s wall, 3.82 s CPU, 540 MiB peak RSS, against main d87e73f2's 2.61 s, 3.41 s and 516 MiB (warm medians of three runs; cold within 2%). The environment-state view takes 2,093 ms against 1,791 ms and its results grow from 3.69 MB to 3.84 MB; agent tokens were not measured.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Views gain optional members, and existing states load unchanged.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new inputs; reads retained Evidence only.

Author readiness

Human owner: @earakely-scale

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6385 **Author:** [@earakely-scale](https://github.com/earakely-scale) **Created:** 10/22/2026 **Status:** ✅ Merged **Merged:** 10/24/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `feat/environment-state-targets` --- ### 📝 Commits (10+) - [`b49177e`](https://github.com/morluto/rea/commit/b49177e425f102570572a70f98880112c07c8c1a) fix(environment): keep read-only POSTs out of the write list - [`d5a1fa9`](https://github.com/morluto/rea/commit/d5a1fa98e5b8a7e49ab419d56bbdc82487772073) fix(environment): take an entity an unplanned write made when a read first shows it - [`004b064`](https://github.com/morluto/rea/commit/004b064b85465e40944df7ac0cf65a07f44dc6d1) fix(environment): plan a create into the type its returned id is read under - [`d8f9c05`](https://github.com/morluto/rea/commit/d8f9c05264dd8267d12a4462e3b632ed89fef5d2) fix(environment): end a type's initial reads at the write its reads disagree across - [`64e472a`](https://github.com/morluto/rea/commit/64e472a769b974501bbb8db19183d1ac6fc6978a) fix(environment): replay request lines that show entities in another form - [`28c3ae9`](https://github.com/morluto/rea/commit/28c3ae97a9c8b8a45771cc396d2d3516b26c1518) docs(environment): specify and record the state-target changes - [`48d7fae`](https://github.com/morluto/rea/commit/48d7faee9015b5985d0e0cd75fb952f6d7a3af05) fix(environment): keep a read-only POST a write when its evidence is incomplete - [`5d48df7`](https://github.com/morluto/rea/commit/5d48df7bfc2f7a69fdd843ffa333b25233b39e27) fix(environment): answer a field selection only when the state shows it exactly - [`ce2301d`](https://github.com/morluto/rea/commit/ce2301d263138de9871faed04b3ed174a3db5c19) fix(environment): hold adopted entities and linked creates to what the recording shows - [`48106e1`](https://github.com/morluto/rea/commit/48106e103effaa04525edea73af6583d388c4448) Merge remote-tracking branch 'origin/main' into feat/environment-state-targets ### 📊 Changes **41 files changed** (+5404 additions, -236 deletions) <details> <summary>View changed files</summary> 📝 `docs/agent-environments.md` (+12 -9) 📝 `docs/specs/agent-environments/list-queries/spec.md` (+3 -1) ➕ `docs/specs/agent-environments/state-targets/plan.md` (+14 -0) ➕ `docs/specs/agent-environments/state-targets/spec.md` (+66 -0) ➕ `docs/specs/agent-environments/state-targets/tasks.md` (+10 -0) ➕ `docs/specs/agent-environments/state-targets/verification.md` (+98 -0) 📝 `docs/specs/agent-environments/state-view/spec.md` (+50 -8) 📝 `docs/specs/agent-environments/state-writes/spec.md` (+80 -1) 📝 `docs/specs/agent-environments/write-coverage/spec.md` (+7 -2) 📝 `docs/web-network-captures.md` (+17 -5) 📝 `src/application/environment/environmentState.ts` (+12 -6) 📝 `src/application/environment/environmentStateAnswer.ts` (+52 -20) 📝 `src/application/environment/environmentStateStore.ts` (+9 -0) 📝 `src/application/environment/environmentStateWriteAnswer.test.ts` (+372 -0) 📝 `src/application/environment/environmentStateWriteAnswer.ts` (+88 -8) 📝 `src/domain/analysisView/EnvironmentStateAccumulator.ts` (+1 -0) 📝 `src/domain/analysisView/EnvironmentStateExtraction.ts` (+225 -59) 📝 `src/domain/analysisView/EnvironmentStateWriteSimulation.test.ts` (+167 -5) 📝 `src/domain/analysisView/EnvironmentStateWriteSimulation.ts` (+59 -15) 📝 `src/domain/analysisView/WebNetworkProjectionAccumulator.ts` (+8 -0) _...and 21 more files_ </details> ### 📄 Description ## Summary Across the 50 published capture-built environments, 91 of 195 write tools got no state target. The gate refused them before weighing any evidence, because their path named no state type, item or document. We traced each of the 91 to its cause in the recordings and the code. This PR fixes the five largest shared causes: 1. **Read-only POSTs** (lookups, read-only RPC tunnels, session refreshes) were planned as writes. 2. **Entities created by a write the gate doesn't model** failed the first read that showed them. 3. **Creates whose path is not the read path** (`POST /Playlists`, read as `/Items/{Id}`) had no target, and their entity leaked into the initial state. 4. **A type whose reads disagree across an unrelated write** stayed replayed. 5. **Variant request lines** (`?counts=true`, `?expand=`, `?fields=`), which show the same entities in another form, failed the whole view. Over the same captures, served write tools go from **46 to 61** and no-state-target from **91 to 61**. No served write and no state-backed read view is lost, and every served write tool applies. Refs #4933. One commit per cause; each commit's tests fail on the commit before it. The spec is `docs/specs/agent-environments/state-targets/`. ## Problem and expected behavior - **Read-only calls.** A POST stops counting as a state write, and is replayed, when any of these holds: - its response repeats the read of its path that finished last before it began, with no write between; - it tunnels only read operations; - its path's last meaningful word names a lookup; - it renews a session. It stays a write when a related read line changes across one of its calls, or finds its entity gone after it; a read overlapping the call counts on neither side. For example, Grafana's `query-history` stays a write. - **Adopted entities.** A write that no built plan models, or one redirected or unanswered, can create an entity the gate's copy never held. When a successful item read later shows that entity, the gate puts it into the copy, and later writes to it are checked as usual. An entity a modelled write touched is never taken back. - **Linked creates.** A POST whose path names no state type becomes a create into the type whose item read first names an id its response returned, before any other tool's write, as long as the id is new. Its observation is a read of an id its response returned, in an envelope too. Serving creates in that read's collection, for the recorded path arguments in either spelling. If a type's reads only show such an id once the POST began, the POST ends that type's initial window; a list read shows its listed entities' own ids, not those they nest. - **Rewrite cuts.** When a type's reads disagree across a write under another root, its initial window ends at the earliest write within the change, and the state is extracted again. When both cuts apply, the earlier wins. - **Variant lines.** A read line that shows the same entities in another form is a variant. - It is either an extra query parameter that adds or drops members, or it conflicts with a base line on a shared entity. - It leaves the view's checks and the entity observations. - Item and list views name it with the query keys that pick it out (`projected_query_keys`). - A call matching those keys replays its recording, so it does not show served writes. - A 400, 404 or 410 on such a line is still evidence. A learned filter role whose name matches no field needs two agreeing values. ## Change and scope - **Self-contained:** - new modules: `environmentStateQueryCalls.ts`, `environmentStateWriteAdoption.ts`, `environmentStateWriteLinks.ts`, `environmentStateRewrites.ts` and `environmentStateVariants.ts`; - small hooks in the extraction, the write simulation, the gate, list queries, the view schema and serving. - **Not included:** - item actions (`/{type}/{id}/archive`); - bulk writes at a collection path; - templating literal ids by value; - envelope masking; - making a value that differs between two request lines volatile. This contradicts the spec's distinct query strings, and it made 11 recorded reads answer otherwise than replay. - **Scope:** analysis of existing Evidence, and serving from it, inside the existing boundary. No target execution. ## Contract and boundary impact - Semantic owner and earliest changed stage: the environment-state analysis view (`src/domain/analysisView/`), then serving. - Contribution scope fit and linked maintainer decision: extends the existing state and write-gate model; not applicable. - CLI and MCP/tool-catalog contract: - item and list views gain an optional `projected_query_keys`, and list views may now carry `projected_record_ordinals`; - a replayed answer can carry the provenance reason `variant`; - read-only POSTs leave the write plans and stay in the served catalog as reads; - the catalog check passes. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: - States whose views, plans or initial windows change get new Evidence ids. - Supplied states keep loading. - The Grafana and Mailpit initial states no longer hold the dashboard and message their captures created, so those environments should be rebuilt. - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage **Measurements.** All 50 published captures, one commit at a time on main. - The served smoke rebuilds each manifest with the tree under test and applies every served write tool. - The serving check answers every recorded read made before the capture's first write through state serving, and compares it with the recording. - The serving check reads no records from the five browser-scenario captures (Art Institute collection, LoC photos, Lychee, PhotoPrism, Pinry), so its read counts cover 45 apps; their 41 views count among the state-backed views. | Step | Write tools | Served | No state target | State-backed read views | Recorded reads matching | | --- | --- | --- | --- | --- | --- | | main | 195 | 46 | 91 | 1,098 | 1,833 of 1,886 | | read-only POSTs | 189 | 46 | 84 | 1,104 | 1,911 of 1,964 | | adoption | 189 | 47 | 84 | 1,104 | 1,911 of 1,964 | | linked creates | 189 | 48 | 80 | 1,105 | 1,911 of 1,964 | | rewrite cuts | 189 | 51 | 77 | 1,110 | 1,922 of 1,975 | | variant lines | 189 | 61 | 61 | 1,150 | 2,068 of 2,126 | | review fixes | 189 | 61 | 61 | 1,151 | 2,075 of 2,133 | | third round | 189 | 61 | 62 | 1,151 | 2,075 of 2,133 | Newly served write tools: - Grocy product delete; - Jellyfin item delete; - Tandoor recipe create, update and delete; - Baserow row create, update and delete; - Keycloak user delete; - Miniflux category create and update; - PocketBase create, update and delete; - Vikunja task delete. Of the 40 views that variant lines make state-backed, 146 of 151 recorded reads made before the first write match. The other five are Sentry projects, which replay the first of several differing recordings, as before. **Review round.** The 10 Greptile and Codex threads are fixed in three commits, each finding with a test that fails without it: - read-only POSTs: body-level batch wrappers, the last path segment's write word, redirected or unanswered calls, and a veto resolved over all candidates; - adoption and linked creates: adopted entities are charged to the budget, a linked create holds only for its exemplar's path arguments, and short identity-named ids link with a guard; - variant lines: a field selection replays unless the state shows it exactly, and a list's `fields` line is set aside before the identity proof. No write verdict changes; one NocoDB records list becomes state-backed (7 of 7 recorded reads match). A second round fixed 8 more threads in three commits, again each with a test that fails without it: the write gate now skips exactly the reads serving replays; an earlier list or collection read showing a returned id vetoes a linked create; an RPC batch reads only when every entry names a read operation; a planning pass a rewrite cut discards is released from the budget; and no unbounded array is spread into call arguments. No measurement changes. A third round fixed 9 more Codex and Greptile threads in one commit, each with a test that fails without it: - read-only POSTs: a repeated read must have finished before the call began, the veto compares reads that finished before and began after the call and counts a recorded absence as a change, and `findAndModify`-style operations stay writes; - adoption and links: a redirected or unanswered write lets a later read adopt what it made, a read still running when a POST began counts as after it, a list read's ids are its listed entities' own, and a linked create's read must come before any other tool's write; - budget and selections: line shapes are charged as they are built, and a `fields=*` list line is whole. Mailpit's send, a linked create that was never observed, loses its target, because other writes ran before the read naming its id. No served write, read view or matching read changes. A fourth round fixed the 6 threads the bots raised on that push, in one commit, each with a test that fails without it: - a linked create is observed only in a read of an id its response returned, envelopes included, and serving takes its recorded path values in either spelling; - a read-only POST compares the read that finished last before it began; - a list read also shows a suffix-named identity (`projectId`) distinct across its entities; - an entity's first sighting comes from the lines left after variant lines are dropped, so a plain list in another order keeps whole membership; - the pairwise variant pass stops at a work limit, leaving every line checked. No verdict, read view or recorded read changes. A purge of temporary files removed 32 of the local captures, so this round was measured on captures restored from the published datasets. Where a local copy survived they are byte-identical; Baserow's, Grocy's and Keycloak's published captures predate the ones in the table. On that set, the previous head and this one both give 187 write tools, 62 served (smoke 62 of 62), 1,151 views and 2,085 of 2,155 recorded reads matching. A fifth round, after main `6a34be41` was merged, fixed Greptile's 3 threads in one commit, each with a test that fails without it: - a list read counts a suffix-named field such as `projectId` as its entities' ids only when no id-named field identifies them, so a foreign key beside `id` cuts no window; - a read-only POST is compared with the read that finished last among those no other write follows, so a consuming call stays a write; - a linked create's source path values compare as spelled on both sides, so no spelling is decoded twice. Nothing changes on the restored captures. A sixth round fixed 5 more Greptile and Codex threads in one commit, each with a test that fails without it: - a linked create's source values compare segment by segment, so a template naming two parameters alike refuses a changed first one; - serving fills a linked create's path fields with its arguments decoded once, as the gate learned them; - the read-only check reads each related line's records once per call; - member signatures are JSON arrays, so a member name holding the separator still conflicts, and the variant work limit counts line pairs as well as entities. Nothing changes on the restored captures. A seventh round, after main `96ea32b7` was merged, fixed 4 more Greptile and Codex threads in one commit, each with a test that fails without it: - an item line first recorded after a write that names a new shaping parameter and shows other members is a replayed variant, like an initial one; - an item only a replayed line showed, which the state never started with, answers with its own recording; - a linked create fills its path fields from its exemplar's path decoded once, whichever accepted spelling the call sends; - each membership change span is charged against the budget. WordPress's post create moves from side-effect to unconfirmed, still unserved: its only later read asks for `context=edit`, which shows the edit representation, so it is now replayed rather than checked. An eighth round, after main `3e7e102a` was merged, fixed 6 more Greptile and Codex threads in two commits, each with a test that fails without it: - item lines first recorded after a write are decided once variants are dropped, against the lines left, so a later `?include=shelf` beside a dropped `?expand=shelf` replays, and a later `?fields=id,name` that keeps the identity replays too; - a linked create's earlier-read checks count only reads finished before the call began, so a list still running when it began no longer refuses the link; - each replay key is charged against the budget, and a rewrite span's earliest write is found by bisection. Nothing changes on the restored captures. A ninth round, after main `b7ebccf8` was merged, fixed 5 more Greptile and Codex threads in one commit, each with a test that fails without it: - an item only a replayed line showed replays only while no served create has added to its type, so a created and deleted item stays missing; - a linked create's earlier item-read veto checks only the candidate's type or collection, and an item read answered with JSON `null` shows no id; - late lines' member names are charged before they are kept, and the read-only check passes once over each write-free stretch. Nothing changes on the restored captures, session replay included (2,764 of 3,022 session reads matching). A tenth round, after main `a2235b09` was merged, fixed 3 more Greptile threads in one commit, each with a test that fails without it: a linked create passes over an item read naming its id that shows no item; repeated reads for read-only POSTs are found in one sweep; and a creation cut keeps and charges only the ids some creation returned. Nothing changes on the restored captures. Three serving cases an independent review flagged were fixed before this PR: - a paged Paperless list now answers all five documents; - a PocketBase item without `expand` now answers without it; - a NetBox filter learned from one line is not adopted. **End-to-end rebuilds.** Five published environments last built on 10-18 were re-recorded on disposable Linux VMs and rebuilt with this branch. - **Recording:** each write tool got two scratch objects, and every write was read back at once by item and by whole list. Updates and deletes also ran on a seeded entity. - **Results:** each passed smoke, retrieval and write evals, and was published. | Environment | Served write tools | Retrieval (gpt-oss / GLM) | Write tasks (gpt-oss / GLM) | | --- | --- | --- | --- | | Grocy | product create, update, delete | 12/12 / 12/12 | 3/3 / 3/3 | | Mattermost | post create, update, delete | 12/12 / 12/12 | 3/3 / 3/3 | | Baserow | row update, delete | 12/12 / 12/12 | 3/3 / 3/3 | | Keycloak | user delete | 12/12 / 12/12 | 1/1 / 1/1 | | Grafana | folder delete | 12/12 / 12/12 | 1/2 / 2/2 | - **Grafana:** its initial state holds no dashboard or folder the capture created. The gpt-oss write miss is a provider rejection of the model's tool call. - **Mattermost:** `POST /channels/{id}/members/ids` replays as a lookup, and its channel, members and pinned reads answer from state. - **Gaps these rebuilds expose,** still unserved: - Keycloak's user create: its list shows `access` as a summary while its item shows the full map, and a created entity carries only the item's values; - Baserow's row create: a position sequence written as a decimal string, `5.0000…`; - Grafana's folder create and rename: server-generated ids, URLs and versions. **Tests**, each failing on the commit before: - read-only rules and their guard; - adopting an entity a write with no plan made, and not one a modelled write touched; - linked creates, and the creation cut with its two guards; - rewrite cuts and their composition with the creation cut; - variant list and item lines, by added, dropped and conflicting members; - a variant call that replays while the plain call answers from state; - a 404 on a variant line still confirming a delete; - single-line filter roles. Nine existing view tests that asserted a rewritten type stays `fields-differ` now assert the cut and re-plan. **Remaining proof gaps:** - After a write that is not served, a state-backed read can disagree with a recording made after that write (for example, a 404 right after a replayed create). This follows the existing rule. - Variant-line calls replay, and do not show served writes. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. ## Validation performed Run on macOS with Node v22.23.1, on the branch with main `a2235b09` merged. - `npm run -s check:fast` (no new oxlint warnings in the changed files), `npm run -s format:check`, `npx knip`, `npm run -s scan:todos`, `npm run -s docs:check`, `npm run -s verify:test-discovery`, `npm run -s verify:module-boundaries`, `npm run -s mcp-catalog:check` and `npm run jscpd` (3.59%): passed. - `npx vitest run --project domain --project services`: 9393 passed, 6 pre-existing skips, and 10 failures that main has on its own (`javascriptStaticAnalysis.mapEndpoints`, `javascriptStaticAnalysisReturnedInvocation`, `FridaInstrumentationService`). - `npm run -s test:prepare`, then the 8 focused boundary files (environment serve and export CLI, serve and served-writes MCP, analysis-view CLI and MCP, compact input schema, tool schema validity): 90 passed. - The census on the restored captures, with the serving check passing a repeated query value as an array and a full session replay: unchanged by this round. - CI tests that fail on main itself are listed in the earlier rounds above; the agentenv bundle job failed once on a Docker Hub 502 and passed on the next run. After merging main `3d4f5565` and then `2968d1e9`, both without conflicts, into `c166a694`: - The static gates above: passed; `npm run jscpd`: 3.52%, main's own level. - `npx vitest run --project domain --project services` at `3d4f5565`: the same 148 failing tests as main `3d4f5565` on this machine (Node v22.23.1, below main's new `^22.23.4` floor), none new. The boundary failures (`clientConfigurationEnvironment` and one `analysisViewCli` inventory test) fail on main too. - At `c166a694`: typecheck and format passed, the environment tests passed (1013), and the 8 focused boundary files passed (91). The census, served smoke, initial serve check and session replay are unchanged: 187 write tools, 62 served (smoke 62 of 62), 59 without a state target, 1,151 views, 2,085 of 2,155 initial reads and 2,764 of 3,022 session reads matching. - CI on `c166a694`: every build and test job passed. After the four performance review threads on `c166a694` (`9614465f`, checks recorded in `53b2966c`): - The variant pass indexes lines by the entities they show and compares only lines sharing one; read-only decisions keep each body's canonical digest, taken once inside its charged parse; the creation cut skips reads at or after the write that ends the initial reads; adoption looks up a target only after an unplanned write. - Each of the 7 new tests failed on `c166a694` with the old accessors wired as production wired them (work counts, a budget charge, and body access outside a lease); the test I/O now hands bodies through `withBody`, so the files' other tests no longer run against the old interface. - The static gates above passed (`npm run jscpd`: 3.53%); `npx vitest run --project domain --project services`: 9814 passed, 6 skipped, none failed; the environment boundary projects: 100 passed; the 8 focused boundary files: 91 passed. - Every census, serving-check and session-replay row equals the previous head's (62 served, 2,085 of 2,155 initial reads, 2,764 of 3,022 session reads). On a synthetic list plus 1,000 items read plainly and with `?expand=`, the variant pass falls from 401 ms to 2 ms and the state build from 659 ms to 216 ms. After the reviewer's second performance suggestion (`3dd85f21`), main `d87e73f2` (`f55c7315`, and the same merge pushed by the reviewer, `f151988a`) and the round's checks (`dee4dd75`), at `3608c5aa`, on macOS with Node v24.22.1 and main's locked dependencies: - The write gate lists a type's collections only when the path's own collection holds no id, and the variant pass parses base lines once (and not at all when it finds no variant). Both new tests fail on `53b2966c`; the other 180 tests in their files pass there. - The static gates above passed (`npm run jscpd`: 3.58%); `npx vitest run --project domain --project services`: 9852 passed, 6 skipped, none failed; the environment boundary projects: 101 passed; the 8 focused boundary files: 93 passed. Every census, serving-check and session-replay row equals the previous head's. - The reviewer's synthetic capture (a list, then 1,000 items read plainly and with `?expand=`): 259–264 ms against main's 244–247 ms, from 378–397 ms (Node v22.23.1, medians of seven warm runs). - The 50 captures' environment-state and environment-manifest views in one process: 2.96 s wall, 3.82 s CPU, 540 MiB peak RSS, against main `d87e73f2`'s 2.61 s, 3.41 s and 516 MiB (warm medians of three runs; cold within 2%). The environment-state view takes 2,093 ms against 1,791 ms and its results grow from 3.69 MB to 3.84 MB; agent tokens were not measured. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Views gain optional members, and existing states load unchanged. - Real Hopper/Ghidra, browser, or OS coverage: not applicable. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new inputs; reads retained Evidence only. ## Author readiness Human owner: @earakely-scale - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-01 21:47:05 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7326
No description provided.