morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #7025] [MERGED] feat(environment): explain values an application derives from other fields #7584

New issue

Closed

opened 2026-11-03 13:38:53 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:38:53 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7025
Author: @earakely-scale
Created: 10/24/2026
Status: ✅ Merged
Merged: 10/24/2026
Merged by: @morluto

Base: main ← Head: feat/environment-derived-values

📝 Commits (10+)

a1edeaf feat(environment): explain values an application derives from other fields
8ce1127 docs(environment): record the derived-values verification
1ec053e chore: merge main into derived values
a80a96f fix(environment): keep clock spellings, linked ids and word boundaries honest
1e6c02c test(environment): keep a same-width fixed clock out of trimming
8e23f18 fix(environment): check composed parts and clock fits before they shape a plan
5a976d7 chore: merge main into derived values
09c48c5 test(environment): split the derived-values view tests by rule
2e5f56d fix(environment): keep fixed-width, zone and generated-integer evidence through a plan
96783c7 chore: merge main into derived values

📊 Changes

31 files changed (+3467 additions, -85 deletions)

View changed files

📝 docs/agent-environments.md (+15 -4)
➕ docs/specs/agent-environments/write-derived-values/plan.md (+12 -0)
➕ docs/specs/agent-environments/write-derived-values/spec.md (+59 -0)
➕ docs/specs/agent-environments/write-derived-values/tasks.md (+10 -0)
➕ docs/specs/agent-environments/write-derived-values/verification.md (+91 -0)
📝 docs/web-network-captures.md (+20 -5)
📝 src/application/environment/environmentCompareDiff.test.ts (+352 -0)
📝 src/application/environment/environmentCompareDiff.ts (+89 -15)
➕ src/application/environment/environmentStateDerivedAnswer.test.ts (+84 -0)
📝 src/application/environment/environmentStateWriteAnswer.ts (+8 -4)
📝 src/domain/analysisView/EnvironmentStateWriteSimulation.ts (+1 -0)
➕ src/domain/analysisView/environmentStateDerivedWrites.test.ts (+517 -0)
📝 src/domain/analysisView/environmentStateView.ts (+22 -0)
📝 src/domain/analysisView/environmentStateWriteApply.test.ts (+78 -0)
📝 src/domain/analysisView/environmentStateWriteApply.ts (+20 -6)
📝 src/domain/analysisView/environmentStateWriteCalls.ts (+17 -0)
📝 src/domain/analysisView/environmentStateWriteChanges.ts (+5 -0)
➕ src/domain/analysisView/environmentStateWriteClockFormats.test.ts (+111 -0)
➕ src/domain/analysisView/environmentStateWriteClockFormats.ts (+164 -0)
➕ src/domain/analysisView/environmentStateWriteComposed.test.ts (+302 -0)

...and 11 more files

📄 Description

Summary

Environments built from a prompt withheld creates in three of five applications. In each case the response carried a
value the application derives deterministically from other fields, which the write model either kept as a constant or
could not match across calls:

Linkding: web_archive_snapshot_url is https://web.archive.org/web/<creation time as YYYYMMDDhhmmss>/<sent url>.
The plan kept it as a constant, which pinned url to its recorded value, so a second create was refused.
Grafana: a folder's url is /dashboards/f/<uid>/<slug of the title>, which pinned uid the same way. Its
numeric id is a large time-based integer, which the plan copied from the first create, so the second create's
read-back contradicted it.
Homebox: being a Go application, it spells nanosecond clocks with trailing fraction zeros dropped
(…34.38812218Z), so a later create's clock did not follow the format learned from the first.

This PR adds three rules to the write model:

Composed strings. Literal text plus parts read from fields other rules write: a field's value, its slug, or a
clock's calendar digits.
Trimmed clock formats. Trailing fraction zeros are dropped, as Go does.
Generated integer ids. A large integer is learned as unique and served as a fresh seeded integer.

On the same timed captures, the baseline's 0/2 on the new create tasks becomes 2/2 with both models for Linkding's
bookmarks and Homebox's tags. A new timed Grafana recording serves its folder create, rename and delete, where the
baseline serves only the delete. Over 69 recorded captures no write goes from served to withheld, and the 50 published
environments serve the same writes as before.

Problem and expected behavior

A value the application builds from the call and its clock should be rebuilt for each served call, not replayed from
the recording, and should not stop the call from sending other values of the fields it reads.
A clock spelled with a varying number of fraction digits should be learned when the variation is trailing-zero
trimming, and served in that spelling.
A generated integer id should be generated for each served create, not copied from the first.
Plans that need none of this must not change.

Change and scope

Composed strings (environmentStateWriteComposed.ts):
- A create's unexplained string field is segmented left to right, longest delimited match first, against the texts
  other ruled fields offer. Generated values may be short; others need eight characters, as a copy does.
- A create composes a string only when at least one other entity of the type spells it with the same parts and none
  spells it otherwise. One call alone cannot show which text is literal.
- It refuses when one text could come from two sources that serving fills differently. Two clocks the write stamps
  count as one source.
- An update composes an unsent change from members stored as sent and fields it left unchanged. It is tried after
  the existing rules, and only when the same parts spell the value before the call and no other entity spells it
  otherwise.
- Serving rebuilds composed fields after every other rule. A composed field no longer pins the members it reads.
- A call whose composed part has nothing to spell is refused, and a composition learned on a later call is checked
  against every earlier call.
- Two stamped clocks are one source only when they share a zone offset, and a held entity that holds the field as
  something the parts do not spell, a non-string included, vetoes the composition.
Trimmed clocks (environmentStateWriteClockFormats.ts):
- ClockFormat gains trimmed, and a clock reading gains fixedWidth (its fraction ends in a zero).
- A plan's clock widens to a trimmed format when a field's values differ only in how many fraction digits they show
  and none ends in a zero. This is learned from the type's held values, a later create or a later update.
- Once a held entity, the exemplar or any accepted call shows a field at a fixed width (its fraction ends in a zero),
  no fit or sibling trim makes that clock trimmed, and a later call that shows it shorter is withheld. An update
  adopts a fitted spelling only after its checks pass.
Generated integers:
- A created integer of 10 to 20 digits, among held entities that each hold a distinct one of the same length and JSON
  kind (and none of them a clock), is unique.
- Serving seeds an integer of that length and kind that no held entity has. A number stays a safe integer.
- An integer another field of the created entity also holds, at any depth, is linked, as a repeated token is, and the
  create is withheld.
- Compare treats two integers of one JSON kind and length under a unique rule as generated values.
Docs and spec: the write-plan rules in docs/web-network-captures.md, the update row and a served-write limit in
docs/agent-environments.md, and docs/specs/agent-environments/write-derived-values/.

Not included:

learning a create plan from every call rather than the first (a later Linkding create that sends shared, Homebox
item creates after a location create), which is the next PR;
objects that embed a referenced entity (Homebox's parent);
list orders that change when an update changes a sort key.

Contract and boundary impact

Semantic owner and earliest changed stage: the environment-state view's write plans (src/domain/analysisView)
and served write answers (src/application/environment).
Contribution scope fit and linked maintainer decision: within the existing environment-state write model. It
reads recorded captures only and adds no new target interaction.
CLI and MCP/tool-catalog contract:
- No tool or argument changes.
- The environment-state view's write plans may carry a composed rule and a clock format with trimmed, and a
  unique rule may now name an integer field.
Provider, bridge, target-format, or platform compatibility: unchanged.
Evidence, artifact, provenance, or reconstruction contract:
- State views and bundles built by this version may hold the new rule and format. REA versions before it refuse them,
  because the plan schemas are strict.
- Earlier views parse unchanged.
Target selection, authority, upstream behavior, and cleanup details: unchanged.
Generated metadata, package, or installation impact: none.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Write plans over 69 recorded captures, built by the baseline and by this branch. The captures are the 50 published
ones, 18 Claude Code recorded from a prompt for #6946's checks, and one new timed Grafana recording:

Captures	Write plans	Served on the baseline	Served on this branch
50 published	187	68	68
19 recorded from a prompt	120	56	61

The five gains: Linkding's bookmark create, Homebox's tag create, and Grafana's folder create (two recordings) and
rename (the timed one).
No regressions: no write goes from served to withheld.
Published plans that change: Linkding's bookmark create and BookStack's book create and update. Each composes a
value and now accepts the member it reads, with the same served verdicts.
Left as before on purpose:
- Grafana's published folder: its uid and title slug are the same text, so the address names two sources.
- Ghost's lexical: keeps its derived rule.
- BookStack's page HTML and Jellyfin's playlist path: other entities of the type spell them otherwise.

Served write tasks on the same timed captures, rebuilt by the baseline and by this branch, with
fireworks_ai/gpt-oss-120b and fireworks_ai/glm-5p3-flash. The create tasks are new. Each checks the created entity
from the environment's state, including the composed address and that nothing else changed:

Application	Tasks	Baseline (gpt-oss, GLM)	This branch (gpt-oss, GLM)
Linkding	2 bookmark creates	0/2, 0/2	2/2, 2/2
Linkding	3 recorded (update, notes, delete)	3/3, 3/3	3/3, 3/3
Homebox	2 tag creates	0/2, 0/2	2/2, 2/2
Homebox	2 recorded deletes	0/2, 0/2	0/2, 0/2
Grafana	folder create and rename	not served	1/2, 2/2
Grafana	2 recorded deletes		2/2, 2/2

Smoke and retrieval: match on both builds. Grafana's new recording passes smoke and 9/9 retrieval with both models.
gpt-oss on the Grafana rename: it sent the update without the title it was asked to change, ten times. The served
environment refused each call and named the members it accepts.
Homebox's recorded deletes: they fail on both builds. This recording's served entity list holds its locations
only, so the models never find the item.
How the captures were recorded and evaluated:
- Claude Code (claude-sonnet-5-5) recorded each app from a one-paragraph prompt on Ubuntu 24.04 VMs with snap
  Chromium 155.
- Both the recording and the evaluation used a local, unpushed build of this branch merged with #6967, because
  browser scenarios need its request times for clock rules. The baseline is the same merge without this branch.

Tests added:

View: Go-spelled tag creates and renames, composed bookmark snapshot addresses, and folder creates and renames
with composed addresses and generated integer ids are served.
Serving:
- a tag create's clocks drop trailing zeros;
- a bookmark's snapshot address follows the served clock and the sent URL;
- a folder gets a fresh sixteen-digit id and an address that follows its uid and title, and a rename moves it.
Unit:
- clock format fitting, calendar digits, seeded integers and generated integer detection;
- composition and each refusal (one call alone, a held entity spelling otherwise, two sources with one text, a member
  stored otherwise than sent, a before value the parts do not spell);
- a wide body within the existing time limit.

Remaining proof gaps:

A composed string copies sent text as sent. An application that escapes or renders it (HTML entities, Markdown)
differs for text that needs it, which is documented as a served-write limit.
Calendar digits cover four patterns only.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Current follow-up at 5fd87e9c6, based on the author's aeae47f7b (macOS, Node 24.22.1 / npm 11.16.0):

Composition comparison verifies each response's own spelling, then compares the rendered parts or reuses the source fields' clock/ID decisions. It preserves business-input differences, rejects identical stale URLs, handles destination-before-source ordering, and counts each source once. The core composition formatter and generated-integer matcher are shared rather than duplicated in compare.
Create validation checks generated shapes and captured composed fields in one place, preserves omitted response fields as unknown, and now also refuses UUID-to-opaque-token drift. The author's later-create contradiction and integer-family fixes are retained.
26 source files / 858 tests passed for environment write/compare behavior. 3 compiled CLI/MCP boundary files / 36 tests passed, including the new composed-bookmark environment-compare case and existing trimmed-clock case; input Evidence files are unchanged by those workflows.
check:fast (typecheck/lint), format:check, docs:check, and pre-push checks passed. Four added assertions fail on aeae47f7b and pass here: equal stale compositions, differing actual inputs, unproven generated-looking inputs, and UUID-to-token drift.
The author's 69-capture census and model evaluations below were not rerun after this follow-up. Fresh hosted CI and automated review are pending.

Completed environment-compare CLI tasks with the same tag/bookmark captures and one create probe per case, before aeae47f7b versus this change. Complete canonical reports are identical for these valid tasks.

Task / invocation	Elapsed before → after	CPU before → after	Sampled peak process-family RSS before → after
Tag create, first fresh process	0.604 → 0.564 s	0.786 → 0.730 s	365.4 → 387.4 MiB
Tag create, median of 3 repeat processes	0.493 → 0.473 s	0.666 → 0.660 s	375.0 → 377.2 MiB
Bookmark create, first fresh process	0.469 → 0.484 s	0.667 → 0.659 s	365.7 → 362.2 MiB
Bookmark create, median of 3 repeat processes	0.470 → 0.471 s	0.657 → 0.656 s	369.5 → 383.8 MiB

Each invocation starts a fresh CLI process; OS caches were not cleared. CPU is child-family resource usage; RSS is sampled at roughly 15 ms plus process-list overhead. No resource stop occurred. These small samples establish no speedup or memory improvement. Agent/model tokens and costs on larger responses were not measured; no token savings are claimed.

Earlier revision validation and application results

Follow-up at 94f17534f (2026-10-25), after merging main 2acdd5413, on macOS with Node 24.22.1 and npm 11.16.0:

Addressed all five open threads: missing composition parts on earlier updates; held fixed-width evidence for update plans (two reports); trimmed-clock comparison; and order-independent propagation of the widest trimmed peer per separator/zone group.
Shared the domain clock matcher between learning and comparison, preserving parsed readings rather than reparsing values or duplicating the format policy.
Environment write/compare source tests: 26 files, 839 passed at 5f8a954c6; the subsequent main merge changes no environment source.
Relevant CLI/MCP/process boundaries, including the new environment-compare regression, managed bundle read guards, and setup registration prewarm: 6 distinct files, 77 passed. Managed read guards, environment-compare CLI, and setup prewarm were rerun on 94f17534f: 3 files, 21 passed.
check:fast (typecheck/lint), format:check, docs:check, and the pre-push check: passed.
Main's #7036 fixes the managed Evidence low-headroom CI failure; the failing test now passes. Main independently fixed its setup test-length lint error in 2acdd5413; the merge adopts that file verbatim, retaining all four scenarios and removing the temporary branch-specific fixture cleanup. check:fast passes again after the merge.
Representative completed CLI task: the same tag capture and single Go-style create probe on fb0eece97 versus 5f8a954c6. First invocation: 0.552 → 0.529 s elapsed, 0.735 → 0.731 s CPU, 362.5 → 365.9 MiB sampled peak process-family RSS. Median of three repeat invocations: 0.519 → 0.500 s elapsed, 0.713 → 0.703 s CPU, 367.2 → 374.8 MiB RSS. Each uses a fresh CLI process; OS caches were not cleared. RSS is sampled at roughly 15 ms plus process-list overhead. These small samples establish no speedup claim. The original reports two false clock-format differences; this head reports agreement. Agent/model tokens and broader performance costs were not measured.
Original application recordings and model evaluations below were not rerun for this follow-up. Hosted CI subsequently passed on the earlier head.

Follow-up at 3705faeb3, on macOS with Node 24.22.1:

Ports the #7034 finding that a generated integer repeated in another string (an echoed serialText, or a token like
folder-<id>) is withheld unless a template or composed rule rebuilds that string from it; its tests fail on
94f17534f, and a templated copy is still served.
check:fast, format:check, docs:check, mcp-catalog:check, verify:test-discovery, scan:todos, jscpd and
npx knip: passed. domain + services + adapters: 17,323 passed, with the Inspector stat test failing as it does
on main in this environment. The 10 focused boundary files: 82 passed. Managed bundle read guards: 11 passed.
Write plans over the 69 captures on main 2acdd541 and on this head: 124 → 129 served, the same five gains, no
losses. The 50 published environments serve the same 68 writes with 68/68 smoke calls, and all 54 rebuilt bundles are
byte-identical to the three-step path's.

Follow-up at aeae47f7b (e54b75a17 plus main aced9fa9b), on macOS with Node 24.22.1:

Withholds a later create whose response spells a composed field otherwise, or answers a field learned from a generated
integer with anything but an integer of its length and kind; compare counts a composed field as equal when each side
spells it from its own fields. Each new test fails on 3705faeb3.
The same gates as above pass (domain + services + adapters: 17,358 passed, with the Inspector stat test failing
as on main); the 10 focused boundary files: 82 passed. The 69-capture census on main aced9fa9b is still 124 → 129
with the same five gains, the 50 published environments are unchanged, and all 54 bundles are byte-identical.

Earlier author validation:

Run on macOS with Node v24.22.1 at 96783c72 (main 42577617 plus this branch).

tsc, npm run -s check:fast, format:check, docs:check, mcp-catalog:check, verify:test-discovery,
scan:todos and jscpd (3.64%), and npx knip: passed.
npx vitest run --project domain --project services --project adapters: 17,274 passed and 1 failed ("stops Inspector
target authorization after the current stat settles"), which fails the same way on main in this environment.
npm run test:focused -- the environment build, export and serve CLI boundaries, the analysis-view CLI and MCP
boundaries, the environment serve, served-writes and bundle MCP boundaries, MCP schema validation and output schema
constraints: 10 files, 82 passed.
Write plans over the 69 captures, rebuilt on main 42577617 and on this head: 124 → 129 served, with the same five
gains and no write going from served to withheld. The review fixes change no plan in these captures, so the
served-write task results above, measured before the review fixes, still describe this head.
The 50 published environments serve the same 68 writes with 68/68 smoke calls and the same session and served reads,
and all 54 rebuilt bundles are byte-identical to the three-step path's.
Each review fix has a test that fails on the head it was reported on (8ce1127b or 09c48c59).
Earlier CI failure from main (now fixed by #7036, verified above):
tests/process-global/managedBundleReadGuards.test.ts ("projects a rejected managed parent by reference without
cloning under low headroom") failed from 9b3a6332 through the earlier main 42577617, with and without this branch.

Compatibility, safety, and release

Breaking changes or migration steps: none for callers. Bundles and state views built by this version need this
version or later to serve.
Real Hopper/Ghidra, browser, or OS coverage: real applications on Ubuntu 24.04 with snap Chromium 155 for the
recordings; macOS for gates.
Package or release metadata impact: none.
Security, privacy, process, or containment review: reads recorded captures only. A composed value repeats text the
call sent or the state already holds.

Author readiness

Human owner: @earakely-scale

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

Part of #4933.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7025 **Author:** [@earakely-scale](https://github.com/earakely-scale) **Created:** 10/24/2026 **Status:** ✅ Merged **Merged:** 10/24/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `feat/environment-derived-values` --- ### 📝 Commits (10+) - [`a1edeaf`](https://github.com/morluto/rea/commit/a1edeafdd58558e766cee3f8eda6d007c216d41c) feat(environment): explain values an application derives from other fields - [`8ce1127`](https://github.com/morluto/rea/commit/8ce1127b5d09c47b0139235f02db2512704c5b77) docs(environment): record the derived-values verification - [`1ec053e`](https://github.com/morluto/rea/commit/1ec053e35830f25be80fdca489884a639ae74348) chore: merge main into derived values - [`a80a96f`](https://github.com/morluto/rea/commit/a80a96fc017f63db9bcb5a60009e23ce9d0207da) fix(environment): keep clock spellings, linked ids and word boundaries honest - [`1e6c02c`](https://github.com/morluto/rea/commit/1e6c02c04e930642090d27f1d351adb4ad0876f4) test(environment): keep a same-width fixed clock out of trimming - [`8e23f18`](https://github.com/morluto/rea/commit/8e23f18eebddf3a67206b6dda891dc103d232cca) fix(environment): check composed parts and clock fits before they shape a plan - [`5a976d7`](https://github.com/morluto/rea/commit/5a976d7cfa08e572116f5eb439d826dc94cb8c2d) chore: merge main into derived values - [`09c48c5`](https://github.com/morluto/rea/commit/09c48c5928edb7b07ea69c59a612d84dcd721baa) test(environment): split the derived-values view tests by rule - [`2e5f56d`](https://github.com/morluto/rea/commit/2e5f56dc7617c48ed1fdf35cbc21453d4e9c50ab) fix(environment): keep fixed-width, zone and generated-integer evidence through a plan - [`96783c7`](https://github.com/morluto/rea/commit/96783c72045966a8ebcfc0b1fa5db1c72a09c877) chore: merge main into derived values ### 📊 Changes **31 files changed** (+3467 additions, -85 deletions) <details> <summary>View changed files</summary> 📝 `docs/agent-environments.md` (+15 -4) ➕ `docs/specs/agent-environments/write-derived-values/plan.md` (+12 -0) ➕ `docs/specs/agent-environments/write-derived-values/spec.md` (+59 -0) ➕ `docs/specs/agent-environments/write-derived-values/tasks.md` (+10 -0) ➕ `docs/specs/agent-environments/write-derived-values/verification.md` (+91 -0) 📝 `docs/web-network-captures.md` (+20 -5) 📝 `src/application/environment/environmentCompareDiff.test.ts` (+352 -0) 📝 `src/application/environment/environmentCompareDiff.ts` (+89 -15) ➕ `src/application/environment/environmentStateDerivedAnswer.test.ts` (+84 -0) 📝 `src/application/environment/environmentStateWriteAnswer.ts` (+8 -4) 📝 `src/domain/analysisView/EnvironmentStateWriteSimulation.ts` (+1 -0) ➕ `src/domain/analysisView/environmentStateDerivedWrites.test.ts` (+517 -0) 📝 `src/domain/analysisView/environmentStateView.ts` (+22 -0) 📝 `src/domain/analysisView/environmentStateWriteApply.test.ts` (+78 -0) 📝 `src/domain/analysisView/environmentStateWriteApply.ts` (+20 -6) 📝 `src/domain/analysisView/environmentStateWriteCalls.ts` (+17 -0) 📝 `src/domain/analysisView/environmentStateWriteChanges.ts` (+5 -0) ➕ `src/domain/analysisView/environmentStateWriteClockFormats.test.ts` (+111 -0) ➕ `src/domain/analysisView/environmentStateWriteClockFormats.ts` (+164 -0) ➕ `src/domain/analysisView/environmentStateWriteComposed.test.ts` (+302 -0) _...and 11 more files_ </details> ### 📄 Description ## Summary Environments built from a prompt withheld creates in three of five applications. In each case the response carried a value the application derives deterministically from other fields, which the write model either kept as a constant or could not match across calls: - **Linkding:** `web_archive_snapshot_url` is `https://web.archive.org/web/<creation time as YYYYMMDDhhmmss>/<sent url>`. The plan kept it as a constant, which pinned `url` to its recorded value, so a second create was refused. - **Grafana:** a folder's `url` is `/dashboards/f/<uid>/<slug of the title>`, which pinned `uid` the same way. Its numeric `id` is a large time-based integer, which the plan copied from the first create, so the second create's read-back contradicted it. - **Homebox:** being a Go application, it spells nanosecond clocks with trailing fraction zeros dropped (`…34.38812218Z`), so a later create's clock did not follow the format learned from the first. This PR adds three rules to the write model: 1. **Composed strings.** Literal text plus parts read from fields other rules write: a field's value, its slug, or a clock's calendar digits. 2. **Trimmed clock formats.** Trailing fraction zeros are dropped, as Go does. 3. **Generated integer ids.** A large integer is learned as `unique` and served as a fresh seeded integer. On the same timed captures, the baseline's 0/2 on the new create tasks becomes 2/2 with both models for Linkding's bookmarks and Homebox's tags. A new timed Grafana recording serves its folder create, rename and delete, where the baseline serves only the delete. Over 69 recorded captures no write goes from served to withheld, and the 50 published environments serve the same writes as before. ## Problem and expected behavior - A value the application builds from the call and its clock should be rebuilt for each served call, not replayed from the recording, and should not stop the call from sending other values of the fields it reads. - A clock spelled with a varying number of fraction digits should be learned when the variation is trailing-zero trimming, and served in that spelling. - A generated integer id should be generated for each served create, not copied from the first. - Plans that need none of this must not change. ## Change and scope - **Composed strings** (`environmentStateWriteComposed.ts`): - A create's unexplained string field is segmented left to right, longest delimited match first, against the texts other ruled fields offer. Generated values may be short; others need eight characters, as a copy does. - A create composes a string only when at least one other entity of the type spells it with the same parts and none spells it otherwise. One call alone cannot show which text is literal. - It refuses when one text could come from two sources that serving fills differently. Two clocks the write stamps count as one source. - An update composes an unsent change from members stored as sent and fields it left unchanged. It is tried after the existing rules, and only when the same parts spell the value before the call and no other entity spells it otherwise. - Serving rebuilds composed fields after every other rule. A composed field no longer pins the members it reads. - A call whose composed part has nothing to spell is refused, and a composition learned on a later call is checked against every earlier call. - Two stamped clocks are one source only when they share a zone offset, and a held entity that holds the field as something the parts do not spell, a non-string included, vetoes the composition. - **Trimmed clocks** (`environmentStateWriteClockFormats.ts`): - `ClockFormat` gains `trimmed`, and a clock reading gains `fixedWidth` (its fraction ends in a zero). - A plan's clock widens to a trimmed format when a field's values differ only in how many fraction digits they show and none ends in a zero. This is learned from the type's held values, a later create or a later update. - Once a held entity, the exemplar or any accepted call shows a field at a fixed width (its fraction ends in a zero), no fit or sibling trim makes that clock trimmed, and a later call that shows it shorter is withheld. An update adopts a fitted spelling only after its checks pass. - **Generated integers:** - A created integer of 10 to 20 digits, among held entities that each hold a distinct one of the same length and JSON kind (and none of them a clock), is `unique`. - Serving seeds an integer of that length and kind that no held entity has. A number stays a safe integer. - An integer another field of the created entity also holds, at any depth, is linked, as a repeated token is, and the create is withheld. - Compare treats two integers of one JSON kind and length under a `unique` rule as generated values. - **Docs and spec:** the write-plan rules in `docs/web-network-captures.md`, the update row and a served-write limit in `docs/agent-environments.md`, and `docs/specs/agent-environments/write-derived-values/`. **Not included:** - learning a create plan from every call rather than the first (a later Linkding create that sends `shared`, Homebox item creates after a location create), which is the next PR; - objects that embed a referenced entity (Homebox's `parent`); - list orders that change when an update changes a sort key. ## Contract and boundary impact - **Semantic owner and earliest changed stage:** the environment-state view's write plans (`src/domain/analysisView`) and served write answers (`src/application/environment`). - **Contribution scope fit and linked maintainer decision:** within the existing environment-state write model. It reads recorded captures only and adds no new target interaction. - **CLI and MCP/tool-catalog contract:** - No tool or argument changes. - The `environment-state` view's write plans may carry a `composed` rule and a clock format with `trimmed`, and a `unique` rule may now name an integer field. - **Provider, bridge, target-format, or platform compatibility:** unchanged. - **Evidence, artifact, provenance, or reconstruction contract:** - State views and bundles built by this version may hold the new rule and format. REA versions before it refuse them, because the plan schemas are strict. - Earlier views parse unchanged. - **Target selection, authority, upstream behavior, and cleanup details:** unchanged. - **Generated metadata, package, or installation impact:** none. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage **Write plans over 69 recorded captures**, built by the baseline and by this branch. The captures are the 50 published ones, 18 Claude Code recorded from a prompt for #6946's checks, and one new timed Grafana recording: | Captures | Write plans | Served on the baseline | Served on this branch | | ------------------------- | ----------- | ---------------------- | --------------------- | | 50 published | 187 | 68 | 68 | | 19 recorded from a prompt | 120 | 56 | 61 | - **The five gains:** Linkding's bookmark create, Homebox's tag create, and Grafana's folder create (two recordings) and rename (the timed one). - **No regressions:** no write goes from served to withheld. - **Published plans that change:** Linkding's bookmark create and BookStack's book create and update. Each composes a value and now accepts the member it reads, with the same served verdicts. - **Left as before on purpose:** - Grafana's published folder: its uid and title slug are the same text, so the address names two sources. - Ghost's `lexical`: keeps its `derived` rule. - BookStack's page HTML and Jellyfin's playlist path: other entities of the type spell them otherwise. **Served write tasks** on the same timed captures, rebuilt by the baseline and by this branch, with `fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash`. The create tasks are new. Each checks the created entity from the environment's state, including the composed address and that nothing else changed: | Application | Tasks | Baseline (gpt-oss, GLM) | This branch (gpt-oss, GLM) | | ----------- | ---------------------------------- | ----------------------- | -------------------------- | | Linkding | 2 bookmark creates | 0/2, 0/2 | 2/2, 2/2 | | Linkding | 3 recorded (update, notes, delete) | 3/3, 3/3 | 3/3, 3/3 | | Homebox | 2 tag creates | 0/2, 0/2 | 2/2, 2/2 | | Homebox | 2 recorded deletes | 0/2, 0/2 | 0/2, 0/2 | | Grafana | folder create and rename | not served | 1/2, 2/2 | | Grafana | 2 recorded deletes | | 2/2, 2/2 | - **Smoke and retrieval:** match on both builds. Grafana's new recording passes smoke and 9/9 retrieval with both models. - **gpt-oss on the Grafana rename:** it sent the update without the title it was asked to change, ten times. The served environment refused each call and named the members it accepts. - **Homebox's recorded deletes:** they fail on both builds. This recording's served entity list holds its locations only, so the models never find the item. - **How the captures were recorded and evaluated:** - Claude Code (`claude-sonnet-5-5`) recorded each app from a one-paragraph prompt on Ubuntu 24.04 VMs with snap Chromium 155. - Both the recording and the evaluation used a local, unpushed build of this branch merged with #6967, because browser scenarios need its request times for clock rules. The baseline is the same merge without this branch. **Tests added:** - **View:** Go-spelled tag creates and renames, composed bookmark snapshot addresses, and folder creates and renames with composed addresses and generated integer ids are served. - **Serving:** - a tag create's clocks drop trailing zeros; - a bookmark's snapshot address follows the served clock and the sent URL; - a folder gets a fresh sixteen-digit id and an address that follows its uid and title, and a rename moves it. - **Unit:** - clock format fitting, calendar digits, seeded integers and generated integer detection; - composition and each refusal (one call alone, a held entity spelling otherwise, two sources with one text, a member stored otherwise than sent, a before value the parts do not spell); - a wide body within the existing time limit. **Remaining proof gaps:** - A composed string copies sent text as sent. An application that escapes or renders it (HTML entities, Markdown) differs for text that needs it, which is documented as a served-write limit. - Calendar digits cover four patterns only. For evidence-bearing changes: - [ ] Observed, derived, and inferred claims remain distinguishable. - [ ] Artifact identity, source provenance, and failed attempts remain preserved. - [ ] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Current follow-up at `5fd87e9c6`, based on the author's `aeae47f7b` (macOS, Node 24.22.1 / npm 11.16.0): - Composition comparison verifies each response's own spelling, then compares the rendered parts or reuses the source fields' clock/ID decisions. It preserves business-input differences, rejects identical stale URLs, handles destination-before-source ordering, and counts each source once. The core composition formatter and generated-integer matcher are shared rather than duplicated in compare. - Create validation checks generated shapes and captured composed fields in one place, preserves omitted response fields as unknown, and now also refuses UUID-to-opaque-token drift. The author's later-create contradiction and integer-family fixes are retained. - **26 source files / 858 tests passed** for environment write/compare behavior. **3 compiled CLI/MCP boundary files / 36 tests passed**, including the new composed-bookmark `environment-compare` case and existing trimmed-clock case; input Evidence files are unchanged by those workflows. - `check:fast` (typecheck/lint), `format:check`, `docs:check`, and pre-push checks passed. Four added assertions fail on `aeae47f7b` and pass here: equal stale compositions, differing actual inputs, unproven generated-looking inputs, and UUID-to-token drift. - The author's 69-capture census and model evaluations below were not rerun after this follow-up. Fresh hosted CI and automated review are pending. Completed `environment-compare` CLI tasks with the same tag/bookmark captures and one create probe per case, before `aeae47f7b` versus this change. Complete canonical reports are identical for these valid tasks. | Task / invocation | Elapsed before → after | CPU before → after | Sampled peak process-family RSS before → after | | --- | --- | --- | --- | | Tag create, first fresh process | 0.604 → 0.564 s | 0.786 → 0.730 s | 365.4 → 387.4 MiB | | Tag create, median of 3 repeat processes | 0.493 → 0.473 s | 0.666 → 0.660 s | 375.0 → 377.2 MiB | | Bookmark create, first fresh process | 0.469 → 0.484 s | 0.667 → 0.659 s | 365.7 → 362.2 MiB | | Bookmark create, median of 3 repeat processes | 0.470 → 0.471 s | 0.657 → 0.656 s | 369.5 → 383.8 MiB | Each invocation starts a fresh CLI process; OS caches were not cleared. CPU is child-family resource usage; RSS is sampled at roughly 15 ms plus process-list overhead. No resource stop occurred. These small samples establish no speedup or memory improvement. Agent/model tokens and costs on larger responses were not measured; no token savings are claimed. <details> <summary>Earlier revision validation and application results</summary> Follow-up at `94f17534f` (2026-10-25), after merging main `2acdd5413`, on macOS with Node 24.22.1 and npm 11.16.0: - Addressed all five open threads: missing composition parts on earlier updates; held fixed-width evidence for update plans (two reports); trimmed-clock comparison; and order-independent propagation of the widest trimmed peer per separator/zone group. - Shared the domain clock matcher between learning and comparison, preserving parsed readings rather than reparsing values or duplicating the format policy. - Environment write/compare source tests: **26 files, 839 passed** at `5f8a954c6`; the subsequent main merge changes no environment source. - Relevant CLI/MCP/process boundaries, including the new environment-compare regression, managed bundle read guards, and setup registration prewarm: **6 distinct files, 77 passed**. Managed read guards, environment-compare CLI, and setup prewarm were rerun on `94f17534f`: **3 files, 21 passed**. - `check:fast` (typecheck/lint), `format:check`, `docs:check`, and the pre-push check: passed. - Main's #7036 fixes the managed Evidence low-headroom CI failure; the failing test now passes. Main independently fixed its setup test-length lint error in `2acdd5413`; the merge adopts that file verbatim, retaining all four scenarios and removing the temporary branch-specific fixture cleanup. `check:fast` passes again after the merge. - Representative completed CLI task: the same tag capture and single Go-style create probe on `fb0eece97` versus `5f8a954c6`. First invocation: **0.552 → 0.529 s elapsed**, **0.735 → 0.731 s CPU**, **362.5 → 365.9 MiB sampled peak process-family RSS**. Median of three repeat invocations: **0.519 → 0.500 s elapsed**, **0.713 → 0.703 s CPU**, **367.2 → 374.8 MiB RSS**. Each uses a fresh CLI process; OS caches were not cleared. RSS is sampled at roughly 15 ms plus process-list overhead. These small samples establish no speedup claim. The original reports two false clock-format differences; this head reports agreement. Agent/model tokens and broader performance costs were not measured. - Original application recordings and model evaluations below were not rerun for this follow-up. Hosted CI subsequently passed on the earlier head. Follow-up at `3705faeb3`, on macOS with Node 24.22.1: - Ports the #7034 finding that a generated integer repeated in another string (an echoed `serialText`, or a token like `folder-<id>`) is withheld unless a `template` or `composed` rule rebuilds that string from it; its tests fail on `94f17534f`, and a templated copy is still served. - `check:fast`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`, `scan:todos`, `jscpd` and `npx knip`: passed. `domain` + `services` + `adapters`: 17,323 passed, with the Inspector stat test failing as it does on main in this environment. The 10 focused boundary files: 82 passed. Managed bundle read guards: 11 passed. - Write plans over the 69 captures on main `2acdd541` and on this head: 124 → 129 served, the same five gains, no losses. The 50 published environments serve the same 68 writes with 68/68 smoke calls, and all 54 rebuilt bundles are byte-identical to the three-step path's. Follow-up at `aeae47f7b` (`e54b75a17` plus main `aced9fa9b`), on macOS with Node 24.22.1: - Withholds a later create whose response spells a composed field otherwise, or answers a field learned from a generated integer with anything but an integer of its length and kind; compare counts a composed field as equal when each side spells it from its own fields. Each new test fails on `3705faeb3`. - The same gates as above pass (`domain` + `services` + `adapters`: 17,358 passed, with the Inspector stat test failing as on main); the 10 focused boundary files: 82 passed. The 69-capture census on main `aced9fa9b` is still 124 → 129 with the same five gains, the 50 published environments are unchanged, and all 54 bundles are byte-identical. Earlier author validation: Run on macOS with Node v24.22.1 at `96783c72` (main `42577617` plus this branch). - `tsc`, `npm run -s check:fast`, `format:check`, `docs:check`, `mcp-catalog:check`, `verify:test-discovery`, `scan:todos` and `jscpd` (3.64%), and `npx knip`: passed. - `npx vitest run --project domain --project services --project adapters`: 17,274 passed and 1 failed ("stops Inspector target authorization after the current stat settles"), which fails the same way on main in this environment. - `npm run test:focused --` the environment build, export and serve CLI boundaries, the analysis-view CLI and MCP boundaries, the environment serve, served-writes and bundle MCP boundaries, MCP schema validation and output schema constraints: 10 files, 82 passed. - Write plans over the 69 captures, rebuilt on main `42577617` and on this head: 124 → 129 served, with the same five gains and no write going from served to withheld. The review fixes change no plan in these captures, so the served-write task results above, measured before the review fixes, still describe this head. - The 50 published environments serve the same 68 writes with 68/68 smoke calls and the same session and served reads, and all 54 rebuilt bundles are byte-identical to the three-step path's. - Each review fix has a test that fails on the head it was reported on (`8ce1127b` or `09c48c59`). - **Earlier CI failure from main (now fixed by #7036, verified above):** `tests/process-global/managedBundleReadGuards.test.ts` ("projects a rejected managed parent by reference without cloning under low headroom") failed from `9b3a6332` through the earlier main `42577617`, with and without this branch. </details> ## Compatibility, safety, and release - **Breaking changes or migration steps:** none for callers. Bundles and state views built by this version need this version or later to serve. - **Real Hopper/Ghidra, browser, or OS coverage:** real applications on Ubuntu 24.04 with snap Chromium 155 for the recordings; macOS for gates. - **Package or release metadata impact:** none. - **Security, privacy, process, or containment review:** reads recorded captures only. A composed value repeats text the call sent or the state already holds. ## Author readiness Human owner: @earakely-scale - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. Part of #4933. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:38:53 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7584

No description provided.

Rows
Columns