morluto/rea

Fork

You've already forked rea

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

Code Issues 1.3k Projects Packages Wiki Activity

[PR #5350] [MERGED] perf(environment): index observed-state write cutoffs #6886

New issue

Closed

opened 2026-11-01 21:43:50 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:43:50 +00:00

Owner

Copy link

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5350
Author: @gensokyo-dev
Created: 10/20/2026
Status: ✅ Merged
Merged: 10/20/2026
Merged by: @gensokyo-dev

Base: main ← Head: gensokyo-dev/state-write-cutoffs-20261020

📝 Commits (1)

27e3d5c perf(environment): index observed-state write cutoffs

📊 Changes

6 files changed (+314 additions, -28 deletions)

View changed files

📝 docs/specs/agent-environments/state-view/verification.md (+39 -1)
📝 src/domain/analysisView/EnvironmentStateExtraction.ts (+30 -15)
📝 src/domain/analysisView/environmentStateRules.ts (+0 -12)
📝 src/domain/analysisView/environmentStateView.test.ts (+58 -0)
➕ src/domain/analysisView/environmentStateWriteCutoffs.test.ts (+86 -0)
➕ src/domain/analysisView/environmentStateWriteCutoffs.ts (+101 -0)

📄 Description

Summary

State extraction reparsed every write root for each read type and repeated the search while assembling output. It now admits one prefix/wildcard index per projection, caches each type’s cutoff (including no match), and derives backed type membership once.

Follow-up to the final performance review on #5319. That PR was merged while this optimization was being validated; this diff is based on current main 3fca2e0ca.

Problem and expected behavior

With 1,000 collection tools and 1,000 unrelated writes, repeated root parsing costs CPU even though the projected state is unchanged. The same completed file-to-state task changed from 1,301 ms warm / 1,300 ms CPU to 624 ms warm / 624 ms CPU, with identical parent IDs and normalized output digests. First-call and process-family memory results are in verification.md; no general memory reduction is claimed.

Change and scope

Index write roots once, preserve ancestor/descendant and wildcard matching, and choose the earliest write using the existing timeline owner.
Admit index nodes and cutoff/type sets before retaining them; use iterative lookup for deep paths. Keep the 96 MiB representation budget.
Cache positive and negative per-type cutoffs within one projection. Remove the superseded linear matcher and repeated output plan scan.

Contract and boundary impact

Semantic owner: analysis-view domain, EnvironmentStateExtraction and StateWriteCutoffs.
Scope decision: not applicable; routine optimization of existing read-only capture analysis.
CLI/MCP contract, providers, formats and installation: unchanged; no new capability or dependency.
Evidence and reconstruction: equivalent complete outputs; original ordinals, uncertainty and inference semantics remain intact.
Target/cleanup authority: unchanged. All new state is bounded to the projection; no cross-request cache.
Generated outputs: regenerated and ignored as usual; no generated catalog or binary committed.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.

Evidence and regression coverage

Prefix, parameter, redacted-root and timestamp-order semantics; independent linear matching oracle; 12,000-segment lookup; existing capacity tests.
Same authenticated Evidence and complete normalized output digest in the before/after workload.
Real Gitea 28.2.0 HAR CRUD/search captures and fresh Chromium scenario capture, including file CLI/default stdio MCP parity and six exact HTTP replays.
Model token usage was not measured; no new macOS/Windows or state-serving claim.

Validation performed

Relevant domain/application/contracts plus capture validation: 41 files, 508 passed, 3 existing skips.
Serial static/build/artifact gate: 11 tasks passed; both selected compiled state CLI/MCP cases passed.
Docs/catalog/format checks: 9 tasks passed; architecture guards passed across 1,603 source files; README consistency: 5 tests passed.
One CPU/worker throughout; static/build peak process-family RSS 4,745 MiB. No workspace OOM.
GitHub CI: all selected lanes passed, including source tests, all five built test shards, portable aggregation, AgentEnv bundle execution and Linux native package verification.
Greptile reviewed 27e3d5cd61: 5/5, no actionable findings. Codex cloud review was unavailable because its review quota was exhausted.
The separate human-readiness declaration check is intentionally not attested and is bypassed under the operator’s explicit standing instruction.

Compatibility, safety, and release

No public schema, transport default, target authority or write-effect change. Matching and complete output digests are preserved. This optimization uses the existing first-related-recorded-write model and retains its documented inference limitations.

Author readiness

Human owner: @gensokyo-dev

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Submitted ready under the repository operator’s explicit instruction. Human-review attestations remain unchecked.

Before merge

Concrete performance problem and linked original review.
Existing read-only scope and authority preserved.
Focused conventional title, semantic regression checks and current validation.
Owning verification document updated; no secrets or unrelated changes.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5350 **Author:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Created:** 10/20/2026 **Status:** ✅ Merged **Merged:** 10/20/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `gensokyo-dev/state-write-cutoffs-20261020` --- ### 📝 Commits (1) - [`27e3d5c`](https://github.com/morluto/rea/commit/27e3d5cd61cd7d9136b2ae64764f5f7d48fc6b84) perf(environment): index observed-state write cutoffs ### 📊 Changes **6 files changed** (+314 additions, -28 deletions) <details> <summary>View changed files</summary> 📝 `docs/specs/agent-environments/state-view/verification.md` (+39 -1) 📝 `src/domain/analysisView/EnvironmentStateExtraction.ts` (+30 -15) 📝 `src/domain/analysisView/environmentStateRules.ts` (+0 -12) 📝 `src/domain/analysisView/environmentStateView.test.ts` (+58 -0) ➕ `src/domain/analysisView/environmentStateWriteCutoffs.test.ts` (+86 -0) ➕ `src/domain/analysisView/environmentStateWriteCutoffs.ts` (+101 -0) </details> ### 📄 Description ## Summary State extraction reparsed every write root for each read type and repeated the search while assembling output. It now admits one prefix/wildcard index per projection, caches each type’s cutoff (including no match), and derives backed type membership once. Follow-up to [the final performance review on #5319](https://github.com/morluto/rea/pull/5319#discussion_r4321115857). That PR was merged while this optimization was being validated; this diff is based on current main `3fca2e0ca`. ## Problem and expected behavior With 1,000 collection tools and 1,000 unrelated writes, repeated root parsing costs CPU even though the projected state is unchanged. The same completed file-to-state task changed from 1,301 ms warm / 1,300 ms CPU to 624 ms warm / 624 ms CPU, with identical parent IDs and normalized output digests. First-call and process-family memory results are in [verification.md](docs/specs/agent-environments/state-view/verification.md); no general memory reduction is claimed. ## Change and scope - Index write roots once, preserve ancestor/descendant and wildcard matching, and choose the earliest write using the existing timeline owner. - Admit index nodes and cutoff/type sets before retaining them; use iterative lookup for deep paths. Keep the 96 MiB representation budget. - Cache positive and negative per-type cutoffs within one projection. Remove the superseded linear matcher and repeated output plan scan. ## Contract and boundary impact - Semantic owner: analysis-view domain, `EnvironmentStateExtraction` and `StateWriteCutoffs`. - Scope decision: not applicable; routine optimization of existing read-only capture analysis. - CLI/MCP contract, providers, formats and installation: unchanged; no new capability or dependency. - Evidence and reconstruction: equivalent complete outputs; original ordinals, uncertainty and inference semantics remain intact. - Target/cleanup authority: unchanged. All new state is bounded to the projection; no cross-request cache. - Generated outputs: regenerated and ignored as usual; no generated catalog or binary committed. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. ## Evidence and regression coverage - Prefix, parameter, redacted-root and timestamp-order semantics; independent linear matching oracle; 12,000-segment lookup; existing capacity tests. - Same authenticated Evidence and complete normalized output digest in the before/after workload. - Real Gitea 28.2.0 HAR CRUD/search captures and fresh Chromium scenario capture, including file CLI/default stdio MCP parity and six exact HTTP replays. - Model token usage was not measured; no new macOS/Windows or state-serving claim. ## Validation performed - Relevant domain/application/contracts plus capture validation: 41 files, 508 passed, 3 existing skips. - Serial static/build/artifact gate: 11 tasks passed; both selected compiled state CLI/MCP cases passed. - Docs/catalog/format checks: 9 tasks passed; architecture guards passed across 1,603 source files; README consistency: 5 tests passed. - One CPU/worker throughout; static/build peak process-family RSS 4,745 MiB. No workspace OOM. - [GitHub CI](https://github.com/morluto/rea/actions/runs/39367372967): all selected lanes passed, including source tests, all five built test shards, portable aggregation, AgentEnv bundle execution and Linux native package verification. - Greptile reviewed `27e3d5cd61`: 5/5, no actionable findings. Codex cloud review was unavailable because its review quota was exhausted. - The separate human-readiness declaration check is intentionally not attested and is bypassed under the operator’s explicit standing instruction. ## Compatibility, safety, and release No public schema, transport default, target authority or write-effect change. Matching and complete output digests are preserved. This optimization uses the existing first-related-recorded-write model and retains its documented inference limitations. ## Author readiness Human owner: @gensokyo-dev - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. Submitted ready under the repository operator’s explicit instruction. Human-review attestations remain unchecked. ## Before merge - [x] Concrete performance problem and linked original review. - [x] Existing read-only scope and authority preserved. - [x] Focused conventional title, semantic regression checks and current validation. - [x] Owning verification document updated; no secrets or unrelated changes. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:43:50 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:47:49 +00:00

[PR #6886] test: keep deliberate fixture signals out of systemd-coredump #7432

No Branch/Tag specified

main

codex/procedure-reference-stacked

release-please--branches--main--components--rea-agents

release-please--branches--main--components--rea-agents--release-notes

codex/kuna-independent-retry

codex/byte-search-overlay-coverage

codex/open-failure-target-receipt

codex/kuna-xref-evidence

codex/ghidra-ambiguous-procedure-recovery

codex/function-source-bundle

codex/trace-function-integration-prototype

codex/snapshot-source-closure

codex/recorded-evidence-array-pages

codex/evidence-recovery-redesign

codex/trace-feature-address-scope

codex/call-discovery-scope

codex/ghidra-literal-candidate-evidence

codex/decompile-directory-independent-retry

codex/retry-provenance-base

codex/value-flow-capability-experiment

codex/ghidra-containing-instruction

cursor/readme-discord-button-ef9f

dependabot/github_actions/actions/setup-node-7.1.0

dependabot/github_actions/actions/upload-artifact-7.0.2

dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df

codex/kuna-1989

fix/8559-native-ui-fixture-ownership

fix/8555-go-verifier-mcp-ownership

codex/provider-capability-recovery

fix/8642-rizin-target-stdin

codex/ghidra-memory-effect-scope

codex/ghidra-annotation-entry-selector

codex/ghidra-readonly-recovery

codex/ghidra-regex-verifier-recovery

codex/evidence-discovery-view-guidance

fix/8545-bulk-append-overflow

fix/8428-navigation-refusal-alternatives

codex/address-context-recovery

codex/matching-member-limitations

codex/unknown-scope-contract

codex/evidence-view-guidance

feat/8373-native-trace-views

codex/ghidra-snapshot-verifier-binding

codex/elf-export-visibility

codex/evidence-view-recovery

codex/native-reference-cli-help

codex/provider-candidate-observation-scope

codex/kuna-session-verification

codex/ghidra-switch-lifetime-guidance

codex/pseudocode-paging-discovery

codex/clarify-evidence-view-support

codex/kuna-xrefs-selector-recovery

codex/context-capacity-recovery

codex/evidence-closure-large-links

codex/same-function-evidence-identity

codex/ghidra-verifier-inspected-jdk

codex/audit-guided-workflows

codex/crash-prompt-retained-dossiers

codex/analysis-view-error-recovery

codex/kuna-verifier-target-selection

codex/ghidra-selected-project-call-coverage

codex/docs-native-analysis-scope

codex/ghidra-java-socket-boundary

codex/ghidra-retained-snapshot-verifier

codex/ghidra-search-inventory-facts

fix/8284-zero-argument-parameter-bindings

fix/8198-ui-action-handler-offset

fix/8257-elf-entry-label-identity

fix/8304-deferred-output-schema-projection

fix/8277-call-path-computed-target-uncertainty

fix/pr-8089-conflicts

fix/ci-static-duplicate-export

dependabot/npm_and_yarn/lint-staged-17.6.0

dependabot/github_actions/actions/download-artifact-8.0.2

dependabot/npm_and_yarn/write-file-atomic-8.0.0

dependabot/npm_and_yarn/canonicalize-5.1.0

dependabot/github_actions/actions/cache/restore-6.1.0

dependabot/npm_and_yarn/toon-format/toon-4.4.0

dependabot/github_actions/actions/deploy-pages-5.0.1

codex/review-fixes-pr-2403

codex/review-fixes-pr-2400

codex/process-terminal-width

codex/test-brittleness-cleanup

verify/7167-browser-connect-macos

fix/7167-browser-connect

feat/native-matching-allowance-setting

feat/browser-scenario-dry-run

test/managed-caller-parameters-20261024

codex/parser-fortythird-boundaries

fix/6926-pinvoke-platform-decorations-20261023

fix/6889-har-body-encoding-20261023

fix/6888-sqlite-schema-budget-20261023

fix/dependency-security-gates-6708

cursor/evidence-overwrite-rebase-b4ac

feat/4057-verify-native-exports

fix/windows-acl-observer-overhead-20261022

cursor/erlang-beam-restack-4476

cursor/pe-certificate-review-bf32

fix/6116-javascript-application-fs-errors

fix/6061-unmapped-runtime-script-unknown

feat/browser-scenario-summary

fix/pe-certificates

codex/verify-3407

cursor/rebase-5699

cursor/rebase-5352

cursor/rebase-5600

cursor/fix-cli-schema-omission-10a8

codex/audit-hopper-shutdown-reasons-20261021

fix/demangle-swift-omitted-record

fix/nib-boolean-value-types

cursor/cancellation-partial-observation-a5ac

fix/ib-object-limit-reasons

fix/electron-fuse-limit-advice

fix/kuna-doctor-dangling-cache

fix/fixed-budget-refusal-advice

fix/js-ipc-listener-dedup

fix/js-return-member-provenance

fix/dmg-integrity-retry-advice

fix/ida-doctor-readiness

fix/kuna-version-probe-output-drain

fix/legacy-swift-runtime-name-matching

fix/dylib-lazy-load-commands

fix/browser-scenario-startup-cancel

fix/swift-nested-punycode-classification

fix/compare-unselected-scopes

fix/objc-method-budget-coverage

fix/dmg-mounted-root-observations

audit/ci-tests-20261019-round2

release/6.7.0

fix/wabt-cleanup-retry-resolution

fix/electron-fuse-elf-pe-architecture

fix/native-plist-descriptor-ownership

fix/kuna-caller-cleanup-ownership

experiment/atomic-codemode-20261016

review/first-contributor-3776

fix/call-resolution-full-read-cycles

fix/member-mutation-selection-repeat

experiment/atomic-re-tools-20261015

perf/member-mutation-lazy-origins

perf/static-network-mutation-index

fix/hopper-startup-lifecycle

audit/3050-windows-proof-20261015

feat/windows-window-capture-2183

audit/3049-windows-proof-20261015

audit/3735-macos-fixtures-20261015

fix/batch-decompile-null-pseudocode

fix/kuna-inventory-honors-jobs

experiment/browser-codemode-tool-design-20261015

fix/hopper-clean-runtime-argv

codex/review-2049-27f1b367

review-fixes/pr-2787-routing

fix/3360-scoped-this-emitters

fix/3367-replaced-global-effects

fix/3838-compact-any-value-items

test/hopper-download-probe-20261015

codex/windows-runtime-creation-red-green-20261015

codex/windows-readonly-root-red-green-20261015

codex/windows-root-share-red-green-20261015

codex/exp-1685-win-mcp-startup

codex/bug-sweep-20261015

investigate/2190-windows-package-phases

investigate/3543-macos-cli-deadline

perf/3281-reverse-tools-feedback-quality

fix/3406-hopper-doctor-fixture

fix/3341-wrapper-require-test-context

fix/3301-owned-uninstall-fixtures

fix/3164-compiler-input-ownership

fix/pcap-cleanup-stdout-newline-20261014

perf/interface-builder-summary

codex/review-fixes-pr-1977

fix/ghidra-effective-jvm-platform

fix/docs-turkish-rust-link-20261013

fix/main-maintenance-ci-20261010

fix/recover-binary-session-process-effects

codex/review-fixes-pr-2457

codex/ci-required-cancellation-gate

fix/main-format-baseline-20261013

fix/setup-preserve-entry-fields

cursor/restore-main-checks-1ffc

fix/ghidra-bridge-hashmap-import

codex/shared-ci-jadx-json-default-20261013

codex/shared-ci-ghidra-test-groups-20261013

codex/shared-ci-java-fixture-format-20261013

codex/fix-intrinsic-invocation-boundaries

test/release-preflight-reusable-jobs

test/js-parameter-provenance-expectation

fix/linux-hopper-verification-current

test/2419-ghidra-elf-probe-size

followup/pr-1544-native-ida

codex/fold-pr-2230-contribution

docs/readme-responsible-use

fix/issue-2064

fix/issue-2050

fix/issue-2053

fix/issue-1968

fix/issue-1945

experiment/codemode-bridge-20261011

fix/tool-effects-negative-wording-1904-20261012

fix/1842-electron-interop-provenance

feat/recover-javascript-multi-input

refactor/website-i18n-prep

fix/nativeaot-pe-virtual-extent

fix/browser-verifier-ci-hang

fix/pe-resources-input-selection

docs/beginner-friendly-guides

docs/beginner-setup-guide

fix/javascript-analysis-worker-1563

fix/1649-ghidra-duplicate-names

fix/javascript-export-keys-integration-1369

test/maintenance-boundary-contracts-20261011

review/javascript-export-keys-1369

review/process-live-progress-1483

test/javascript-1495-growth-ratio

fix/ci-escape-timing

docs/normalized-terminal-geometry-1450

release/6.3.0

archive/release-6.3.0-pr1381

release/6.2.0

fix/electron-preload-dynamic-override

fix/javascript-self-referential-bundler-edges

release/6.1.0

cursor/grok-client-cherry-pick-0b56

feat/signature-posture

feat/dyld-shared-cache

feat/artifact-integrity-policy

feat/pkg-inventory

release/6.0.0

chore/sync-release-5.0.0

release/5.0.0

cursor/cloud-agent-node-787e

fix/javascript-xhr-open-endpoints

review/pr-711-realpath-fallback-wording

cleanup/replay-confinement

codex/test-suite-performance

codex/issue-foundations

codex/address-open-issues-2026-07-20

agent/javascript-vite-rollup-esbuild-adapters

agent/process-cleanup

agent/triage-integration

agent/unknown-commands-registry

feat/mcp-identity-metadata

morlvps/issue_cleanup

codex/installer-redesign

morl/roadmap-investigation-core

morl/roadmap-investigation-docs

release-please--branches--main--components--rea

rea-agents-6.8.0

rea-agents-6.7.0

rea-agents-6.6.0

experiment/atomic-codemode-wait-runtime-20261016

rea-agents-6.5.0

rea-agents-6.4.0

rea-agents-6.3.0

rea-agents-6.2.0

rea-agents-6.1.0

rea-agents-6.0.0

rea-agents-5.0.0

rea-agents-4.1.0

rea-agents-4.0.1

rea-agents-4.0.0

rea-agents-3.2.1

rea-agents-3.2.0

rea-agents-3.1.0

rea-agents-3.0.0

rea-agents-2.7.0

rea-agents-2.6.0

rea-agents-2.5.1

rea-agents-2.5.0

rea-agents-2.4.0

rea-agents-2.3.0

rea-agents-2.2.0

rea-agents-2.1.0

rea-agents-2.0.0

rea-agents-1.7.0

rea-agents-1.6.0

rea-agents-1.5.0

rea-agents-1.4.0

rea-agents-1.3.0

rea-agents-1.2.0

rea-agents-1.1.0

rea-agents-1.0.0

rea-agents-0.5.0

rea-agents-0.4.0

rea-agents-0.3.0

rea-0.2.1

rea-0.2.0

Labels

Clear labels

Mirrored from GitHub Pull Request

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6886

Reference in a new issue

Repository

morluto/rea

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns