morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3134] test(perf): verify efficient paging of large retained investigations #1095

New issue

Closed

opened 2026-10-15 18:31:01 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-15 18:31:01 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3134

Originally assigned to: @N0zoM1z0 on GitHub.

Follow-up to #2016, whose original three-stage implementation and scoped verification are complete. This issue owns only the bounded continuation below. #3010 remains the owner of managed-member delivery changes and #1065 of broader MCP representation costs; this task measures existing retained workflows rather than duplicating those implementations.

Analyst question and outcome

When an analyst resumes a large retained investigation, can existing unknown-head/Evidence paging answer a bounded question with less model context and latency without rerunning producers or changing record lifetime?

Existing evidence and decision boundary

#2016 validated specific inventory/WASM/EVM/managed/Android/Flutter views and old-Evidence continuation. Large mixed investigation histories and existing unknown-head paging remain an unmeasured continuation; an additional generic projection/paging API is not established.

Bounded work

Build two bounded histories from real shipped-artifact workflows already available in the public CLI/MCP surface, such as a native investigation and a packaged JavaScript inventory/static investigation. Archive authentic original bundles and producer/runtime pins; synthetic large arrays are not real-workflow evidence.
Fix a question requiring selected records and their uncertainty/provenance, then compare existing complete reads with exact retained paging. Include discovery, duplicate transport/text representations and follow-up calls in costs. Verify examined coverage rather than deriving completeness from displayed rows.
Prove producer reuse through retained original records or observable producer-call accounting. Export/import authentic old bundles and verify record IDs, links, omissions, scalar types, errors and limits. Follow the currently documented close/retention semantics; do not add implicit persistence or eviction.
Run repeated SDK and representative actual Codex pairs. If existing paging already meets the question cheaply, record that result. Any new projection or cache needs a separate evidence-backed implementation proposal.

Acceptance

Real source workflows, authentic bundles, exact questions and current lifetime/schema semantics are pinned.
Existing complete and selected views preserve equivalent facts, original IDs/links/coverage and old-Evidence continuation.
Producer reuse and output/resource capacity are verified; unknown counters and partial/unavailable records remain explicit.
Repeated SDK/model costs include discovery, all pages, retries and factual/workflow grades; record a no-change or justified next-step decision.

Compatibility, measurement and host constraints

Token consumption and time to a correct answer are the primary outcomes. Record total/cached/uncached input, output, discovery, RPCs, retries and failed attempts separately from JSON bytes. Preserve negative results; a smaller response alone is not a speed or token win. Separate observed producer/transport/client work and unknown stages, and compare equivalent completed questions.

Keep current complete defaults, caller-selected inputs/providers, source bytes, Evidence identity/links, omission/null/type distinctions, partial coverage, and lifecycle/export/import behavior. Inspect the current advertised contracts and reuse existing selectors/views before adding code. Any implementation needs measured benefit and relevant compatibility verification; a documented no-change decision is a successful research outcome. Run checks for affected boundaries rather than repeating unrelated suites after each small correction.

Run heavy work serially under one CPU, 3 GiB early stop / 4 GiB hard memory, no swap, owned scopes and host-availability checks. Distinguish verified payload-phase observations from unavailable terminal whole-scope counters; missing counters stay unknown. For performance experiments, use real public CLI/MCP workflows and actual Codex CLI tasks when assessing agent token/time outcomes, with a private temporary profile pointing at the tested REA MCP; pin the requested model/reasoning and runtime within each cohort. Planning or demand-only work does not require an unrelated model/provider matrix. Simulated providers do not establish real-provider behavior. Keep raw captures/binaries/replies/transcripts/configuration private, commit source and curated reproducible findings, and record unrelated product bugs in the existing /tmp team handoff. No release is part of this task.

Originally created by @N0zoM1z0 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3134 Originally assigned to: @N0zoM1z0 on GitHub. Follow-up to #2016, whose original three-stage implementation and scoped verification are complete. This issue owns only the bounded continuation below. #3010 remains the owner of managed-member delivery changes and #1065 of broader MCP representation costs; this task measures existing retained workflows rather than duplicating those implementations. ## Analyst question and outcome When an analyst resumes a large retained investigation, can existing unknown-head/Evidence paging answer a bounded question with less model context and latency without rerunning producers or changing record lifetime? ## Existing evidence and decision boundary #2016 validated specific inventory/WASM/EVM/managed/Android/Flutter views and old-Evidence continuation. Large mixed investigation histories and existing unknown-head paging remain an unmeasured continuation; an additional generic projection/paging API is not established. ## Bounded work 1. Build two bounded histories from real shipped-artifact workflows already available in the public CLI/MCP surface, such as a native investigation and a packaged JavaScript inventory/static investigation. Archive authentic original bundles and producer/runtime pins; synthetic large arrays are not real-workflow evidence. 2. Fix a question requiring selected records and their uncertainty/provenance, then compare existing complete reads with exact retained paging. Include discovery, duplicate transport/text representations and follow-up calls in costs. Verify examined coverage rather than deriving completeness from displayed rows. 3. Prove producer reuse through retained original records or observable producer-call accounting. Export/import authentic old bundles and verify record IDs, links, omissions, scalar types, errors and limits. Follow the currently documented close/retention semantics; do not add implicit persistence or eviction. 4. Run repeated SDK and representative actual Codex pairs. If existing paging already meets the question cheaply, record that result. Any new projection or cache needs a separate evidence-backed implementation proposal. ## Acceptance - [ ] Real source workflows, authentic bundles, exact questions and current lifetime/schema semantics are pinned. - [ ] Existing complete and selected views preserve equivalent facts, original IDs/links/coverage and old-Evidence continuation. - [ ] Producer reuse and output/resource capacity are verified; unknown counters and partial/unavailable records remain explicit. - [ ] Repeated SDK/model costs include discovery, all pages, retries and factual/workflow grades; record a no-change or justified next-step decision. ## Compatibility, measurement and host constraints Token consumption and time to a correct answer are the primary outcomes. Record total/cached/uncached input, output, discovery, RPCs, retries and failed attempts separately from JSON bytes. Preserve negative results; a smaller response alone is not a speed or token win. Separate observed producer/transport/client work and unknown stages, and compare equivalent completed questions. Keep current complete defaults, caller-selected inputs/providers, source bytes, Evidence identity/links, omission/null/type distinctions, partial coverage, and lifecycle/export/import behavior. Inspect the current advertised contracts and reuse existing selectors/views before adding code. Any implementation needs measured benefit and relevant compatibility verification; a documented no-change decision is a successful research outcome. Run checks for affected boundaries rather than repeating unrelated suites after each small correction. Run heavy work serially under one CPU, 3 GiB early stop / 4 GiB hard memory, no swap, owned scopes and host-availability checks. Distinguish verified payload-phase observations from unavailable terminal whole-scope counters; missing counters stay unknown. For performance experiments, use real public CLI/MCP workflows and actual Codex CLI tasks when assessing agent token/time outcomes, with a private temporary profile pointing at the tested REA MCP; pin the requested model/reasoning and runtime within each cohort. Planning or demand-only work does not require an unrelated model/provider matrix. Simulated providers do not establish real-provider behavior. Keep raw captures/binaries/replies/transcripts/configuration private, commit source and curated reproducible findings, and record unrelated product bugs in the existing `/tmp` team handoff. No release is part of this task.

bsakaguchi added the

labels

2026-10-15 18:31:01 +00:00

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-10-17 21:01:35 +00:00

bsakaguchi commented

2026-10-17 21:01:36 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 17, 2026):

Release checkpoint: implementation/verification artifacts and both review fixes are committed and pushed in #4814 (a9f8dff11). This batch changes experimental verification/reporting only, with no production runtime change.

Authentic SQLite ELF and npm retained queries, mixed history, exact record/Unknown revision round trips, stale-update and tamper refusals, and close/reference lifetime were verified. Local fast/docs checks and 36 CLI/MCP boundary + 34 agent-context + 4 receipt-regression tests passed. The report retains 10 successful and 2 failed agent tasks; no universal model-token/speed advantage is claimed.
Latest Greptile review is 5/5 with no new actionable findings; both original threads are resolved. Codex review is unavailable due to repository quota.
Merge is still blocked: current-head Build package fails in the portable-build-cache snapshot on missing .turbo/cache/ebb12def75fdaae8.tar.zst, matching main's failure. Dependent package/engine shards are skipped, so broad CI verification is incomplete. A normal squash merge was refused by branch policy; no bypass or automatic merge was enabled.

Per the owner's release-pause instruction, stop here. #4810 records the separate imported-format validation-reuse bottleneck; no implementation has started. Apktool (#4777), JEB and Hopper remain queued for an explicit resumption. Raw evidence and resource receipts are retained locally; curated results and repeatable checks are in the PR. Resource-guarded local work has ended.

@N0zoM1z0 commented on GitHub (Oct 17, 2026): Release checkpoint: implementation/verification artifacts and both review fixes are committed and pushed in #4814 (`a9f8dff11`). This batch changes experimental verification/reporting only, with no production runtime change. - Authentic SQLite ELF and npm retained queries, mixed history, exact record/Unknown revision round trips, stale-update and tamper refusals, and close/reference lifetime were verified. Local fast/docs checks and 36 CLI/MCP boundary + 34 agent-context + 4 receipt-regression tests passed. The report retains 10 successful and 2 failed agent tasks; no universal model-token/speed advantage is claimed. - Latest Greptile review is 5/5 with no new actionable findings; both original threads are resolved. Codex review is unavailable due to repository quota. - Merge is still blocked: current-head [Build package](https://github.com/morluto/rea/actions/runs/39043791728/job/117428449211) fails in the portable-build-cache snapshot on missing `.turbo/cache/ebb12def75fdaae8.tar.zst`, matching [main's failure](https://github.com/morluto/rea/actions/runs/39042013698/job/117424201017). Dependent package/engine shards are skipped, so broad CI verification is incomplete. A normal squash merge was refused by branch policy; no bypass or automatic merge was enabled. Per the owner's release-pause instruction, stop here. #4810 records the separate imported-format validation-reuse bottleneck; no implementation has started. Apktool (#4777), JEB and Hopper remain queued for an explicit resumption. Raw evidence and resource receipts are retained locally; curated results and repeatable checks are in the PR. Resource-guarded local work has ended.

bsakaguchi referenced this issue

2026-10-17 21:08:33 +00:00

[PR #1095] [MERGED] fix(apple): decode Interface Builder XML byte encodings #2644

bsakaguchi referenced this issue

2026-10-17 21:08:39 +00:00

[PR #1104] [MERGED] fix(apple): report missing keyed archive hierarchy references #2652

bsakaguchi closed this issue

2026-10-31 00:29:55 +00:00

bsakaguchi commented

2026-10-31 00:29:56 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 18, 2026):

Resumed on 2026-10-18 and merged main 2183c9598 into #4814 without conflicts (head e3caff367). The previous cache-snapshot blocker is fixed: Build package now passes. Fresh fast/build/docs checks, 36 exact CLI/MCP boundary tests, 34 context + 4 receipt regressions and 5 README consistency tests pass. Both authentic mixed-history scenarios pass again with unchanged answer digests/original records, all 64 follow-up heads/revision two, stale/tampered refusal, close/reference lifetime and verified cleanup. Local validation ran serially under the 8 GiB family cap, peaked at 2.70 GiB, with no payload OOM. Latest Greptile review is 5/5; no unresolved review threads.

A separate baseline blocker remains: formatting src/domain/propertyListXml.test.ts fails identically on main and the refreshed PR. That file is outside this PR and is still byte-identical at newest observed main bbfe6edd6. A normal squash merge was attempted and refused by branch policy. Source/build/docs CI pass; five package test shards are still running. No unrelated formatting fix or admin bypass was made. #4810 implementation, Apktool, JEB and Hopper remain unstarted, preserving the requested merge-before-next sequence.

@N0zoM1z0 commented on GitHub (Oct 18, 2026): Resumed on 2026-10-18 and merged main `2183c9598` into #4814 without conflicts (head `e3caff367`). The previous cache-snapshot blocker is fixed: Build package now passes. Fresh fast/build/docs checks, 36 exact CLI/MCP boundary tests, 34 context + 4 receipt regressions and 5 README consistency tests pass. Both authentic mixed-history scenarios pass again with unchanged answer digests/original records, all 64 follow-up heads/revision two, stale/tampered refusal, close/reference lifetime and verified cleanup. Local validation ran serially under the 8 GiB family cap, peaked at 2.70 GiB, with no payload OOM. Latest Greptile review is 5/5; no unresolved review threads. A separate baseline blocker remains: formatting `src/domain/propertyListXml.test.ts` fails identically on [main](https://github.com/morluto/rea/actions/runs/39083000773/job/117545072989) and [the refreshed PR](https://github.com/morluto/rea/actions/runs/39083547956/job/117545347757). That file is outside this PR and is still byte-identical at newest observed main `bbfe6edd6`. A normal squash merge was attempted and refused by branch policy. Source/build/docs CI pass; five package test shards are still running. No unrelated formatting fix or admin bypass was made. #4810 implementation, Apktool, JEB and Hopper remain unstarted, preserving the requested merge-before-next sequence.

bsakaguchi commented

2026-10-31 00:29:56 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 18, 2026):

Completed at the release checkpoint: #4814 was squash-merged as 47d3c6b1a15c166091b47666c9cac6dbd882454c after all final-head CI checks passed, including all five test shards, portable aggregate, native package/platform verification and the refreshed title check. Greptile is 5/5; all review threads are resolved. Both requested co-author trailers are present in the actual merged commit.

The final authentic SQLite/npm mixed workflows preserve original records and answer digests, check all 64 follow-up heads/revision two, reject stale/tampered changes and verify reference/Unknown lifetime and cleanup. Local focused verification passed 89 tests; all 10 RPC/finalization/publication regressions pass. Production runtime behavior is unchanged. Full evidence/limits are in the committed checkpoint; this does not claim a general agent-token/speed improvement.

Stopping here as requested for the release. #4810 remains an open follow-up proposal with no implementation started; Apktool, JEB and Hopper remain queued. All local guarded work has ended, and raw evidence/receipts are preserved for resumption.

@N0zoM1z0 commented on GitHub (Oct 18, 2026): Completed at the release checkpoint: #4814 was squash-merged as `47d3c6b1a15c166091b47666c9cac6dbd882454c` after all final-head CI checks passed, including all five test shards, portable aggregate, native package/platform verification and the refreshed title check. Greptile is 5/5; all review threads are resolved. Both requested co-author trailers are present in the actual merged commit. The final authentic SQLite/npm mixed workflows preserve original records and answer digests, check all 64 follow-up heads/revision two, reject stale/tampered changes and verify reference/Unknown lifetime and cleanup. Local focused verification passed 89 tests; all 10 RPC/finalization/publication regressions pass. Production runtime behavior is unchanged. Full evidence/limits are in the committed checkpoint; this does not claim a general agent-token/speed improvement. Stopping here as requested for the release. #4810 remains an open follow-up proposal with no implementation started; Apktool, JEB and Hopper remain queued. All local guarded work has ended, and raw evidence/receipts are preserved for resumption.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1095

No description provided.

Rows
Columns