morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #4814] [MERGED] test(perf): verify authentic retained investigation queries #6422

New issue

Closed

opened 2026-10-31 01:14:21 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:14:21 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4814
Author: @N0zoM1z0
Created: 10/17/2026
Status: ✅ Merged
Merged: 10/18/2026
Merged by: @N0zoM1z0

Base: main ← Head: perf/3134-retained-investigation-20261017

📝 Commits (6)

5d4abd6 test(perf): measure authentic retained investigation queries
a9f8dff test(perf): retain failed RPC costs and repeat mixed checks
e3caff3 Merge latest main to verify retained investigation checks
b025a91 Merge main CI repairs for retained investigation verification
5d6259e test(perf): preserve retained receipts through cleanup errors
5f5df9c test(perf): preserve saved costs through receipt publication failures

📊 Changes

9 files changed (+1610 additions, -0 deletions)

View changed files

📝 experiments/analysis-efficiency/README.md (+7 -0)
📝 experiments/analysis-efficiency/agent-trial.py (+1 -0)
➕ experiments/analysis-efficiency/retained-investigation-results.json (+635 -0)
➕ experiments/analysis-efficiency/retained-investigation-workflows.md (+201 -0)
➕ experiments/analysis-efficiency/retained-investigation.mjs (+310 -0)
➕ experiments/analysis-efficiency/retained-mixed-verification.mjs (+163 -0)
➕ experiments/analysis-efficiency/retained-query-calls.mjs (+46 -0)
➕ experiments/analysis-efficiency/retained-query-finalization.mjs (+81 -0)
➕ experiments/analysis-efficiency/test_retained_queries.py (+166 -0)

📄 Description

Summary

Verify existing retained Evidence queries against authentic SQLite ELF and npm investigations, including a mixed history of 97 records and 77 Unknown revisions. Existing contracts preserve the bounded answers and original records; no new production API/cache/default is justified by this checkpoint. Fixes #3134. The separately observed imported-format validation bottleneck is tracked in #4810.

Change and scope

Add a reproducible public-SDK comparator, permit the existing static retained-query tools in the isolated agent-trial configuration, and record scoped measurements. Complete-delivery capacity refusals, recovery calls and failed agent tasks remain in costs. Linux static analysis/retained data only; no new provider or target-execution capability. Maintainer scope decision: not applicable.

Contract and boundary impact

Owner: experimental verification at the public CLI/MCP boundary; runtime contracts, Evidence lifetime, provider behavior and the complete default remain unchanged.
Evidence: exact original records/links/omissions/nulls/scalar values and Unknown revisions survive export/import. Close clears the registry; stale revisions and tampered imports fail atomically.
Effects: read selected local data, launch the REA stdio server, and write owned receipts. Existing optional model trials contact the configured model service. No application runtime execution was requested.
Package/generated metadata: none. Full SDK catalog remains 207 tools; model trials explicitly enable four existing tools.

Evidence and validation

Real SDK: two fresh-server repetitions per source/arm, three traversals each; exact selected facts and complete original-record round trips pass. Both mixed-source queries and 32 Unknown pages pass, including revision two, stale update rejection and atomic tamper rejection.
Real CLI: bundle summary and portable inline view pass for both authentic sources. Independent ELF byte parsing checks every symbol; shipped npm source digests corroborate the source-module identity.
Twelve actual Codex tasks: ten complete factual answers, two failed tasks retained. Native total tokens/time overlap; guided JavaScript has only one completed complete-arm sample. No general model-token/time benefit claimed. Individual total/cached/uncached input, output, wall/CPU/family memory are in the linked checkpoint.
npm run check:fast, npm run docs:check — pass.
npm run test:focused -- tests/boundary/mcp/evidenceBundleInline.test.ts tests/boundary/mcp/serverUnknownsMcp.test.ts tests/boundary/mcp/analysisViewMcp.test.ts tests/boundary/cli/analysisViewCli.test.ts — 36 tests pass.
python3 -m unittest discover -s experiments/analysis-efficiency -p test_codex_context.py — 34 tests pass; Python compile, Node syntax, explicit Oxlint and formatting checks pass. Ten deterministic RPC/finalization/publication regressions pass. The committed --verify-mixed mode passes both authentic source scenarios; the final refactored native verification also passes.
Heavy verification: serial, one CPU, 8 GiB hard/7 GiB early stop, zero observed payload OOM events and verified cleanup. Terminal whole-scope counters and remote compute remain unknown.

See the checkpoint for pins, results, failures and limits. Other hosts/providers, thousands of registry heads and general complex-software superiority remain unverified.

Author readiness

Human owner: @N0zoM1z0

The owner explicitly delegated preparation and initial-readiness entry in this session. The following diff/evidence review was performed by the agent under that delegation; independent human review of this final diff is not asserted.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Current verification

Synced main through d5fc7e6eb (including #4930 CI repairs). Final PR head: 5f5df9c93. Pre-push typecheck/lint pass. The focused plist/keyed-archive and four CLI/MCP files pass: 89 tests. All 10 receipt regressions pass (4 RPC + 6 cleanup/output cases); original cleanup and partial-publication failures were reproduced against their prior committed owners. Formatting/syntax/explicit lint pass.

Both authentic mixed-history workflows pass at the final head: identical original records and answer digests, 64 follow-up heads/revision two, stale/tampered refusal, cleared Unknowns and invalid old references after close. Final guarded push/regressions/two-workflow rerun: 109.4 s elapsed / 100.2 s family CPU, 1.57 GiB kernel family peak, one CPU, 8 GiB cap, no payload OOM, cleanup verified. These are validation costs, not a speedup comparison; terminal whole-scope counters remain unknown.

Before merge

One focused outcome; issue, report and repeatable verification linked.
Final diff checked; no production runtime changes or raw captures.
Actionable review fixes implemented and verified; original task costs survive cleanup/output failures.
Final-head CI completed: all 39 advertised checks pass, including all five test shards, Portable tests, native platform packages, Windows package verification and the required aggregate. CI run. Latest Greptile review at 5f5df9c93 is 5/5 and all review threads are resolved. The previous main cache/plist/keyed-archive failures are repaired upstream. Codex review was unavailable due to repository quota.

At the release checkpoint, finish this PR before starting any follow-up implementation. #4810, Apktool, JEB and Hopper remain unstarted.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4814 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/17/2026 **Status:** ✅ Merged **Merged:** 10/18/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `perf/3134-retained-investigation-20261017` --- ### 📝 Commits (6) - [`5d4abd6`](https://github.com/morluto/rea/commit/5d4abd6217178181b25d3057dd59659539284aa0) test(perf): measure authentic retained investigation queries - [`a9f8dff`](https://github.com/morluto/rea/commit/a9f8dff11e71229045efaaa72471843e89269c80) test(perf): retain failed RPC costs and repeat mixed checks - [`e3caff3`](https://github.com/morluto/rea/commit/e3caff367a9ed911cbce19593c202f79f624a5f0) Merge latest main to verify retained investigation checks - [`b025a91`](https://github.com/morluto/rea/commit/b025a919b36ec091ca888339df6b24e78e89b79b) Merge main CI repairs for retained investigation verification - [`5d6259e`](https://github.com/morluto/rea/commit/5d6259e658ee54fc9829b9143a8721a2ab426e80) test(perf): preserve retained receipts through cleanup errors - [`5f5df9c`](https://github.com/morluto/rea/commit/5f5df9c93186a51ee199a57e93fcb3bdcba2cbb4) test(perf): preserve saved costs through receipt publication failures ### 📊 Changes **9 files changed** (+1610 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `experiments/analysis-efficiency/README.md` (+7 -0) 📝 `experiments/analysis-efficiency/agent-trial.py` (+1 -0) ➕ `experiments/analysis-efficiency/retained-investigation-results.json` (+635 -0) ➕ `experiments/analysis-efficiency/retained-investigation-workflows.md` (+201 -0) ➕ `experiments/analysis-efficiency/retained-investigation.mjs` (+310 -0) ➕ `experiments/analysis-efficiency/retained-mixed-verification.mjs` (+163 -0) ➕ `experiments/analysis-efficiency/retained-query-calls.mjs` (+46 -0) ➕ `experiments/analysis-efficiency/retained-query-finalization.mjs` (+81 -0) ➕ `experiments/analysis-efficiency/test_retained_queries.py` (+166 -0) </details> ### 📄 Description ## Summary Verify existing retained Evidence queries against authentic SQLite ELF and npm investigations, including a mixed history of 97 records and 77 Unknown revisions. Existing contracts preserve the bounded answers and original records; no new production API/cache/default is justified by this checkpoint. Fixes #3134. The separately observed imported-format validation bottleneck is tracked in #4810. ## Change and scope Add a reproducible public-SDK comparator, permit the existing static retained-query tools in the isolated agent-trial configuration, and record scoped measurements. Complete-delivery capacity refusals, recovery calls and failed agent tasks remain in costs. Linux static analysis/retained data only; no new provider or target-execution capability. Maintainer scope decision: not applicable. ## Contract and boundary impact - Owner: experimental verification at the public CLI/MCP boundary; runtime contracts, Evidence lifetime, provider behavior and the complete default remain unchanged. - Evidence: exact original records/links/omissions/nulls/scalar values and Unknown revisions survive export/import. Close clears the registry; stale revisions and tampered imports fail atomically. - Effects: read selected local data, launch the REA stdio server, and write owned receipts. Existing optional model trials contact the configured model service. No application runtime execution was requested. - Package/generated metadata: none. Full SDK catalog remains 207 tools; model trials explicitly enable four existing tools. ## Evidence and validation - Real SDK: two fresh-server repetitions per source/arm, three traversals each; exact selected facts and complete original-record round trips pass. Both mixed-source queries and 32 Unknown pages pass, including revision two, stale update rejection and atomic tamper rejection. - Real CLI: bundle summary and portable inline view pass for both authentic sources. Independent ELF byte parsing checks every symbol; shipped npm source digests corroborate the source-module identity. - Twelve actual Codex tasks: ten complete factual answers, two failed tasks retained. Native total tokens/time overlap; guided JavaScript has only one completed complete-arm sample. No general model-token/time benefit claimed. Individual total/cached/uncached input, output, wall/CPU/family memory are in the linked checkpoint. - `npm run check:fast`, `npm run docs:check` — pass. - `npm run test:focused -- tests/boundary/mcp/evidenceBundleInline.test.ts tests/boundary/mcp/serverUnknownsMcp.test.ts tests/boundary/mcp/analysisViewMcp.test.ts tests/boundary/cli/analysisViewCli.test.ts` — 36 tests pass. - `python3 -m unittest discover -s experiments/analysis-efficiency -p test_codex_context.py` — 34 tests pass; Python compile, Node syntax, explicit Oxlint and formatting checks pass. Ten deterministic RPC/finalization/publication regressions pass. The committed --verify-mixed mode passes both authentic source scenarios; the final refactored native verification also passes. - Heavy verification: serial, one CPU, 8 GiB hard/7 GiB early stop, zero observed payload OOM events and verified cleanup. Terminal whole-scope counters and remote compute remain unknown. See [the checkpoint](https://github.com/morluto/rea/blob/perf/3134-retained-investigation-20261017/experiments/analysis-efficiency/retained-investigation-workflows.md) for pins, results, failures and limits. Other hosts/providers, thousands of registry heads and general complex-software superiority remain unverified. ## Author readiness Human owner: @N0zoM1z0 The owner explicitly delegated preparation and initial-readiness entry in this session. The following diff/evidence review was performed by the agent under that delegation; independent human review of this final diff is not asserted. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Current verification Synced main through `d5fc7e6eb` (including #4930 CI repairs). Final PR head: `5f5df9c93`. Pre-push typecheck/lint pass. The focused plist/keyed-archive and four CLI/MCP files pass: 89 tests. All 10 receipt regressions pass (4 RPC + 6 cleanup/output cases); original cleanup and partial-publication failures were reproduced against their prior committed owners. Formatting/syntax/explicit lint pass. Both authentic mixed-history workflows pass at the final head: identical original records and answer digests, 64 follow-up heads/revision two, stale/tampered refusal, cleared Unknowns and invalid old references after close. Final guarded push/regressions/two-workflow rerun: 109.4 s elapsed / 100.2 s family CPU, 1.57 GiB kernel family peak, one CPU, 8 GiB cap, no payload OOM, cleanup verified. These are validation costs, not a speedup comparison; terminal whole-scope counters remain unknown. ## Before merge - [x] One focused outcome; issue, report and repeatable verification linked. - [x] Final diff checked; no production runtime changes or raw captures. - [x] Actionable review fixes implemented and verified; original task costs survive cleanup/output failures. - [x] Final-head CI completed: all 39 advertised checks pass, including all five test shards, Portable tests, native platform packages, Windows package verification and the required aggregate. [CI run](https://github.com/morluto/rea/actions/runs/39089333235). Latest Greptile review at `5f5df9c93` is 5/5 and all review threads are resolved. The previous main cache/plist/keyed-archive failures are repaired upstream. Codex review was unavailable due to repository quota. At the release checkpoint, finish this PR before starting any follow-up implementation. #4810, Apktool, JEB and Hopper remain unstarted. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:14:21 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:47:14 +00:00

[PR #6422] feat(ghidra): admit PS2 Emotion Engine executables through ghidra-emotionengine-reloaded #7349

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6422

No description provided.

Rows
Columns