morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #5017] [MERGED] perf(hopper): reuse decoded records for literal string matches #6735

New issue

Closed

opened 2026-11-01 21:42:40 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:42:40 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5017
Author: @gensokyo-dev
Created: 10/18/2026
Status: ✅ Merged
Merged: 10/18/2026
Merged by: @gensokyo-dev

Base: main ← Head: perf/5010-hopper-selected-string-records

📝 Commits (1)

986aa14 perf(hopper): reuse decoded records for literal string matches

📊 Changes

4 files changed (+1813 additions, -20 deletions)

View changed files

📝 bridge/hopper_bridge.py (+10 -20)
📝 experiments/analysis-efficiency/README.md (+4 -0)
➕ experiments/analysis-efficiency/hopper-string-records.json (+1563 -0)
➕ experiments/analysis-efficiency/hopper-string-records.md (+236 -0)

📄 Description

Summary

Fixes #5010. Hopper literal string search discarded decoded records into a value-only view, then sorted/projected the complete record inventory again after every positive match. Keep the owner's records in the existing sorted search inventory and project matches directly; remove the superseded view and intermediate list.

Three paired real Linux SQLite workflows found shorter warm queries (median 14.849 → 12.628 ms), but whole guarded tasks were slower (89.384 → 93.630 s; CPU 88.333 → 92.590 s). No overall speedup, memory reduction or context saving is claimed. Measurements, negative results and limits and the adjacent JSON receipt retain per-run costs, equality proofs and incomplete attempts.

Contract and boundary impact

Semantic owner: bridge/hopper_bridge.py; reuse its decoded records at the search-inventory boundary. Existing document keys/lifetime and annotation invalidation apply.
Scope decision: not applicable; ordinary optimization of existing local static analysis, with existing process/script authority and cleanup.
CLI/MCP contracts and catalog: unchanged; both adapters use the same implementation. Regex retains its supervised worker.
Evidence: selected facts, stable IDs, native display, inferred encoding/extent, decoding unknowns and coverage/limitations preserved. All 42 applicable Evidence payloads agree across three pairs on their semantic/provenance fields; live environment observations are excluded.
Installation/package/generated metadata: no changes. Minimal-runtime Python bootstrap investigation is separately tracked in #5015.

Existing workflow effects exercised during verification:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.

Scripts and annotation checks ran in the owned provider analysis document. Target bytes remained unchanged; targets were not executed.

Validation performed

Real publisher-authenticated Hopper 6.6.0 Linux demo: three paired 17-call public stdio MCP SQLite workflows, plus a separate reordered-call control pair. Facts, metadata, constant-return oracle and owned cleanup passed. Guard: serial, one CPU, 8 GiB hard / 7 GiB early stop, zero swap; observed payload limit/OOM counters zero. One host-memory stop remains incomplete and excluded from completed statistics.
Real source-fixture semantic workflows: 70 calls per revision, including six expected errors; Unicode symbols, literal/regex/case behavior, annotation invalidation, native string fragments and independent source-byte checks passed. Two real CLI queries matched MCP facts and Evidence IDs.
npm run test:focused -- tests/boundary/providers/hopper/hopperStringSearch.test.ts src/hopper/HopperRegexSearch.test.ts tests/boundary/providers/hopper/hopperBridgeFacade.test.ts tests/conformance/providers/hopper/realHopperSearch.test.ts tests/conformance/providers/hopper/realHopperExhaustiveSearch.test.ts — 35 tests / 5 files passed. Existing coverage retained; no implementation-mirroring test added.
npm run check:fast — typecheck/lint passed, both cached.
npm run docs:check — metadata checks passed, cached; five README consistency tests passed.
git diff --check and targeted Prettier formatting passed; Python AST parsed successfully.

Native Hopper ran in a private Ubuntu Base 24.04 namespace because host Debian glibc was incompatible. This does not qualify actual rea setup, native Debian installation, other OSes, real UTF-16 decoding, full provider parity or complex agent investigations. Some known source literals were not native-recognized: their bytes are verified while decoded-object coverage stays unverified. Context numbers are offline serialization estimates, including the full catalog and validation oracle, rather than model usage.

Author readiness

Human owner: @gensokyo-dev

The owner explicitly authorized ready PRs and delegated readiness entry in this session. The agent reviewed the final diff, relevant source and validation under that delegation; independent human review of this final diff is not asserted.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5017 **Author:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Created:** 10/18/2026 **Status:** ✅ Merged **Merged:** 10/18/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `perf/5010-hopper-selected-string-records` --- ### 📝 Commits (1) - [`986aa14`](https://github.com/morluto/rea/commit/986aa143973159210941832388914dc6e20a4b08) perf(hopper): reuse decoded records for literal string matches ### 📊 Changes **4 files changed** (+1813 additions, -20 deletions) <details> <summary>View changed files</summary> 📝 `bridge/hopper_bridge.py` (+10 -20) 📝 `experiments/analysis-efficiency/README.md` (+4 -0) ➕ `experiments/analysis-efficiency/hopper-string-records.json` (+1563 -0) ➕ `experiments/analysis-efficiency/hopper-string-records.md` (+236 -0) </details> ### 📄 Description ## Summary Fixes #5010. Hopper literal string search discarded decoded records into a value-only view, then sorted/projected the complete record inventory again after every positive match. Keep the owner's records in the existing sorted search inventory and project matches directly; remove the superseded view and intermediate list. Three paired real Linux SQLite workflows found shorter warm queries (median 14.849 → 12.628 ms), but whole guarded tasks were slower (89.384 → 93.630 s; CPU 88.333 → 92.590 s). No overall speedup, memory reduction or context saving is claimed. [Measurements, negative results and limits](https://github.com/morluto/rea/blob/986aa143973159210941832388914dc6e20a4b08/experiments/analysis-efficiency/hopper-string-records.md) and the adjacent JSON receipt retain per-run costs, equality proofs and incomplete attempts. ## Contract and boundary impact - Semantic owner: `bridge/hopper_bridge.py`; reuse its decoded records at the search-inventory boundary. Existing document keys/lifetime and annotation invalidation apply. - Scope decision: not applicable; ordinary optimization of existing local static analysis, with existing process/script authority and cleanup. - CLI/MCP contracts and catalog: unchanged; both adapters use the same implementation. Regex retains its supervised worker. - Evidence: selected facts, stable IDs, native display, inferred encoding/extent, decoding unknowns and coverage/limitations preserved. All 42 applicable Evidence payloads agree across three pairs on their semantic/provenance fields; live environment observations are excluded. - Installation/package/generated metadata: no changes. Minimal-runtime Python bootstrap investigation is separately tracked in #5015. Existing workflow effects exercised during verification: - [x] Read existing files or captured data. - [x] Launch or attach to a process. - [x] Inject instrumentation or run scripts/code. - [x] Change process memory, registers, files, or configuration. Scripts and annotation checks ran in the owned provider analysis document. Target bytes remained unchanged; targets were not executed. ## Validation performed - Real publisher-authenticated Hopper 6.6.0 Linux demo: three paired 17-call public stdio MCP SQLite workflows, plus a separate reordered-call control pair. Facts, metadata, constant-return oracle and owned cleanup passed. Guard: serial, one CPU, 8 GiB hard / 7 GiB early stop, zero swap; observed payload limit/OOM counters zero. One host-memory stop remains incomplete and excluded from completed statistics. - Real source-fixture semantic workflows: 70 calls per revision, including six expected errors; Unicode symbols, literal/regex/case behavior, annotation invalidation, native string fragments and independent source-byte checks passed. Two real CLI queries matched MCP facts and Evidence IDs. - `npm run test:focused -- tests/boundary/providers/hopper/hopperStringSearch.test.ts src/hopper/HopperRegexSearch.test.ts tests/boundary/providers/hopper/hopperBridgeFacade.test.ts tests/conformance/providers/hopper/realHopperSearch.test.ts tests/conformance/providers/hopper/realHopperExhaustiveSearch.test.ts` — 35 tests / 5 files passed. Existing coverage retained; no implementation-mirroring test added. - `npm run check:fast` — typecheck/lint passed, both cached. - `npm run docs:check` — metadata checks passed, cached; five README consistency tests passed. - `git diff --check` and targeted Prettier formatting passed; Python AST parsed successfully. Native Hopper ran in a private Ubuntu Base 24.04 namespace because host Debian glibc was incompatible. This does not qualify actual `rea setup`, native Debian installation, other OSes, real UTF-16 decoding, full provider parity or complex agent investigations. Some known source literals were not native-recognized: their bytes are verified while decoded-object coverage stays unverified. Context numbers are offline serialization estimates, including the full catalog and validation oracle, rather than model usage. ## Author readiness Human owner: @gensokyo-dev The owner explicitly authorized ready PRs and delegated readiness entry in this session. The agent reviewed the final diff, relevant source and validation under that delegation; independent human review of this final diff is not asserted. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:42:40 +00:00

closed this issue
added the
pull-request
label

bsakaguchi changed title from ~~[PR #5017] [MERGED] perf(hopper): reuse decoded records for literal string matches~~ to [PR #5017] perf(hopper): reuse decoded records for literal string matches

2026-11-03 19:45:53 +00:00

bsakaguchi changed title from ~~[PR #5017] perf(hopper): reuse decoded records for literal string matches~~ to [PR #5017] [MERGED] perf(hopper): reuse decoded records for literal string matches

2026-11-03 20:48:00 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6735

No description provided.

Rows
Columns