[PR #941] [MERGED] perf(evidence): scale imports and dependency graphs #2518

New issue

Closed

opened 2026-10-17 21:07:04 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:07:04 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/941
Author: @morluto
Created: 10/7/2026
Status: ✅ Merged
Merged: 10/7/2026
Merged by: @morluto

Base: main ← Head: codex/application-evidence-audit

📝 Commits (8)

9e512a1 perf(evidence): avoid rebuilding ledger on record imports
859d857 Merge remote-tracking branch 'origin/main' into codex/application-evidence-audit
2738839 Merge remote-tracking branch 'origin/main' into codex/application-evidence-audit
e3713d1 perf(evidence): scale unknown graph validation
af2e557 fix(evidence): preserve import and projection semantics
20b1627 fix(evidence): invalidate cache on metadata replacement
999dd76 test(evidence): isolate snapshot import regression
9dc47f8 fix(evidence): preserve snapshots on additive imports

📊 Changes

8 files changed (+613 additions, -47 deletions)

View changed files

📝 src/application/binary/BinarySessionRecords.ts (+2 -1)
📝 src/application/investigation/EvidenceLedger.residualUnknown.test.ts (+72 -0)
📝 src/application/investigation/EvidenceLedger.test.ts (+115 -2)
📝 src/application/investigation/EvidenceLedger.ts (+42 -1)
📝 src/application/investigation/InvestigationRecords.test.ts (+12 -2)
➕ src/domain/evidenceBundle.test.ts (+122 -0)
📝 src/domain/evidenceBundle.ts (+108 -41)
📝 tests/composition/analysis-sessions/binarySession.test.ts (+140 -0)

📄 Description

EvidenceLedger imports retain the record-only fast path while validating complete incoming bundles before committing staged state. Equivalent semantic records replace stored local path/name metadata; changed is true when the ledger changes and false on exact replay, while metadataChanged specifically identifies replacement of an existing record's representation. Binary session imports invalidate cached analysis snapshots only for changed existing-record metadata. Additive evidence and unknown imports notify snapshot observers while preserving cached analysis entries and allowing snapshot export to include the new records. Unknown graph validation handles deep dependency chains iteratively, and target projection prunes out-of-scope dependency chains without spreading large dependent sets into function arguments.

Validation on Node v24.18.0: 60 focused tests passed across evidence ledger/domain, investigation records, filesystem and MCP bundle boundaries, application MCP, and session composition; the final snapshot-import regressions also passed in the 45-test focused suite. Those session regressions prove additive imports preserve cached query results and export the added evidence, while path metadata replacement invalidates the cache and prevents stale replay. npm run build:cached, npm run typecheck, npm run check:fast -- --force, npm run check:ci -- --force, npm run format:check, and npm run docs:check passed. CI lint reported 153 warnings and 0 errors. Real provider verification was not applicable.

Absolute-dist paired profiles use Node v24.18.0 on macOS; fixture setup was excluded where noted. The independent comparisons and their exact hashes are:

Record-only full bundle import, baseline 37d2ebac vs candidate 9e512a1a: at 1,000 records, median 7,328.70 ms vs 17.97 ms. Input SHA-256 4a13e43fe9eabff2689db8e9af907807e590f8f1709b4541123daf25055ccaab; result SHA-256 e991cb61b093d2c3a71f50db06536a78c974b45f9582a3c2ef76ea199d46c9b1. Complete export equality and duplicate replay equality both passed.
Unknown dependency-chain creation, baseline dda8e6a7 vs graph candidate 71c8faf7: at 5,000 records/unknowns, median 1,219.01 ms vs 368.08 ms; exact output SHA-256 1e7075a574c7cc217b58c3ff7e3e557db7a2d82c343f65cf356937b070c18a2c.
Target projection with an out-of-scope root and dependent chain, same graph comparison: at 5,000 unknowns, median 1,543.55 ms vs 3.30 ms; exact output SHA-256 000bef2a45fbd4b17adc4e2ab0acd65b076ef5c25f0d8479244c8088730de9cd.
Deep valid dependency-chain import, same graph comparison: baseline failed validation at 10,000 records/unknowns; candidate imported all 10,000 and exported exactly the input, SHA-256 4a2fca7f29b6321544fcbf632db50ef99baf05d172d62fdc06018cb7a87101de.
High-fanout projection, baseline e3713d12 vs candidate 20b1627d: a fully parsed graph with 2 evidence records and 125,002 unknowns (foreign root -> target-scoped middle -> 125,000 target-scoped dependents) caused the baseline spread implementation to throw RangeError: Maximum call stack size exceeded. The candidate returned the complete empty target projection. Input SHA-256 088acc4a437006c5432c594c24eea216b4f2366f8144aa7b037945266875620c; output SHA-256 7befdb905fde719f24abe9ca56675b6daf80b6e5d60fc04d6c8af104546e1144.

The paired profile artifacts and full high-fanout reproduction script/result are retained locally under .cache/performance-893/; each comparison uses its own baseline to isolate the relevant change.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/941 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/7/2026 **Status:** ✅ Merged **Merged:** 10/7/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `codex/application-evidence-audit` --- ### 📝 Commits (8) - [`9e512a1`](https://github.com/morluto/rea/commit/9e512a1a4b0c3f9f8adf4dacf9add3bdc4c25be9) perf(evidence): avoid rebuilding ledger on record imports - [`859d857`](https://github.com/morluto/rea/commit/859d8570257e432deb6aff1f329bf82eb95ce43d) Merge remote-tracking branch 'origin/main' into codex/application-evidence-audit - [`2738839`](https://github.com/morluto/rea/commit/27388394f39ed515c4a85a0608d5117f2a4d8bc0) Merge remote-tracking branch 'origin/main' into codex/application-evidence-audit - [`e3713d1`](https://github.com/morluto/rea/commit/e3713d12e0182263558f25f31e1e9ff1b635c789) perf(evidence): scale unknown graph validation - [`af2e557`](https://github.com/morluto/rea/commit/af2e55732fe2f7f115efa0dcbc5f23b678478121) fix(evidence): preserve import and projection semantics - [`20b1627`](https://github.com/morluto/rea/commit/20b1627dc6fed1cf2f4145949923f10f9ba5f5b9) fix(evidence): invalidate cache on metadata replacement - [`999dd76`](https://github.com/morluto/rea/commit/999dd7646d91f50c18af605d9786c5b6b815ac02) test(evidence): isolate snapshot import regression - [`9dc47f8`](https://github.com/morluto/rea/commit/9dc47f86513a96710e941e7d8c18a921e8209111) fix(evidence): preserve snapshots on additive imports ### 📊 Changes **8 files changed** (+613 additions, -47 deletions) <details> <summary>View changed files</summary> 📝 `src/application/binary/BinarySessionRecords.ts` (+2 -1) 📝 `src/application/investigation/EvidenceLedger.residualUnknown.test.ts` (+72 -0) 📝 `src/application/investigation/EvidenceLedger.test.ts` (+115 -2) 📝 `src/application/investigation/EvidenceLedger.ts` (+42 -1) 📝 `src/application/investigation/InvestigationRecords.test.ts` (+12 -2) ➕ `src/domain/evidenceBundle.test.ts` (+122 -0) 📝 `src/domain/evidenceBundle.ts` (+108 -41) 📝 `tests/composition/analysis-sessions/binarySession.test.ts` (+140 -0) </details> ### 📄 Description EvidenceLedger imports retain the record-only fast path while validating complete incoming bundles before committing staged state. Equivalent semantic records replace stored local path/name metadata; `changed` is true when the ledger changes and false on exact replay, while `metadataChanged` specifically identifies replacement of an existing record's representation. Binary session imports invalidate cached analysis snapshots only for changed existing-record metadata. Additive evidence and unknown imports notify snapshot observers while preserving cached analysis entries and allowing snapshot export to include the new records. Unknown graph validation handles deep dependency chains iteratively, and target projection prunes out-of-scope dependency chains without spreading large dependent sets into function arguments. Validation on Node v24.18.0: 60 focused tests passed across evidence ledger/domain, investigation records, filesystem and MCP bundle boundaries, application MCP, and session composition; the final snapshot-import regressions also passed in the 45-test focused suite. Those session regressions prove additive imports preserve cached query results and export the added evidence, while path metadata replacement invalidates the cache and prevents stale replay. `npm run build:cached`, `npm run typecheck`, `npm run check:fast -- --force`, `npm run check:ci -- --force`, `npm run format:check`, and `npm run docs:check` passed. CI lint reported 153 warnings and 0 errors. Real provider verification was not applicable. Absolute-dist paired profiles use Node v24.18.0 on macOS; fixture setup was excluded where noted. The independent comparisons and their exact hashes are: - Record-only full bundle import, baseline `37d2ebac` vs candidate `9e512a1a`: at 1,000 records, median 7,328.70 ms vs 17.97 ms. Input SHA-256 `4a13e43fe9eabff2689db8e9af907807e590f8f1709b4541123daf25055ccaab`; result SHA-256 `e991cb61b093d2c3a71f50db06536a78c974b45f9582a3c2ef76ea199d46c9b1`. Complete export equality and duplicate replay equality both passed. - Unknown dependency-chain creation, baseline `dda8e6a7` vs graph candidate `71c8faf7`: at 5,000 records/unknowns, median 1,219.01 ms vs 368.08 ms; exact output SHA-256 `1e7075a574c7cc217b58c3ff7e3e557db7a2d82c343f65cf356937b070c18a2c`. - Target projection with an out-of-scope root and dependent chain, same graph comparison: at 5,000 unknowns, median 1,543.55 ms vs 3.30 ms; exact output SHA-256 `000bef2a45fbd4b17adc4e2ab0acd65b076ef5c25f0d8479244c8088730de9cd`. - Deep valid dependency-chain import, same graph comparison: baseline failed validation at 10,000 records/unknowns; candidate imported all 10,000 and exported exactly the input, SHA-256 `4a2fca7f29b6321544fcbf632db50ef99baf05d172d62fdc06018cb7a87101de`. - High-fanout projection, baseline `e3713d12` vs candidate `20b1627d`: a fully parsed graph with 2 evidence records and 125,002 unknowns (foreign root -> target-scoped middle -> 125,000 target-scoped dependents) caused the baseline spread implementation to throw `RangeError: Maximum call stack size exceeded`. The candidate returned the complete empty target projection. Input SHA-256 `088acc4a437006c5432c594c24eea216b4f2366f8144aa7b037945266875620c`; output SHA-256 `7befdb905fde719f24abe9ca56675b6daf80b6e5d60fc04d6c8af104546e1144`. The paired profile artifacts and full high-fanout reproduction script/result are retained locally under `.cache/performance-893/`; each comparison uses its own baseline to isolate the relevant change. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:07:04 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 01:07:09 +00:00

[PR #2936] [MERGED] fix(package-verifier): reject overview responses without a procedure count #5502

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2518

No description provided.

Rows
Columns