morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #7134] [MERGED] feat(native): page retained load-image observations #7663

New issue

Closed

opened 2026-11-03 13:39:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:29 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7134
Author: @gensokyo-dev
Created: 10/25/2026
Status: ✅ Merged
Merged: 10/25/2026
Merged by: @gensokyo-dev

Base: main ← Head: feat/6426-native-response-views

📝 Commits (3)

d4c521c feat(native): page retained load-image observations
559735b docs(native): record load-image workflow measurements
a0a301a fix(native): preserve CLI summary paths and cover format views

📊 Changes

21 files changed (+1695 additions, -96 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/references/native-and-artifacts.md (+8 -0)
📝 docs/ghidra-provider.md (+37 -0)
➕ experiments/analysis-efficiency/native-load-image-views.md (+89 -0)
➕ experiments/analysis-efficiency/native-load-image-views.mjs (+357 -0)
➕ src/application/analysisView/NativeLoadImageViews.test.ts (+320 -0)
📝 src/cli/analysisEvidenceDelivery.ts (+12 -2)
📝 src/cli/coreNativeCommands.ts (+28 -8)
📝 src/contracts/analysisViewToolContracts.ts (+1 -1)
📝 src/contracts/officialToolContracts.ts (+2 -2)
📝 src/contracts/toolOutputSchemaGroups.ts (+5 -1)
📝 src/domain/analysisView/analysisView.ts (+31 -3)
📝 src/domain/analysisView/nativeInventoryView.ts (+0 -8)
➕ src/domain/analysisView/nativeLoadImageView.ts (+301 -0)
➕ src/domain/analysisView/nativeViewContext.ts (+10 -0)
📝 src/domain/native/nativeLoadImage.ts (+68 -62)
📝 src/server/registerOfficialTools.ts (+4 -3)
➕ tests/boundary/cli/nativeLoadImageSummaryCli.test.ts (+79 -0)
📝 tests/boundary/mcp/ghidraNativeApiMcp.test.ts (+97 -0)
📝 tests/boundary/mcp/nativeInventoryViewsMcp.test.ts (+8 -6)
➕ tests/boundary/mcp/nativeLoadImageViewsMcp.test.ts (+95 -0)

...and 1 more files

📄 Description

Summary

Refs #6426: this handles the load-image delivery slice; repeated primitive context, graph limitations and lookup diagnostics remain separate work.

Real Ghidra inspect_native_load_image on system OpenSSL returned 9,114,888 bytes of parent Evidence and exceeded the default 10 MiB MCP response boundary. Optional detail: "summary" now retains that complete parent and delivers verification state, identity and counts. inspect_analysis_view kind native-load-image pages source files, mappings, relocations, entries, entry context, checks and DOL sections. CLI supports --detail summary --evidence_output FILE.

Contracts and ownership

Named format schemas remain the semantic owner. CLI/MCP use the same domain projection and existing Evidence publication. Complete remains the default; delivery options never enter semantic parameters or change the parent ID.
Pages preserve (parent ID, collection, source_index), same-address alternatives, null facts, provider/profile linkage and structured source verification_status. Unavailable checks/sections stay null. Page exhaustion never upgrades unsupported verification or runtime coverage.
Relocation bytes and potentially large check expected/observed bodies require explicit fields. Complete facts remain available. Successful validation is reused only for immutable retained parents; exact indices read directly without scanning the inventory, and ordinary pages materialize only selected rows.
Shared native source context has one schema owner. There is no new provider, target execution, mutation or integration authority; a new capability scope decision is not applicable. Generated catalogs and packaged skills remain ignored build outputs.

Validation performed

npm run check:fast passed again on rebased a0a301a5c (54.1 s, 4.03 GiB sampled family RSS); npm run docs:check passed before the rebase; documentation checks included nine README tests.
44 applicable projection/CLI/MCP cases passed across the focused run and its corrected raw/DOL fixture rerun. These cover actual SDK-converted output, stable source identity, opt-in payloads, unknown versus empty collections, duplicate/reversed/out-of-range indices, malformed unselected facts, raw/DOL verification and selected DOL sections. A registered CLI relative-path regression fails before the publication fix and passes afterward; later independent views retain their own request scope.
All 16 schema cases executed. A full-file run hit 30-second timeouts in two expensive cases under the two-CPU limit; both passed in fresh isolated processes (22.3 s / 18.7 s). The comparable main case passed in 23.3 s. These passes predate rebasing onto main 25f0f9934. Current CI has six advertised-input assertion failures caused by main’s 64 KiB schema fitting: two inline-Evidence cases, three analysis-view cases and the item-selector case. All six reproduce unchanged in a clean 25f0f9934 worktree with its own built runtime/catalog. The server still validates canonical calls; this PR does not change schema fitting, weaken assertions or alter CI. After removing only the newly added view branch, the fitted input schema is structurally identical to main’s. Merge bypass for these existing failures is authorized by the requester. One further CI failure in the untouched JavaScript timeout-recovery case passed in fresh isolated processes on both main and this PR, in the complete main file, and in the unchanged CI shard retry. No production/test changes were made for that transient result.
Real Ghidra 12.1.4 / Linux x64 / Node 24.22.1: cold/warm stdio MCP captures passed, selected pages matched complete authenticated parents and the independent readelf coordinate oracle, and cleanup succeeded. All 21,117 file-declared coordinates survived; the extra provider observation at relative 0x47dc60 was preserved without claiming its origin was independently verified.
Real CLI summary/bundle publication passed on the same large library. On rebased a0a301a5c, a fresh real CLI run preserved the literal relative request path in both summary and complete bundle (203.27 s, 348.93 CPU seconds, 1.60 GiB sampled family RSS). Its parent ID and all 21,118 observations match the earlier real captures. A new CLI process without configured Ghidra/JDK authenticated that bundle and paged exact first/last relocation fields (4.90 s, 6.03 CPU seconds, 0.65 GiB sampled family RSS). Missing --evidence_output was rejected before target admission.

Target SHA-256: 76dd3d93e5ee48950a92a58d59b94de8143847f91a80d9682c938767b991577d. Reproducer and full accounting notes: experiments/analysis-efficiency/native-load-image-views.mjs and its adjacent guide. Baseline 42382651b (same tree as main fc43e285e), implementation 7a53a4a7f; one fresh two-capture session each, two CPUs, Ghidra heap 1.5 GiB / Node heap 3 GiB.

Whole harness cost	Complete + export/local recovery	Summary + selected reads
Direct capture delivery	Refused at 10 MiB boundary	9,787 bytes
Raw serialization token estimate	912,927	936,730
Elapsed / CPU	164.93 s / 281.75 s	169.88 s / 293.06 s
Sampled family RSS peak	2.01 GiB	2.30 GiB

These are cl100k_base serialization estimates, not model usage. They include catalog discovery, arguments/replies and baseline local recovery facts; local command text, framing, prompts and client rendering are unmeasured. Both runs export/authenticate complete parents; summary additionally checks tail bytes and unavailable checks through extra RPCs. RSS is a 100 ms sum across the process family, can double-count shared pages and miss short peaks. OS caches were warm. This establishes deliverability and faithful selection; lower task tokens, latency or memory are not claimed. Other providers/platforms were not verified.

Author readiness

Human owner: @gensokyo-dev

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7134 **Author:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Created:** 10/25/2026 **Status:** ✅ Merged **Merged:** 10/25/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `feat/6426-native-response-views` --- ### 📝 Commits (3) - [`d4c521c`](https://github.com/morluto/rea/commit/d4c521c92a06cf5a7abfc0b94ea452610015300b) feat(native): page retained load-image observations - [`559735b`](https://github.com/morluto/rea/commit/559735b5a8a63b631949ceff8d63555959ac4792) docs(native): record load-image workflow measurements - [`a0a301a`](https://github.com/morluto/rea/commit/a0a301a5c076ac16d193a4acd08f25e43f15c191) fix(native): preserve CLI summary paths and cover format views ### 📊 Changes **21 files changed** (+1695 additions, -96 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md` (+8 -0) 📝 `docs/ghidra-provider.md` (+37 -0) ➕ `experiments/analysis-efficiency/native-load-image-views.md` (+89 -0) ➕ `experiments/analysis-efficiency/native-load-image-views.mjs` (+357 -0) ➕ `src/application/analysisView/NativeLoadImageViews.test.ts` (+320 -0) 📝 `src/cli/analysisEvidenceDelivery.ts` (+12 -2) 📝 `src/cli/coreNativeCommands.ts` (+28 -8) 📝 `src/contracts/analysisViewToolContracts.ts` (+1 -1) 📝 `src/contracts/officialToolContracts.ts` (+2 -2) 📝 `src/contracts/toolOutputSchemaGroups.ts` (+5 -1) 📝 `src/domain/analysisView/analysisView.ts` (+31 -3) 📝 `src/domain/analysisView/nativeInventoryView.ts` (+0 -8) ➕ `src/domain/analysisView/nativeLoadImageView.ts` (+301 -0) ➕ `src/domain/analysisView/nativeViewContext.ts` (+10 -0) 📝 `src/domain/native/nativeLoadImage.ts` (+68 -62) 📝 `src/server/registerOfficialTools.ts` (+4 -3) ➕ `tests/boundary/cli/nativeLoadImageSummaryCli.test.ts` (+79 -0) 📝 `tests/boundary/mcp/ghidraNativeApiMcp.test.ts` (+97 -0) 📝 `tests/boundary/mcp/nativeInventoryViewsMcp.test.ts` (+8 -6) ➕ `tests/boundary/mcp/nativeLoadImageViewsMcp.test.ts` (+95 -0) _...and 1 more files_ </details> ### 📄 Description ## Summary Refs #6426: this handles the load-image delivery slice; repeated primitive context, graph limitations and lookup diagnostics remain separate work. Real Ghidra `inspect_native_load_image` on system OpenSSL returned 9,114,888 bytes of parent Evidence and exceeded the default 10 MiB MCP response boundary. Optional `detail: "summary"` now retains that complete parent and delivers verification state, identity and counts. `inspect_analysis_view` kind `native-load-image` pages source files, mappings, relocations, entries, entry context, checks and DOL sections. CLI supports `--detail summary --evidence_output FILE`. ## Contracts and ownership - Named format schemas remain the semantic owner. CLI/MCP use the same domain projection and existing Evidence publication. Complete remains the default; delivery options never enter semantic parameters or change the parent ID. - Pages preserve `(parent ID, collection, source_index)`, same-address alternatives, null facts, provider/profile linkage and structured source `verification_status`. Unavailable checks/sections stay null. Page exhaustion never upgrades unsupported verification or runtime coverage. - Relocation bytes and potentially large check expected/observed bodies require explicit fields. Complete facts remain available. Successful validation is reused only for immutable retained parents; exact indices read directly without scanning the inventory, and ordinary pages materialize only selected rows. - Shared native source context has one schema owner. There is no new provider, target execution, mutation or integration authority; a new capability scope decision is not applicable. Generated catalogs and packaged skills remain ignored build outputs. ## Validation performed - `npm run check:fast` passed again on rebased `a0a301a5c` (54.1 s, 4.03 GiB sampled family RSS); `npm run docs:check` passed before the rebase; documentation checks included nine README tests. - 44 applicable projection/CLI/MCP cases passed across the focused run and its corrected raw/DOL fixture rerun. These cover actual SDK-converted output, stable source identity, opt-in payloads, unknown versus empty collections, duplicate/reversed/out-of-range indices, malformed unselected facts, raw/DOL verification and selected DOL sections. A registered CLI relative-path regression fails before the publication fix and passes afterward; later independent views retain their own request scope. - All 16 schema cases executed. A full-file run hit 30-second timeouts in two expensive cases under the two-CPU limit; both passed in fresh isolated processes (22.3 s / 18.7 s). The comparable main case passed in 23.3 s. These passes predate rebasing onto main `25f0f9934`. Current CI has six advertised-input assertion failures caused by main’s 64 KiB schema fitting: two inline-Evidence cases, three analysis-view cases and the item-selector case. All six reproduce unchanged in a clean `25f0f9934` worktree with its own built runtime/catalog. The server still validates canonical calls; this PR does not change schema fitting, weaken assertions or alter CI. After removing only the newly added view branch, the fitted input schema is structurally identical to main’s. Merge bypass for these existing failures is authorized by the requester. One further CI failure in the untouched JavaScript timeout-recovery case passed in fresh isolated processes on both main and this PR, in the complete main file, and in the unchanged CI shard retry. No production/test changes were made for that transient result. - Real Ghidra 12.1.4 / Linux x64 / Node 24.22.1: cold/warm stdio MCP captures passed, selected pages matched complete authenticated parents and the independent readelf coordinate oracle, and cleanup succeeded. All 21,117 file-declared coordinates survived; the extra provider observation at relative `0x47dc60` was preserved without claiming its origin was independently verified. - Real CLI summary/bundle publication passed on the same large library. On rebased `a0a301a5c`, a fresh real CLI run preserved the literal relative request path in both summary and complete bundle (203.27 s, 348.93 CPU seconds, 1.60 GiB sampled family RSS). Its parent ID and all 21,118 observations match the earlier real captures. A new CLI process without configured Ghidra/JDK authenticated that bundle and paged exact first/last relocation fields (4.90 s, 6.03 CPU seconds, 0.65 GiB sampled family RSS). Missing `--evidence_output` was rejected before target admission. Target SHA-256: `76dd3d93e5ee48950a92a58d59b94de8143847f91a80d9682c938767b991577d`. Reproducer and full accounting notes: `experiments/analysis-efficiency/native-load-image-views.mjs` and its adjacent guide. Baseline `42382651b` (same tree as main `fc43e285e`), implementation `7a53a4a7f`; one fresh two-capture session each, two CPUs, Ghidra heap 1.5 GiB / Node heap 3 GiB. | Whole harness cost | Complete + export/local recovery | Summary + selected reads | | --- | ---: | ---: | | Direct capture delivery | Refused at 10 MiB boundary | 9,787 bytes | | Raw serialization token estimate | 912,927 | 936,730 | | Elapsed / CPU | 164.93 s / 281.75 s | 169.88 s / 293.06 s | | Sampled family RSS peak | 2.01 GiB | 2.30 GiB | These are cl100k_base serialization estimates, not model usage. They include catalog discovery, arguments/replies and baseline local recovery facts; local command text, framing, prompts and client rendering are unmeasured. Both runs export/authenticate complete parents; summary additionally checks tail bytes and unavailable checks through extra RPCs. RSS is a 100 ms sum across the process family, can double-count shared pages and miss short peaks. OS caches were warm. This establishes deliverability and faithful selection; lower task tokens, latency or memory are not claimed. Other providers/platforms were not verified. ## Author readiness Human owner: @gensokyo-dev - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:39:29 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7663

No description provided.

Rows
Columns