[PR #1345] [MERGED] feat(native): inspect oversized retained function dossiers via bounded views #2812

New issue

Closed

opened 2026-10-17 21:10:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:10:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1345
Author: @jhuang-tw
Created: 10/9/2026
Status: ✅ Merged
Merged: 10/10/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/native-dossier-selected-views

📝 Commits (5)

adb96bd feat(native): inspect oversized retained function dossiers via bounded views
d0e72d4 chore: integrate frozen main for native views review
5fc0874 fix(native): preserve selected view provenance and caller bounds
3a2866b fix(native): preserve recorded empty artifact paths
d21ba97 chore: integrate latest main for native view verification

📊 Changes

13 files changed (+769 additions, -22 deletions)

View changed files

📝 docs/mcp-contracts.md (+6 -1)
📝 skill-src/reverse-engineer-anything/references/native-and-artifacts.md (+8 -0)
📝 src/application/analysisView/AnalysisViewService.test.ts (+92 -0)
📝 src/application/analysisView/AnalysisViewService.ts (+17 -1)
📝 src/cli/analysisViewCommands.ts (+1 -1)
📝 src/contracts/analysisViewToolContracts.ts (+17 -2)
📝 src/domain/analysisView/analysisView.test.ts (+184 -13)
📝 src/domain/analysisView/analysisView.ts (+60 -3)
📝 src/domain/analysisView/binaryLayoutView.ts (+4 -0)
📝 src/domain/analysisView/javascriptApplicationView.ts (+4 -0)
➕ src/domain/analysisView/nativeFunctionView.ts (+202 -0)
📝 tests/boundary/cli/analysisViewCli.test.ts (+44 -0)
📝 tests/boundary/mcp/analysisViewMcp.test.ts (+130 -1)

📄 Description

Refs #1247 and #1065.

A complete native analyze_function dossier can exceed default MCP framing. Extend the existing inspect_analysis_view workflow so callers can read selected native facts from authenticated same-session Evidence or portable inline Evidence without restarting a provider.

Native facets cover procedure, pseudocode, assembly, callers/callees, typed references and high-pcode summaries/operations/def-use/effects/parameters. Offset defaults to 0 and positive caller-selected limit defaults to 64. Pseudocode uses UTF-16 code units, rejects offsets inside surrogate pairs and returns safe continuation offsets. Other pages count rows. Actual serialized bytes govern transport admission; no arbitrary row ceiling substitutes for that check.

Views keep artifact path/hash, exact parent Evidence ID, distinct view digest, procedure address as recorded, provider limitations and explicit unknowns. Unavailable API residual unknowns and decompiler caveats survive selection. Empty recorded addresses and paths stay empty and do not create invalid address/path locations. Malformed native dossiers report the specific field. Complete default analysis and retained Evidence remain available, including an oversized selected row; CLI help, contracts and packaged skill source describe the added views.

Example (replace the placeholder with an actual same-session Evidence ID):

{"source":{"kind":"retained-evidence","evidence_id":"ev_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"},"view":{"kind":"native","facet":"value_flow_operations","offset":0,"limit":32}}

Validation on latest integrated main 4d8ef3f, pinned Node 24.18.0/npm 11.16.0, Linux x64, two CPUs and a 2 GiB process-family RSS guard:

34 focused tests across domain, application, compiled public CLI and SDK MCP boundaries.
11 MiB source-owned retained native fixture produces a <12 KiB selected result; published SDK-converted input/output schemas validate with Ajv2020, including caller limit 500, and subsequent ping succeeds.
One native assembly row expands beyond framing: typed transport recovery retains the exact complete selected row, then a smaller view and ping work.
Regressions first reproduced four initial defects: caller page rejection, missing API unknowns/caveats, empty address normalization and generic invalid-dossier diagnostics; a follow-up also reproduced rejection of a recorded empty path.
Full npm run typecheck -- --singleThreaded, lint/module boundaries, docs generation check, changed-file formatting and Git whitespace checks pass. Runtime compilation uses the same documented single-thread mode. The pinned native TypeScript compiler's documented single-thread mode preserves the full checking scope within the local memory budget.

This covers the selected-facet portion of #1247. Retained-dossier discovery and cross-session snapshot-backed reuse remain open. Our follow-up also ran a real Ghidra 12.1.4 workflow on a 16,784-byte source-owned ELF with JDK 21, a verified 512 MiB JVM heap and two CPUs. Latest main CLI native-view rejection was reproduced; the candidate CLI returns exact pseudocode from that real parent. Candidate stdio MCP verified six native facets with caller limit 500, parent/subject identity, API limitations, text/structured parity, ping and close. The complete native Evidence ID exactly matches main CLI; input bytes remained unchanged. MCP native responses were 15,580–27,676 bytes versus 84,046 bytes for the complete dossier, and selected calls took 4–70 ms after a roughly 91-second cold analysis; these are one small-target observation and byte measurements, not token or provider startup improvements. Family RSS peaked at 1,271,345,152 bytes under the native 3 GiB guard. Hopper was not run. Existing contributor larger-target real-Ghidra observations remain separately attributed in PR comments.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1345 **Author:** [@jhuang-tw](https://github.com/jhuang-tw) **Created:** 10/9/2026 **Status:** ✅ Merged **Merged:** 10/10/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/native-dossier-selected-views` --- ### 📝 Commits (5) - [`adb96bd`](https://github.com/morluto/rea/commit/adb96bda7e93f931d7074306c664649e18ff10cb) feat(native): inspect oversized retained function dossiers via bounded views - [`d0e72d4`](https://github.com/morluto/rea/commit/d0e72d43231c913c5cdd4eb76b87a1e853dd86d0) chore: integrate frozen main for native views review - [`5fc0874`](https://github.com/morluto/rea/commit/5fc08740b9429f4233b10170feaee6b1b940990e) fix(native): preserve selected view provenance and caller bounds - [`3a2866b`](https://github.com/morluto/rea/commit/3a2866b2dbd64b4cf1c13a4c451c4b1ff5709388) fix(native): preserve recorded empty artifact paths - [`d21ba97`](https://github.com/morluto/rea/commit/d21ba97b2056f6093ad9fb481f6289e4739098e3) chore: integrate latest main for native view verification ### 📊 Changes **13 files changed** (+769 additions, -22 deletions) <details> <summary>View changed files</summary> 📝 `docs/mcp-contracts.md` (+6 -1) 📝 `skill-src/reverse-engineer-anything/references/native-and-artifacts.md` (+8 -0) 📝 `src/application/analysisView/AnalysisViewService.test.ts` (+92 -0) 📝 `src/application/analysisView/AnalysisViewService.ts` (+17 -1) 📝 `src/cli/analysisViewCommands.ts` (+1 -1) 📝 `src/contracts/analysisViewToolContracts.ts` (+17 -2) 📝 `src/domain/analysisView/analysisView.test.ts` (+184 -13) 📝 `src/domain/analysisView/analysisView.ts` (+60 -3) 📝 `src/domain/analysisView/binaryLayoutView.ts` (+4 -0) 📝 `src/domain/analysisView/javascriptApplicationView.ts` (+4 -0) ➕ `src/domain/analysisView/nativeFunctionView.ts` (+202 -0) 📝 `tests/boundary/cli/analysisViewCli.test.ts` (+44 -0) 📝 `tests/boundary/mcp/analysisViewMcp.test.ts` (+130 -1) </details> ### 📄 Description Refs #1247 and #1065. A complete native `analyze_function` dossier can exceed default MCP framing. Extend the existing `inspect_analysis_view` workflow so callers can read selected native facts from authenticated same-session Evidence or portable inline Evidence without restarting a provider. Native facets cover procedure, pseudocode, assembly, callers/callees, typed references and high-pcode summaries/operations/def-use/effects/parameters. Offset defaults to 0 and positive caller-selected limit defaults to 64. Pseudocode uses UTF-16 code units, rejects offsets inside surrogate pairs and returns safe continuation offsets. Other pages count rows. Actual serialized bytes govern transport admission; no arbitrary row ceiling substitutes for that check. Views keep artifact path/hash, exact parent Evidence ID, distinct view digest, procedure address as recorded, provider limitations and explicit unknowns. Unavailable API residual unknowns and decompiler caveats survive selection. Empty recorded addresses and paths stay empty and do not create invalid address/path locations. Malformed native dossiers report the specific field. Complete default analysis and retained Evidence remain available, including an oversized selected row; CLI help, contracts and packaged skill source describe the added views. Example (replace the placeholder with an actual same-session Evidence ID): ```json {"source":{"kind":"retained-evidence","evidence_id":"ev_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"},"view":{"kind":"native","facet":"value_flow_operations","offset":0,"limit":32}} ``` Validation on latest integrated main `4d8ef3f`, pinned Node 24.18.0/npm 11.16.0, Linux x64, two CPUs and a 2 GiB process-family RSS guard: - 34 focused tests across domain, application, compiled public CLI and SDK MCP boundaries. - >11 MiB source-owned retained native fixture produces a <12 KiB selected result; published SDK-converted input/output schemas validate with Ajv2020, including caller limit 500, and subsequent ping succeeds. - One native assembly row expands beyond framing: typed transport recovery retains the exact complete selected row, then a smaller view and ping work. - Regressions first reproduced four initial defects: caller page rejection, missing API unknowns/caveats, empty address normalization and generic invalid-dossier diagnostics; a follow-up also reproduced rejection of a recorded empty path. - Full `npm run typecheck -- --singleThreaded`, lint/module boundaries, docs generation check, changed-file formatting and Git whitespace checks pass. Runtime compilation uses the same documented single-thread mode. The pinned native TypeScript compiler's documented single-thread mode preserves the full checking scope within the local memory budget. This covers the selected-facet portion of #1247. Retained-dossier discovery and cross-session snapshot-backed reuse remain open. Our follow-up also ran a real Ghidra 12.1.4 workflow on a 16,784-byte source-owned ELF with JDK 21, a verified 512 MiB JVM heap and two CPUs. Latest main CLI native-view rejection was reproduced; the candidate CLI returns exact pseudocode from that real parent. Candidate stdio MCP verified six native facets with caller limit 500, parent/subject identity, API limitations, text/structured parity, ping and close. The complete native Evidence ID exactly matches main CLI; input bytes remained unchanged. MCP native responses were 15,580–27,676 bytes versus 84,046 bytes for the complete dossier, and selected calls took 4–70 ms after a roughly 91-second cold analysis; these are one small-target observation and byte measurements, not token or provider startup improvements. Family RSS peaked at 1,271,345,152 bytes under the native 3 GiB guard. Hopper was not run. Existing contributor larger-target real-Ghidra observations remain separately attributed in PR comments. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:10:34 +00:00