[GH-ISSUE #1247] feat(native): inspect selected facets of retained function dossiers #444

Closed
opened 2026-10-15 18:25:06 +00:00 by bsakaguchi · 4 comments
Owner

Originally created by @N0zoM1z0 on GitHub (Oct 9, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1247

Current limitation

Native function dossiers still require transferring the complete inline result, including expansive value flow and repeated Evidence projections. The newer selected-view workflow supports JavaScript applications and ELF layouts, but rejects native analyze_function parents.

Real current result

On the audited PE, analyze_function at 0x410770 completes analysis but returns a typed resource_constraint at the MCP response boundary: the measured response lower bound is 10,568,589 bytes, above the default 10 MiB client framing budget. The approximately 2.6 KiB error includes the retained Evidence reference and export recovery. A following SDK ping succeeds, and close/save retains the complete record.

This means the old generic connection-closed report is fixed. The remaining problem is convenient inspection of already retained native facts through ordinary MCP settings. Exporting complete Evidence or increasing a downstream buffer is an available workaround.

Required behavior

Extend the existing shared selected-view workflow for useful native objects/facets: canonical procedure identity/body ranges, pseudocode, instruction spans, typed callers/callees and value flow on demand. Provide lightweight retained-record discovery by target/profile, canonical procedure, Evidence ID and completeness so an analyst can find an existing dossier before starting Ghidra again.

Preserve complete default analysis and retrievable complete Evidence. Views must retain identity, source locations, limitations and their relation to the parent record; do not reuse a complete-record digest for projected content. Selection should avoid reconstructing/transferring the entire large result when an eligible retained record exists.

Verify default SDK framing, cross-session snapshot-backed reuse, invalid/stale selection and CLI/MCP parity with a source-owned large native fixture. Measure task latency and context usefulness, not only byte count.

src/domain/analysisView/analysisView.ts:268; src/server/toolResult.ts:70. Narrow native follow-up to closed #1050; related #723 and #1065.

Verification and scope

The large-response typed recovery, retained-record save and live connection were verified on published/main with real Ghidra. Native selected-view exclusion was source-confirmed; no new connection-loss defect is claimed.

Audit date: 2026-10-09. Latest npm dist-tag verified as rea-agents@6.1.0; current main tested at f4c99ab8a53806dd57dc24b3d5a600905bcb4686. Linux x64, Node 24.18.0, MCP client SDK 2.3.1. Native observations use unmodified Ghidra 12.1.4 and JDK 21, two CPUs and a 512 MiB Java heap. No target execution or executable mutation was used for the static native checks.

Originally created by @N0zoM1z0 on GitHub (Oct 9, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1247 ### Current limitation Native function dossiers still require transferring the complete inline result, including expansive value flow and repeated Evidence projections. The newer selected-view workflow supports JavaScript applications and ELF layouts, but rejects native `analyze_function` parents. ### Real current result On the audited PE, `analyze_function` at `0x410770` completes analysis but returns a typed `resource_constraint` at the MCP response boundary: the measured response lower bound is 10,568,589 bytes, above the default 10 MiB client framing budget. The approximately 2.6 KiB error includes the retained Evidence reference and export recovery. A following SDK ping succeeds, and close/save retains the complete record. This means the old generic connection-closed report is fixed. The remaining problem is convenient inspection of already retained native facts through ordinary MCP settings. Exporting complete Evidence or increasing a downstream buffer is an available workaround. ### Required behavior Extend the existing shared selected-view workflow for useful native objects/facets: canonical procedure identity/body ranges, pseudocode, instruction spans, typed callers/callees and value flow on demand. Provide lightweight retained-record discovery by target/profile, canonical procedure, Evidence ID and completeness so an analyst can find an existing dossier before starting Ghidra again. Preserve complete default analysis and retrievable complete Evidence. Views must retain identity, source locations, limitations and their relation to the parent record; do not reuse a complete-record digest for projected content. Selection should avoid reconstructing/transferring the entire large result when an eligible retained record exists. Verify default SDK framing, cross-session snapshot-backed reuse, invalid/stale selection and CLI/MCP parity with a source-owned large native fixture. Measure task latency and context usefulness, not only byte count. ### Source and related work [src/domain/analysisView/analysisView.ts:268](https://github.com/morluto/rea/blob/f4c99ab8a53806dd57dc24b3d5a600905bcb4686/src/domain/analysisView/analysisView.ts#L268); [src/server/toolResult.ts:70](https://github.com/morluto/rea/blob/f4c99ab8a53806dd57dc24b3d5a600905bcb4686/src/server/toolResult.ts#L70). Narrow native follow-up to closed #1050; related #723 and #1065. ### Verification and scope The large-response typed recovery, retained-record save and live connection were verified on published/main with real Ghidra. Native selected-view exclusion was source-confirmed; no new connection-loss defect is claimed. Audit date: 2026-10-09. Latest npm dist-tag verified as `rea-agents@6.1.0`; current main tested at `f4c99ab8a53806dd57dc24b3d5a600905bcb4686`. Linux x64, Node 24.18.0, MCP client SDK 2.3.1. Native observations use unmodified Ghidra 12.1.4 and JDK 21, two CPUs and a 512 MiB Java heap. No target execution or executable mutation was used for the static native checks.
Author
Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

Current status: the original selected-view exclusion below is historical. #1345 is merged (a9ee2360) and the existing shared inspect_analysis_view workflow now supports selected facets/pages from retained native function dossiers, preserving parent/subject identity, source evidence and explicit partial/unknown provider metadata. Complete default analysis and retrievable parent Evidence remain available.

Independent real Ghidra 12.1.4 Linux CLI/stdio MCP verification on a 16,784-byte source-owned ELF covers six selected facets, matching identities, source digest, text/structured parity, ping and close. Selected responses measured 15,580–27,676 bytes versus 84,046 bytes for the complete small dossier; warm selected calls took 4–70 ms, while cold provider analysis remained about 91 seconds. The contributor's larger SQLite observations are separately attributed in #1345. None of this establishes model-token savings or broad provider/startup coverage.

Keep this issue open for its remaining acceptance criteria: lightweight retained-dossier discovery, cross-session snapshot-backed reuse and the full large-fixture/default-SDK usability workflow. The merged facet-selection support should not be reimplemented. Adjacent compact native-reference provenance work #1438/#1245 is under separate active review. Refs #1065, #1048.

<!-- gh-comment-id:6095080241 --> @N0zoM1z0 commented on GitHub (Oct 10, 2026): Current status: the original selected-view exclusion below is historical. #1345 is merged (`a9ee2360`) and the existing shared `inspect_analysis_view` workflow now supports selected facets/pages from retained native function dossiers, preserving parent/subject identity, source evidence and explicit partial/unknown provider metadata. Complete default analysis and retrievable parent Evidence remain available. Independent real Ghidra 12.1.4 Linux CLI/stdio MCP verification on a 16,784-byte source-owned ELF covers six selected facets, matching identities, source digest, text/structured parity, ping and close. Selected responses measured 15,580–27,676 bytes versus 84,046 bytes for the complete small dossier; warm selected calls took 4–70 ms, while cold provider analysis remained about 91 seconds. The contributor's larger SQLite observations are separately attributed in #1345. None of this establishes model-token savings or broad provider/startup coverage. Keep this issue open for its remaining acceptance criteria: lightweight retained-dossier discovery, cross-session snapshot-backed reuse and the full large-fixture/default-SDK usability workflow. The merged facet-selection support should not be reimplemented. Adjacent compact native-reference provenance work #1438/#1245 is under separate active review. Refs #1065, #1048.
Author
Owner

@N0zoM1z0 commented on GitHub (Oct 10, 2026):

Independent Linux x64 real-provider follow-up on the current selected-native workflow (Ghidra 12.1.4, Java 21, Node 24.18.0; source-owned tests/conformance/ghidra/inventory.c compiled as a host-native ELF):

  • All 15 collection facets were exercised through the public SDK stdio MCP workflow. Complete page exhaustion used 42 MCP calls; 23 CLI first/last-page checks used the same inline parent Evidence. Row projections and view Evidence IDs matched exactly across CLI/MCP. The run returned 69 observed rows; unresolved_calls was explicitly unavailable (not recorded), not treated as a known empty array.
  • A second real function with one formal parameter separately verified nonempty parameters and parameter_uses through 2 MCP and 2 CLI page checks.
  • Stable analysis-run identity established that retained page selection did not rerun the provider. Session close, stale-reference rejection, following SDK ping, unchanged input SHA-256 and owned ephemeral-database removal passed.
  • The all-facet run stayed within the 2304 MiB family-RSS budget (peak 1,354,092,544 bytes); the parameter run peaked at 1,339,334,656 bytes. MCP page observations were 3.4–21.8 ms in this small fixture. These are single-run measurements, not percentile or oversized-dossier claims.

This establishes real Ghidra coverage for the added collections. #1247 still needs its remaining discovery, cross-session snapshot-backed reuse and source-owned oversized native-response acceptance. No real Hopper or Windows native-page verification was performed here.

<!-- gh-comment-id:6097329985 --> @N0zoM1z0 commented on GitHub (Oct 10, 2026): Independent Linux x64 real-provider follow-up on the current selected-native workflow (Ghidra 12.1.4, Java 21, Node 24.18.0; source-owned `tests/conformance/ghidra/inventory.c` compiled as a host-native ELF): - All 15 collection facets were exercised through the public SDK stdio MCP workflow. Complete page exhaustion used 42 MCP calls; 23 CLI first/last-page checks used the same inline parent Evidence. Row projections and view Evidence IDs matched exactly across CLI/MCP. The run returned 69 observed rows; `unresolved_calls` was explicitly unavailable (`not recorded`), not treated as a known empty array. - A second real function with one formal parameter separately verified nonempty `parameters` and `parameter_uses` through 2 MCP and 2 CLI page checks. - Stable analysis-run identity established that retained page selection did not rerun the provider. Session close, stale-reference rejection, following SDK ping, unchanged input SHA-256 and owned ephemeral-database removal passed. - The all-facet run stayed within the 2304 MiB family-RSS budget (peak 1,354,092,544 bytes); the parameter run peaked at 1,339,334,656 bytes. MCP page observations were 3.4–21.8 ms in this small fixture. These are single-run measurements, not percentile or oversized-dossier claims. This establishes real Ghidra coverage for the added collections. #1247 still needs its remaining discovery, cross-session snapshot-backed reuse and source-owned oversized native-response acceptance. No real Hopper or Windows native-page verification was performed here.
Author
Owner

@N0zoM1z0 commented on GitHub (Oct 11, 2026):

Independent large-native follow-up on main f88c99021b88ce67e513025901f57478d8107d0c (Linux x64, Node 24.18.0, SDK 2.3.1, real Ghidra 12.1.4/Java 21, two CPUs, 512 MiB JVM):

  • The originally audited PE function now returns its complete inline dossier through the default SDK: 7,820,826 response bytes in this run, below 10 MiB. This supersedes the original oversized-response observation for that exact case; it does not establish a universal bound.
  • A source-owned unrolled native C function independently reproduces the oversized boundary. Its normalized dossier is 4,543,224 bytes, retained parent 9,089,744 bytes, and ordinary SDK delivery returns a 1,481-byte typed resource_constraint with an exact retained reference. Following ping and native selected views succeed.
  • First/last pages for all 15 collection facets match the complete exported parent, including nonempty parameters and parameter uses. unresolved_calls remains explicitly not recorded; high-pcode truncation remains explicit. These are first/last checks, not full exhaustion of this large fixture. Twenty-one corresponding CLI page checks return the exact same view Evidence IDs. Text/structured parity is checked in the live MCP workflow.
  • The analyst can retrieve canonical procedure identity, instruction rows, callers, references, pcode operations and parameter-use facts without moving the complete dossier. Selected MCP responses measured 15,372–118,886 bytes and 25.1–75.3 ms in this single run. CLI checks include startup and validating the portable 9 MB parent and took 4.0–5.1 seconds each. No model-token or percentile claim is made.
  • Save/close writes an 18,543,311-byte snapshot. A fresh server rejects an unimported previous-session reference, then imports the snapshot in 8.20 seconds and returns byte-equivalent original retained-view Evidence in 67.7 ms. No Ghidra runtime is allocated. Cached analyze_function still receives typed transport recovery, with a new Evidence ID because the snapshot-origin limitation changes provenance; the original parent remains selectable. Ping, final close, unchanged target digest and empty owned runtime roots pass. Process-family peak: 1,781,719,040 bytes.

Remaining discovery gap is reproduced: get_evidence_bundle itself exceeds ordinary framing for this retained analysis. A focused change is in progress to expose lightweight metadata through that existing tool while preserving complete default delivery. A larger 83 MB saved-snapshot control also hits typed heap-headroom admission; that separate performance boundary is being tracked. Windows cold-start and selected-page coverage are being checked separately and are not established by the Linux result above.

<!-- gh-comment-id:6104758754 --> @N0zoM1z0 commented on GitHub (Oct 11, 2026): Independent large-native follow-up on main `f88c99021b88ce67e513025901f57478d8107d0c` (Linux x64, Node 24.18.0, SDK 2.3.1, real Ghidra 12.1.4/Java 21, two CPUs, 512 MiB JVM): - The originally audited PE function now returns its complete inline dossier through the default SDK: 7,820,826 response bytes in this run, below 10 MiB. This supersedes the original oversized-response observation for that exact case; it does not establish a universal bound. - A source-owned unrolled native C function independently reproduces the oversized boundary. Its normalized dossier is 4,543,224 bytes, retained parent 9,089,744 bytes, and ordinary SDK delivery returns a 1,481-byte typed `resource_constraint` with an exact retained reference. Following ping and native selected views succeed. - First/last pages for all 15 collection facets match the complete exported parent, including nonempty parameters and parameter uses. `unresolved_calls` remains explicitly `not recorded`; high-pcode truncation remains explicit. These are first/last checks, not full exhaustion of this large fixture. Twenty-one corresponding CLI page checks return the exact same view Evidence IDs. Text/structured parity is checked in the live MCP workflow. - The analyst can retrieve canonical procedure identity, instruction rows, callers, references, pcode operations and parameter-use facts without moving the complete dossier. Selected MCP responses measured 15,372–118,886 bytes and 25.1–75.3 ms in this single run. CLI checks include startup and validating the portable 9 MB parent and took 4.0–5.1 seconds each. No model-token or percentile claim is made. - Save/close writes an 18,543,311-byte snapshot. A fresh server rejects an unimported previous-session reference, then imports the snapshot in 8.20 seconds and returns byte-equivalent original retained-view Evidence in 67.7 ms. No Ghidra runtime is allocated. Cached `analyze_function` still receives typed transport recovery, with a new Evidence ID because the snapshot-origin limitation changes provenance; the original parent remains selectable. Ping, final close, unchanged target digest and empty owned runtime roots pass. Process-family peak: 1,781,719,040 bytes. Remaining discovery gap is reproduced: `get_evidence_bundle` itself exceeds ordinary framing for this retained analysis. A focused change is in progress to expose lightweight metadata through that existing tool while preserving complete default delivery. A larger 83 MB saved-snapshot control also hits typed heap-headroom admission; that separate performance boundary is being tracked. Windows cold-start and selected-page coverage are being checked separately and are not established by the Linux result above.
Author
Owner

@N0zoM1z0 commented on GitHub (Oct 11, 2026):

Independent current-main Windows x64 packaged verification at f88c99021b88ce67e513025901f57478d8107d0c adds real native selected-view and snapshot coverage. The fresh local npm installation uses integrity-verified locked production archives, Node 22.19.0, the matching 6.3.0 native-controls manifest/artifact, operator-owned Ghidra 12.1.4 and Java 21. The source-owned PE is 1,024 bytes and is analyzed statically, never executed.

  • The default SDK 60-second cold overview timed out; a separate 240-second cold attempt also timed out. Both cancelled requests cleaned their provider runtime. The connection remained usable after the default attempt.
  • A separate attempt with a 360-second SDK request completed the cold overview in 262.149 seconds. It reported two procedures and two segments. This longer deadline is a measured control, not a general recovery guarantee or a fix for #1048.
  • Real analyze_function returned 42,650 response bytes in 198.9 ms after initialization. Retained procedure and assembly views returned 18,384/17,634 bytes in 22.5/5.8 ms; assembly rows matched the complete parent. Text/structured content matched.
  • Snapshot save/close, fresh MCP process, snapshot import and selected procedure replay passed. Original view Evidence was identical; the replay took 26.7 ms and allocated no Ghidra runtime. Target SHA-256 was unchanged and owned runtime directories were empty after final close.
  • The successful guarded lane peaked at 831,119,360 bytes of observed process-family working set. The actual JVM had -Xmx512M, affinity mask 3, and BelowNormal priority; the supervisor did not stop the run. These are single-run observations on a tiny PE, not large-native or latency-percentile coverage.
  • Progress notifications were operation start/completion only. Combined connection plus full 170-tool catalog preparation took 35.1 seconds initially and 29.7 seconds after reconnect; this combined timing does not isolate SDK compilation from server startup. A separate phase measurement is being taken before attributing that cost.

Windows first-analysis deadlines therefore remain open. This verifies two selected native facets and snapshot reuse on Windows; the source-owned oversized dossier and all 15 collection checks in #1247 remain Linux observations.

<!-- gh-comment-id:6104891634 --> @N0zoM1z0 commented on GitHub (Oct 11, 2026): Independent current-main Windows x64 packaged verification at `f88c99021b88ce67e513025901f57478d8107d0c` adds real native selected-view and snapshot coverage. The fresh local npm installation uses integrity-verified locked production archives, Node 22.19.0, the matching 6.3.0 native-controls manifest/artifact, operator-owned Ghidra 12.1.4 and Java 21. The source-owned PE is 1,024 bytes and is analyzed statically, never executed. - The default SDK 60-second cold overview timed out; a separate 240-second cold attempt also timed out. Both cancelled requests cleaned their provider runtime. The connection remained usable after the default attempt. - A separate attempt with a 360-second SDK request completed the cold overview in 262.149 seconds. It reported two procedures and two segments. This longer deadline is a measured control, not a general recovery guarantee or a fix for #1048. - Real `analyze_function` returned 42,650 response bytes in 198.9 ms after initialization. Retained procedure and assembly views returned 18,384/17,634 bytes in 22.5/5.8 ms; assembly rows matched the complete parent. Text/structured content matched. - Snapshot save/close, fresh MCP process, snapshot import and selected procedure replay passed. Original view Evidence was identical; the replay took 26.7 ms and allocated no Ghidra runtime. Target SHA-256 was unchanged and owned runtime directories were empty after final close. - The successful guarded lane peaked at 831,119,360 bytes of observed process-family working set. The actual JVM had `-Xmx512M`, affinity mask 3, and BelowNormal priority; the supervisor did not stop the run. These are single-run observations on a tiny PE, not large-native or latency-percentile coverage. - Progress notifications were operation start/completion only. Combined connection plus full 170-tool catalog preparation took 35.1 seconds initially and 29.7 seconds after reconnect; this combined timing does not isolate SDK compilation from server startup. A separate phase measurement is being taken before attributing that cost. Windows first-analysis deadlines therefore remain open. This verifies two selected native facets and snapshot reuse on Windows; the source-owned oversized dossier and all 15 collection checks in #1247 remain Linux observations.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#444
No description provided.