morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #8283] [Bug] Native value trace conflates page omission with traversal truncation #6522

New issue

Closed

opened 2026-11-01 21:11:53 +00:00 by bsakaguchi · 8 comments

bsakaguchi commented

2026-11-01 21:11:53 +00:00

Owner

Originally created by @morluto on GitHub (Oct 31, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8283

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

MCP or agent integration

Observed behavior

trace_native_values conflates node-page omission with traversal truncation in truncated. Its last page has truncated: true whether the whole graph was explored or a node/edge budget stopped traversal. In the tested node- and edge-cap cases next_offset was null, unknowns was empty, and the only budget limitation was a generic list of possible limits. The caller cannot tell from the final page whether the assembled pages exhaust the explored graph or a traversal budget stopped collection early.

Reproduction

Using real Ghidra 12.1.4 over stdio MCP on a source-owned arm64 Mach-O, open the target and trace _evaluate with max_depth: 1, max_functions: 4, max_call_sites: 20:

With max_nodes: 100, max_edges: 200, limit: 100, the result has 19 nodes, 17 edges, truncated: false, next_offset: null, and no unknowns.
With the same budgets, limit: 3 and offset: 18, the last page has one node, total_nodes: 19, truncated: true, next_offset: null, and no unknowns. Walking all seven pages recovered all 19 node IDs and 17 edges for this fixture.
With max_nodes: 5, max_edges: 200, limit: 3, offset: 3, the last page has two nodes, total_nodes: 5, truncated: true, next_offset: null, and no unknowns. With max_edges: 1, max_nodes: 100, limit: 3, offset: 12, the last page similarly reports truncated: true, no next page and no unknowns; the trace has 14 nodes and one edge.
Controls with max_depth: 0 and max_functions: 1 reported explicit call-depth and decompilation-budget unknowns respectively. Unpaged node- and edge-capped requests also returned truncated: true without a specific unknown.

These are one fixture and host. The default output is not claimed to be truncated on arbitrary targets. Pagination re-runs analysis; no cross-mutation snapshot consistency is claimed here.

Expected behavior

Expose separately whether the returned page omits retained graph rows and whether the underlying traversal was incomplete, with specific observed truncation reasons where available. An agent should be able to decide whether to follow next_offset, increase a particular budget, or treat the examined graph as exhausted. Keep explicit unknowns, bounded allocation, existing usable rows and per-page provenance. This is a proposed contract improvement; the current single boolean is what was observed.

Impact and scope

Without a distinct traversal-completeness result, a consumer assembling pages cannot reliably decide whether the assembled graph is complete under the requested depth and other limits. Node and edge total_* fields count retained rows, not an independently known total beyond a cap. CLI behavior, other providers, output-byte limits and interruption are untested.

REA version

b43c329df779708e43ae297f498a844e45d9c923 (source build). The relevant NativeValueTrace.ts is unchanged at origin/main 23de10bb2849cdce8ddbe70504aa7cc57de75a98; that later revision was not runtime tested.

Runtime and platform

Node 24.22.1, macOS 26.3 arm64, Ghidra 12.1.4, JDK 21, MCP compact output schema profile.

Target and artifact context

Self-built arm64 Mach-O executable, SHA-256 ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d, unchanged after analysis. The binary was not executed.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

On the normal last page and the two capped last pages, the common structured fields were truncated: true, next_offset: null, unknowns: []. Source: NativeValueTrace.ts sets truncated for traversal caps and also sets the returned boolean when a page is shorter than the collected nodes. The caller-visible total_nodes and total_edges are lengths of the collected arrays.

Additional context

The fixture's evaluate conditionally calls a noinline adjust function that reads a volatile global. Related but separate Evidence-source issue: #8282.

Originally created by @morluto on GitHub (Oct 31, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8283 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area MCP or agent integration ### Observed behavior `trace_native_values` conflates node-page omission with traversal truncation in `truncated`. Its last page has `truncated: true` whether the whole graph was explored or a node/edge budget stopped traversal. In the tested node- and edge-cap cases `next_offset` was `null`, `unknowns` was empty, and the only budget limitation was a generic list of possible limits. The caller cannot tell from the final page whether the assembled pages exhaust the explored graph or a traversal budget stopped collection early. ### Reproduction Using real Ghidra 12.1.4 over stdio MCP on a source-owned arm64 Mach-O, open the target and trace `_evaluate` with `max_depth: 1`, `max_functions: 4`, `max_call_sites: 20`: 1. With `max_nodes: 100`, `max_edges: 200`, `limit: 100`, the result has 19 nodes, 17 edges, `truncated: false`, `next_offset: null`, and no unknowns. 2. With the same budgets, `limit: 3` and `offset: 18`, the last page has one node, `total_nodes: 19`, `truncated: true`, `next_offset: null`, and no unknowns. Walking all seven pages recovered all 19 node IDs and 17 edges for this fixture. 3. With `max_nodes: 5`, `max_edges: 200`, `limit: 3`, `offset: 3`, the last page has two nodes, `total_nodes: 5`, `truncated: true`, `next_offset: null`, and no unknowns. With `max_edges: 1`, `max_nodes: 100`, `limit: 3`, `offset: 12`, the last page similarly reports `truncated: true`, no next page and no unknowns; the trace has 14 nodes and one edge. 4. Controls with `max_depth: 0` and `max_functions: 1` reported explicit call-depth and decompilation-budget unknowns respectively. Unpaged node- and edge-capped requests also returned `truncated: true` without a specific unknown. These are one fixture and host. The default output is not claimed to be truncated on arbitrary targets. Pagination re-runs analysis; no cross-mutation snapshot consistency is claimed here. ### Expected behavior Expose separately whether the returned page omits retained graph rows and whether the underlying traversal was incomplete, with specific observed truncation reasons where available. An agent should be able to decide whether to follow `next_offset`, increase a particular budget, or treat the examined graph as exhausted. Keep explicit unknowns, bounded allocation, existing usable rows and per-page provenance. This is a proposed contract improvement; the current single boolean is what was observed. ### Impact and scope Without a distinct traversal-completeness result, a consumer assembling pages cannot reliably decide whether the assembled graph is complete under the requested depth and other limits. Node and edge `total_*` fields count retained rows, not an independently known total beyond a cap. CLI behavior, other providers, output-byte limits and interruption are untested. ### REA version `b43c329df779708e43ae297f498a844e45d9c923` (source build). The relevant `NativeValueTrace.ts` is unchanged at `origin/main` `23de10bb2849cdce8ddbe70504aa7cc57de75a98`; that later revision was not runtime tested. ### Runtime and platform Node 24.22.1, macOS 26.3 arm64, Ghidra 12.1.4, JDK 21, MCP compact output schema profile. ### Target and artifact context Self-built arm64 Mach-O executable, SHA-256 `ffad1b70c6a5a7ef3f7c3f59126be516520210777d949b35e97c9067f272aa2d`, unchanged after analysis. The binary was not executed. ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence On the normal last page and the two capped last pages, the common structured fields were `truncated: true`, `next_offset: null`, `unknowns: []`. Source: [`NativeValueTrace.ts`](https://github.com/morluto/rea/blob/b43c329df779708e43ae297f498a844e45d9c923/src/application/native/NativeValueTrace.ts) sets `truncated` for traversal caps and also sets the returned boolean when a page is shorter than the collected nodes. The caller-visible `total_nodes` and `total_edges` are lengths of the collected arrays. ### Additional context The fixture's `evaluate` conditionally calls a noinline `adjust` function that reads a volatile global. Related but separate Evidence-source issue: #8282.

bsakaguchi

2026-11-01 21:11:53 +00:00

closed this issue
added the
bug

triage
labels

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the REA tool-design investigation.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Existing test coverage

The real Ghidra verifyNativeValueE2e does test the real CLI and stdio MCP graph and binding semantics, but requests a single limit: 5000 trace; it does not assemble multiple pages or compare page omission against traversal caps.

There are separate simulated-port tests in NativeValueTrace.test.ts for a paged parameter node and a node-cap omission. Both currently expect truncated: true, matching the existing overloaded field. They cover graph/budget behavior, but cannot establish the distinction proposed here. If the result contract changes, update those assertions rather than keeping conflicting semantics. A real-provider regression can walk the small fixture's pages, compare against a larger-limit trace, and separately exercise node and edge caps; do not require a large production binary.

@morluto commented on GitHub (Oct 31, 2026): ### Existing test coverage The real Ghidra [`verifyNativeValueE2e`](https://github.com/morluto/rea/blob/b43c329df779708e43ae297f498a844e45d9c923/scripts/lib/native-value-e2e.mjs) does test the real CLI and stdio MCP graph and binding semantics, but requests a single `limit: 5000` trace; it does not assemble multiple pages or compare page omission against traversal caps. There are separate simulated-port tests in [`NativeValueTrace.test.ts`](https://github.com/morluto/rea/blob/b43c329df779708e43ae297f498a844e45d9c923/src/application/native/NativeValueTrace.test.ts) for a paged parameter node and a node-cap omission. Both currently expect `truncated: true`, matching the existing overloaded field. They cover graph/budget behavior, but cannot establish the distinction proposed here. If the result contract changes, update those assertions rather than keeping conflicting semantics. A real-provider regression can walk the small fixture's pages, compare against a larger-limit trace, and separately exercise node and edge caps; do not require a large production binary.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

TypeScript MCP SDK schema controls

At REA b43c329df779708e43ae297f498a844e45d9c923, the installed TypeScript MCP SDK 2.3.1 connected to four real stdio sessions: Ghidra 12.1.4 and Kuna 1.986, each under the full and compact advertised output-schema profiles. Each session listed 100 tools in the selected session,direct,enhanced,application families. Calls supplied the server-advertised toolDefinition to the SDK.

Across those sessions, 22 successful open/close, full/paged/node-capped value-trace, and Ghidra byte-search responses passed SDK output-schema validation. Ghidra returned 19, 3 and 5 nodes for full/page/capped controls; Kuna returned zero nodes with its explicit missing-def-use unknown. Six malformed or unsupported requests returned normal tool errors. The SDK skips output-schema validation for isError responses; these are not error-schema validation passes.

As a separate control, an intentionally incompatible client-side output schema caused the SDK to reject a successful binary_session response, while the same kind of incompatible schema did not validate a tool-error response. The target SHA-256 remained unchanged and the binary was not executed. This demonstrates schema compatibility for these particular responses, not semantic completeness or all clients/tools; in particular, a schema-valid trace can still have the completeness issue above or the cached-source issue #8282.

@morluto commented on GitHub (Oct 31, 2026): ### TypeScript MCP SDK schema controls At REA `b43c329df779708e43ae297f498a844e45d9c923`, the installed TypeScript MCP SDK 2.3.1 connected to four real stdio sessions: Ghidra 12.1.4 and Kuna 1.986, each under the full and compact advertised output-schema profiles. Each session listed 100 tools in the selected `session,direct,enhanced,application` families. Calls supplied the server-advertised `toolDefinition` to the SDK. Across those sessions, 22 successful open/close, full/paged/node-capped value-trace, and Ghidra byte-search responses passed SDK output-schema validation. Ghidra returned 19, 3 and 5 nodes for full/page/capped controls; Kuna returned zero nodes with its explicit missing-def-use unknown. Six malformed or unsupported requests returned normal tool errors. The SDK skips output-schema validation for `isError` responses; these are not error-schema validation passes. As a separate control, an intentionally incompatible client-side output schema caused the SDK to reject a successful `binary_session` response, while the same kind of incompatible schema did not validate a tool-error response. The target SHA-256 remained unchanged and the binary was not executed. This demonstrates schema compatibility for these particular responses, not semantic completeness or all clients/tools; in particular, a schema-valid trace can still have the completeness issue above or the cached-source issue #8282.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Paging/session-change control on the same unexecuted Mach-O, real REA b43c329 / Ghidra 12.1.4 MCP: trace _evaluate at limit:3, offset:0, then obtain offset 3 and an unpaged graph. Rename its callee _adjust to audit_mutated_callee in the active ephemeral Ghidra database, then repeat offset 3 and the unpaged graph. All calls succeeded and the target hash remained unchanged.

The pre/post full traces each had 19 nodes and 17 edges. Their node IDs and graph payloads matched when per-item Evidence IDs were excluded; the source IDs changed after the metadata edit. Thus this particular rename did not demonstrate mixed graph contents or missing rows. Results already state that pagination reruns analysis; they do not promise a pinned graph across session edits. Also, #8282 independently causes source IDs to differ between cold and warm requests, so an ID difference alone cannot reliably establish that the underlying graph changed.

A future paging-completeness design should state which analysis state pages describe and avoid implying that arbitrary pages from different mutable sessions are one snapshot. A stronger regression would need a permitted operation/fixture that actually changes graph facts and an explicit consistency contract; this control does not establish such behavior.

@morluto commented on GitHub (Oct 31, 2026): Paging/session-change control on the same unexecuted Mach-O, real REA `b43c329` / Ghidra 12.1.4 MCP: trace `_evaluate` at `limit:3, offset:0`, then obtain offset 3 and an unpaged graph. Rename its callee `_adjust` to `audit_mutated_callee` in the active ephemeral Ghidra database, then repeat offset 3 and the unpaged graph. All calls succeeded and the target hash remained unchanged. The pre/post full traces each had 19 nodes and 17 edges. Their node IDs and graph payloads matched when per-item Evidence IDs were excluded; the source IDs changed after the metadata edit. Thus this particular rename did **not** demonstrate mixed graph contents or missing rows. Results already state that pagination reruns analysis; they do not promise a pinned graph across session edits. Also, #8282 independently causes source IDs to differ between cold and warm requests, so an ID difference alone cannot reliably establish that the underlying graph changed. A future paging-completeness design should state which analysis state pages describe and avoid implying that arbitrary pages from different mutable sessions are one snapshot. A stronger regression would need a permitted operation/fixture that actually changes graph facts and an explicit consistency contract; this control does not establish such behavior.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real CLI adapter control on the same source-owned, unexecuted arm64 Mach-O and Ghidra 12.1.4 (rea trace-native-values, REA b43c329): four independent CLI invocations all exited 0.

Request	Returned nodes	Retained nodes / edges	`truncated`	`next_offset`	`unknowns`
Unpaged, limits 100 nodes / 200 edges	19	19 / 17	false	null	empty
Same traversal, last page offset 18 / limit 3	1	19 / 17	true	null	empty
5-node cap, last page offset 3 / limit 3	2	5 / 4	true	null	empty
1-edge cap, last page offset 12 / limit 3	2	14 / 1	true	null	empty

The summarized values match the earlier MCP controls for these same requests. Each CLI invocation started independently; this comparison does not establish page consistency across mutable sessions. The target hash stayed unchanged and the binary was not executed. The CLI currently lacks the trace snapshot/closure options discussed separately in #8285.

@morluto commented on GitHub (Oct 31, 2026): Real CLI adapter control on the same source-owned, unexecuted arm64 Mach-O and Ghidra 12.1.4 (`rea trace-native-values`, REA `b43c329`): four independent CLI invocations all exited 0. | Request | Returned nodes | Retained nodes / edges | `truncated` | `next_offset` | `unknowns` | | --- | ---: | ---: | --- | --- | --- | | Unpaged, limits 100 nodes / 200 edges | 19 | 19 / 17 | false | null | empty | | Same traversal, last page offset 18 / limit 3 | 1 | 19 / 17 | true | null | empty | | 5-node cap, last page offset 3 / limit 3 | 2 | 5 / 4 | true | null | empty | | 1-edge cap, last page offset 12 / limit 3 | 2 | 14 / 1 | true | null | empty | The summarized values match the earlier MCP controls for these same requests. Each CLI invocation started independently; this comparison does not establish page consistency across mutable sessions. The target hash stayed unchanged and the binary was not executed. The CLI currently lacks the trace snapshot/closure options discussed separately in #8285.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Small-graph diagnostic using one page

A bounded recovery control on real REA b43c329 / Ghidra 12.1.4 MCP used the same unexecuted fixture and session. For each last page, I preserved the original traversal budgets and requested a second trace at offset:0, limit:<reported total_nodes>:

Case	Last page: retained nodes / edges, truncated	One-page request: returned nodes, truncated
Uncapped	19 / 17, true	19, false
5-node cap	5 / 4, true	5, true
1-edge cap	14 / 1, true	14, true

All three last pages had next_offset:null and no per-item unknowns. This is a useful diagnostic for these small, stable requests: when every retained node fits in one response, the one-page result removes page omission from the same Boolean and reveals whether the implementation observed a traversal cap. It is not proof that static analysis discovered every possible dependency; provider limitations and unknowns still matter. Totals count retained nodes, which may already have been capped.

Limits: it reruns analysis, so an edited or otherwise changing session is not pinned; it transfers the whole retained graph; output-size limits may still prevent a response. The input schema caps limit at 5,000: a real request with limit:5001 was rejected with an explicit maximum, so this technique cannot cover larger retained graphs in one call. It does not solve the general API ambiguity or the separate missing source-Evidence IDs in #8282. The target hash remained unchanged and no target ran.

@morluto commented on GitHub (Oct 31, 2026): ### Small-graph diagnostic using one page A bounded recovery control on real REA `b43c329` / Ghidra 12.1.4 MCP used the same unexecuted fixture and session. For each last page, I preserved the original traversal budgets and requested a second trace at `offset:0`, `limit:<reported total_nodes>`: | Case | Last page: retained nodes / edges, truncated | One-page request: returned nodes, truncated | | --- | --- | --- | | Uncapped | 19 / 17, true | 19, false | | 5-node cap | 5 / 4, true | 5, true | | 1-edge cap | 14 / 1, true | 14, true | All three last pages had `next_offset:null` and no per-item unknowns. This is a useful **diagnostic for these small, stable requests**: when every retained node fits in one response, the one-page result removes page omission from the same Boolean and reveals whether the implementation observed a traversal cap. It is not proof that static analysis discovered every possible dependency; provider limitations and unknowns still matter. Totals count retained nodes, which may already have been capped. Limits: it reruns analysis, so an edited or otherwise changing session is not pinned; it transfers the whole retained graph; output-size limits may still prevent a response. The input schema caps `limit` at 5,000: a real request with `limit:5001` was rejected with an explicit maximum, so this technique cannot cover larger retained graphs in one call. It does not solve the general API ambiguity or the separate missing source-Evidence IDs in #8282. The target hash remained unchanged and no target ran.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real-provider follow-up for merged #8309 on REA 2d5dec26c31662764c9adfe3ff996174901b8d49, macOS arm64, Node 24.22.1 and Ghidra 12.1.4. Through the real stdio MCP client, an uncapped _evaluate trace returned 19 nodes and 17 edges with page_truncated:false, traversal.status:complete. Walking seven three-node pages in the same unchanged session reassembled the same ordered node IDs and the same edge topology. Each page correctly reported page omission and a complete traversal, including the last page. With max_nodes:5, both pages reported traversal truncated for max_nodes and a located node-budget unknown; with max_edges:1, all five pages reported max_edges and an edge-budget unknown. Advertised compact output schemas were validated by the TypeScript MCP SDK for successful calls.

Four independent real CLI processes also returned the expected distinctions for full, final page, node cap, and edge cap. Fixture SHA-256 was unchanged and the target was not executed. This covers the named caps on one fixture/host; it does not establish all truncation reasons or page consistency under edits. The separate warm source-Evidence closure issue #8282 was still observed and is not resolved by the pagination fix.

@morluto commented on GitHub (Oct 31, 2026): Real-provider follow-up for merged #8309 on REA `2d5dec26c31662764c9adfe3ff996174901b8d49`, macOS arm64, Node 24.22.1 and Ghidra 12.1.4. Through the real stdio MCP client, an uncapped `_evaluate` trace returned 19 nodes and 17 edges with `page_truncated:false`, `traversal.status:complete`. Walking seven three-node pages in the same unchanged session reassembled the same ordered node IDs and the same edge topology. Each page correctly reported page omission and a complete traversal, including the last page. With `max_nodes:5`, both pages reported traversal truncated for `max_nodes` and a located node-budget unknown; with `max_edges:1`, all five pages reported `max_edges` and an edge-budget unknown. Advertised compact output schemas were validated by the TypeScript MCP SDK for successful calls. Four independent real CLI processes also returned the expected distinctions for full, final page, node cap, and edge cap. Fixture SHA-256 was unchanged and the target was not executed. This covers the named caps on one fixture/host; it does not establish all truncation reasons or page consistency under edits. The separate warm source-Evidence closure issue #8282 was still observed and is not resolved by the pagination fix.

bsakaguchi commented

2026-11-01 21:11:54 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

A potential reintroduction is under review in PR #8297's new CLI summary view: https://github.com/morluto/rea/pull/8297#issuecomment-6547543524. On real Ghidra, the complete retained trace correctly distinguishes page-only omission from a max_nodes traversal cap, but the new projected summary exposes only the combined truncated boolean, not page_truncated or traversal. The full bundle retains the fields and the node-cap unknown remains visible; this is a summary-projection gap, not a regression in current main's complete trace. The PR's synthetic fixture also needs the now-required fields when combined with current main.

@morluto commented on GitHub (Oct 31, 2026): A potential reintroduction is under review in PR #8297's new CLI summary view: https://github.com/morluto/rea/pull/8297#issuecomment-6547543524. On real Ghidra, the complete retained trace correctly distinguishes page-only omission from a `max_nodes` traversal cap, but the new projected summary exposes only the combined `truncated` boolean, not `page_truncated` or `traversal`. The full bundle retains the fields and the node-cap unknown remains visible; this is a summary-projection gap, not a regression in current main's complete trace. The PR's synthetic fixture also needs the now-required fields when combined with current main.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6522

No description provided.

Rows
Columns