morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8373] [Design] Bound delivery of large native feature traces and their unknowns #6572

New issue

Open

opened 2026-11-01 21:12:20 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-11-01 21:12:20 +00:00

Owner

Originally created by @morluto on GitHub (Oct 31, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8373

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

How should an agent consume a large native trace_feature result with many unavailable owners in bounded pages or a useful summary, while retaining all observed references and the ability to manage residual unknowns without creating disproportionate session output?

Context and evidence

On the unmerged #8368 branch (d25fd8408), I generated owned arm64 Mach-O fixtures whose _main contains repeated static direct calls to _goal. The targets were not executed. Real verified Kuna 1.986 through stdio MCP reported every site, and the PR preserved an unavailable owner for each. The examples intentionally stress volume; they do not represent measured real-world frequency or model-token usage.

3,000 sites: trace_feature("goal") succeeded with 3,000 references and residual unknowns. The SDK-serialized reply was 1,981,493 bytes. get_evidence_bundle({detail:"summary"}) succeeded but its SDK reply was 4,171,959 bytes; it summarized 3,001 records: the trace plus 3,000 record_unknown Evidence records.
8,000 sites: the trace succeeded with all sites; its SDK reply was 5,271,513 bytes. The unfiltered Evidence summary exceeded the default 10 MiB MCP response allowance and returned resource_constraint. A summary filtered by operation:"trace_feature" succeeded, list_unknowns({offset:0,limit:5}) returned a page of 8,000 heads, and export_evidence_bundle produced a 25,091,963-byte file with 8,001 records and 8,000 unknowns.
16,000 sites: direct Kuna xrefs returned all 16,000 sites in 8,576,265 bytes. The MCP trace returned resource_constraint at the response boundary with an exact retained Evidence ID and export guidance; the exported 50,211,965-byte bundle contained that same trace ID, all 16,000 trace references, 16,000 unknowns and 16,001 records. A filtered summary and a five-item list_unknowns page succeeded; the unfiltered summary again exceeded the budget. The suggested CLI JSON alternative also succeeded on a new run, returning all 16,000 references in 5,043,664 bytes and the same Evidence ID in this controlled fixture. A fresh CLI run should not generally be treated as proof of an earlier live observation.

SDK response sizes include both text and structured representations and are bytes, not token counts. The successful calls used full advertised output schemas; targets' hashes were unchanged. This is one host/provider/fixture shape, not a benchmark or a claim that every missing owner needs a separate durable question.

Expected interaction with targets

Read an explicitly opened local binary, run the existing static provider and write a caller-selected Evidence bundle when requested. No target execution or mutation is proposed.

Boundary details

Keep each source and target identity, unavailable reason, provider/profile, reference-cap coverage, existing explicit unknowns and complete Evidence accessible. An omitted page must not imply an absent reference. Preserve any user-managed independent Unknown revisions and distinguish page coverage from provider discovery. Avoid imposing an arbitrary cap or silently discarding sources. Any grouping of unknowns needs an explicit lifecycle and identity policy.

Options and tradeoffs

Compare a retained summary plus inspect_analysis_view pages for native feature matches/references, more general exact-record retrieval (#8307), and grouping a common unsupported-capability unknown while retaining per-reference outcomes. list_unknowns already pages heads and the bundle export is a working fallback; a bounded trace view would avoid requiring agents to transfer or inspect the entire file for a selected set of references. Decide whether get_evidence_bundle summary needs pagination or better filtered-summary recovery independently. Measure representative workloads and total agent interactions before claiming efficiency gains.

Verification available so far

I have a prototype or observed results

Success criteria and open questions

Exercise empty, single, shared-source, capped, many-source and mixed-owner cases; assert stable page/record identity, offline export/import, uncertainty and cancellation/cleanup behavior. Test different providers, source formats and large Evidence limits. Establish bounded allocation and cleanup independently of transport limits; do not silently drop Unknowns. The MCP overflow error and export worked in the one synthetic Kuna test; Ghidra, other hosts and autonomous-agent decisions at this scale remain unverified.

References

Related #8288 and PR #8368 (new unavailable-owner state), #8184 (reference-discovery coverage), #8307 (exact retained record retrieval), and #8296 (JavaScript feature-trace views, a different workflow).

cc @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Independent resolution and offline paging control

On #8509 7f344af1a, real Kuna 1.988 MCP traced an owned arm64 Mach-O with 100 distinct direct callers to audit_target; direct Kuna reported 100 call rows. The trace returned 100 references and 100 owner-unavailable unknowns. In both full and compact profiles, list_unknowns recovered all 100 unique heads in six pages (limit 17), and export/import into a fresh target-free session preserved every record. The bundle summary found the retained trace, but inspect_analysis_view rejected it as an unsupported parent; trace_feature rejected detail: summary. A tokenx 1.3.0 estimate for the minified complete trace Evidence was 9,593 tokens / 35,063 bytes; this excludes the catalog, request, follow-up calls and transport duplication and is not a full-task or billing measurement.

An unmerged experiment grouped same-reason residual questions while keeping all 100 per-reference unavailable outcomes. Explicitly updating that single group as verified with Ghidra Evidence for only one source was accepted by verify_unknown_resolution. Under current individual questions, resolving that source left 99 open. The registry checks declared authority/confidence and the caller's asserted resolution; this does not prove all owners. The grouping experiment was removed. This controlled case demonstrates why preserving reference rows alone is insufficient to preserve independently resolvable unknown identity. An opt-in summary, retained pages, or shared text with individual heads remains worth investigating. Target SHA-256 b305faa9c5c31ae4f2413cad855402798288fc206867352a502bcbc1c0cb74f5 remained unchanged and it was not executed; one host and fixture only.

CLI Unknown registry handoff

On draft #8585 head 4809afeef, a real Kuna 1.988 run against the same owned arm64 Mach-O produced 100 owner-unavailable question strings in the saved CLI trace, but its exported bundle had zero Unknown registry entries. Separately, full and compact stdio MCP runs of the same query each returned 100 question strings and list_unknowns reported 100 independently addressable heads. A head includes scope, support, status and mutation Evidence; the strings alone do not provide that lifecycle. The target's SHA-256 stayed ce8582268988e3efe7441bdf87c69626be149ff5cb5ddf99b20f5d90d0f6b4d6 and it was not executed. This is a limitation of the proposed CLI handoff, not a claim that main already has its new CLI summary option.

Investigate an explicit selected-Unknown closure for a one-shot handoff, including revision history, relationship and Evidence dependencies, and exclusion of unrelated snapshot records. Do not synthesize new identities from question text or automatically export all user-managed Unknowns. #8546 covers the separate question of selecting scope for new Unknowns. The example is one fixture and provider; the appropriate export contract remains a design decision.

Snapshot handoff and selection control

On #8585 4809afeef, two real Kuna CLI invocations of the 100-reference fixture with the same --snapshot saved the 100 Unknown revisions and 100 record_unknown Evidence records in the analysis snapshot, while both separate trace Evidence bundles had zero registry entries. Copying the snapshot's evidence_bundle object to a separate JSON file and importing it into fresh target-free full and compact MCP sessions succeeded: each restored 101 Evidence records and 100 Unknown heads. This is a whole-snapshot workaround; it can include records beyond the requested trace, and its applicability to all snapshot states is untested.

A selection control then created one separate user-authored question supported by the same trace Evidence and exported the investigation. All 101 Unknowns referenced that trace as supporting Evidence. Thus selecting Unknowns solely by a supporting Evidence ID would also include a separately authored question; support establishes a relationship, not automatic ownership. The manually added question had null scope because it was created target-free. Selection of automatic questions across repeated runs, replay, revised questions and relationships needs an explicit ownership/identity rule and a validated dependency closure, rather than an inferred string match or unconditional snapshot export.

Originally created by @morluto on GitHub (Oct 31, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8373 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed How should an agent consume a large native `trace_feature` result with many unavailable owners in bounded pages or a useful summary, while retaining all observed references and the ability to manage residual unknowns without creating disproportionate session output? ### Context and evidence On the unmerged #8368 branch (`d25fd8408`), I generated owned arm64 Mach-O fixtures whose `_main` contains repeated static direct calls to `_goal`. The targets were not executed. Real verified Kuna 1.986 through stdio MCP reported every site, and the PR preserved an unavailable owner for each. The examples intentionally stress volume; they do not represent measured real-world frequency or model-token usage. - **3,000 sites:** `trace_feature("goal")` succeeded with 3,000 references and residual unknowns. The SDK-serialized reply was 1,981,493 bytes. `get_evidence_bundle({detail:"summary"})` succeeded but its SDK reply was 4,171,959 bytes; it summarized 3,001 records: the trace plus 3,000 `record_unknown` Evidence records. - **8,000 sites:** the trace succeeded with all sites; its SDK reply was 5,271,513 bytes. The unfiltered Evidence summary exceeded the default 10 MiB MCP response allowance and returned `resource_constraint`. A summary filtered by `operation:"trace_feature"` succeeded, `list_unknowns({offset:0,limit:5})` returned a page of 8,000 heads, and `export_evidence_bundle` produced a 25,091,963-byte file with 8,001 records and 8,000 unknowns. - **16,000 sites:** direct Kuna xrefs returned all 16,000 sites in 8,576,265 bytes. The MCP trace returned `resource_constraint` at the response boundary with an exact retained Evidence ID and export guidance; the exported 50,211,965-byte bundle contained that same trace ID, all 16,000 trace references, 16,000 unknowns and 16,001 records. A filtered summary and a five-item `list_unknowns` page succeeded; the unfiltered summary again exceeded the budget. The suggested CLI JSON alternative also succeeded on a new run, returning all 16,000 references in 5,043,664 bytes and the same Evidence ID in this controlled fixture. A fresh CLI run should not generally be treated as proof of an earlier live observation. SDK response sizes include both text and structured representations and are bytes, not token counts. The successful calls used full advertised output schemas; targets' hashes were unchanged. This is one host/provider/fixture shape, not a benchmark or a claim that every missing owner needs a separate durable question. ### Expected interaction with targets Read an explicitly opened local binary, run the existing static provider and write a caller-selected Evidence bundle when requested. No target execution or mutation is proposed. ### Boundary details Keep each source and target identity, unavailable reason, provider/profile, reference-cap coverage, existing explicit unknowns and complete Evidence accessible. An omitted page must not imply an absent reference. Preserve any user-managed independent Unknown revisions and distinguish page coverage from provider discovery. Avoid imposing an arbitrary cap or silently discarding sources. Any grouping of unknowns needs an explicit lifecycle and identity policy. ### Options and tradeoffs Compare a retained summary plus `inspect_analysis_view` pages for native feature matches/references, more general exact-record retrieval (#8307), and grouping a common unsupported-capability unknown while retaining per-reference outcomes. `list_unknowns` already pages heads and the bundle export is a working fallback; a bounded trace view would avoid requiring agents to transfer or inspect the entire file for a selected set of references. Decide whether `get_evidence_bundle` summary needs pagination or better filtered-summary recovery independently. Measure representative workloads and total agent interactions before claiming efficiency gains. ### Verification available so far I have a prototype or observed results ### Success criteria and open questions Exercise empty, single, shared-source, capped, many-source and mixed-owner cases; assert stable page/record identity, offline export/import, uncertainty and cancellation/cleanup behavior. Test different providers, source formats and large Evidence limits. Establish bounded allocation and cleanup independently of transport limits; do not silently drop Unknowns. The MCP overflow error and export worked in the one synthetic Kuna test; Ghidra, other hosts and autonomous-agent decisions at this scale remain unverified. ### References Related #8288 and PR #8368 (new unavailable-owner state), #8184 (reference-discovery coverage), #8307 (exact retained record retrieval), and #8296 (JavaScript feature-trace views, a different workflow). cc @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 ### Independent resolution and offline paging control On #8509 `7f344af1a`, real Kuna 1.988 MCP traced an owned arm64 Mach-O with 100 distinct direct callers to `audit_target`; direct Kuna reported 100 call rows. The trace returned 100 references and 100 owner-unavailable unknowns. In both full and compact profiles, `list_unknowns` recovered all 100 unique heads in six pages (limit 17), and export/import into a fresh target-free session preserved every record. The bundle summary found the retained trace, but `inspect_analysis_view` rejected it as an unsupported parent; `trace_feature` rejected `detail: summary`. A `tokenx` 1.3.0 estimate for the minified complete trace Evidence was 9,593 tokens / 35,063 bytes; this excludes the catalog, request, follow-up calls and transport duplication and is not a full-task or billing measurement. An unmerged experiment grouped same-reason residual questions while keeping all 100 per-reference unavailable outcomes. Explicitly updating that single group as verified with Ghidra Evidence for only one source was accepted by `verify_unknown_resolution`. Under current individual questions, resolving that source left 99 open. The registry checks declared authority/confidence and the caller's asserted resolution; this does not prove all owners. The grouping experiment was removed. This controlled case demonstrates why preserving reference rows alone is insufficient to preserve independently resolvable unknown identity. An opt-in summary, retained pages, or shared text with individual heads remains worth investigating. Target SHA-256 `b305faa9c5c31ae4f2413cad855402798288fc206867352a502bcbc1c0cb74f5` remained unchanged and it was not executed; one host and fixture only. ### CLI Unknown registry handoff On draft #8585 head `4809afeef`, a real Kuna 1.988 run against the same owned arm64 Mach-O produced 100 owner-unavailable question strings in the saved CLI trace, but its exported bundle had zero Unknown registry entries. Separately, full and compact stdio MCP runs of the same query each returned 100 question strings and `list_unknowns` reported 100 independently addressable heads. A head includes scope, support, status and mutation Evidence; the strings alone do not provide that lifecycle. The target's SHA-256 stayed `ce8582268988e3efe7441bdf87c69626be149ff5cb5ddf99b20f5d90d0f6b4d6` and it was not executed. This is a limitation of the proposed CLI handoff, not a claim that main already has its new CLI summary option. Investigate an explicit selected-Unknown closure for a one-shot handoff, including revision history, relationship and Evidence dependencies, and exclusion of unrelated snapshot records. Do not synthesize new identities from question text or automatically export all user-managed Unknowns. #8546 covers the separate question of selecting scope for new Unknowns. The example is one fixture and provider; the appropriate export contract remains a design decision. ### Snapshot handoff and selection control On #8585 `4809afeef`, two real Kuna CLI invocations of the 100-reference fixture with the same `--snapshot` saved the 100 Unknown revisions and 100 `record_unknown` Evidence records in the analysis snapshot, while both separate trace Evidence bundles had zero registry entries. Copying the snapshot's `evidence_bundle` object to a separate JSON file and importing it into fresh target-free full and compact MCP sessions succeeded: each restored 101 Evidence records and 100 Unknown heads. This is a whole-snapshot workaround; it can include records beyond the requested trace, and its applicability to all snapshot states is untested. A selection control then created one separate user-authored question supported by the same trace Evidence and exported the investigation. All 101 Unknowns referenced that trace as supporting Evidence. Thus selecting Unknowns solely by a supporting Evidence ID would also include a separately authored question; support establishes a relationship, not automatic ownership. The manually added question had null scope because it was created target-free. Selection of automatic questions across repeated runs, replay, revised questions and relationships needs an explicit ownership/identity rule and a validated dependency closure, rather than an inferred string match or unconditional snapshot export.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:12:20 +00:00

bsakaguchi commented

2026-11-01 21:12:21 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Fresh-session recovery control on the same macOS arm64 host: after the 16,000-reference MCP result exceeded the transport budget, a separate MCP server imported the exported 50,211,965-byte bundle successfully. import_evidence_bundle reported 16,001 records and 16,000 Unknowns; a trace_feature-filtered summary returned the exact trace Evidence ID from the original overflow, and list_unknowns({offset:0,limit:5}) returned five of 16,000 heads. The bundle SHA-256 was unchanged. Successful replies were validated by the TypeScript SDK against the full advertised schemas. The 8,000-reference exported bundle also imported and supported a filtered summary and Unknown page.

The summary tool description says to use a discovered Evidence ID with inspect_analysis_view, but that advice does not apply to this record: after import, both view: {kind: "summary"} and view: {kind: "native-observation", offset: 0, limit: 5} returned unsupported_target, explicitly because trace_feature is not a supported parent. Qualifying that guidance by supported parent operation would prevent a dead-end recommendation even before a bounded trace view exists.

This verifies export/import as an offline recovery path for these two synthetic Kuna fixtures. It does not establish an efficient way to inspect individual references inside the retained trace, test a fresh provider analysis, or establish behavior for Ghidra or other hosts.

Same-session control on REA PR #8368 head 0ebfcabf with the two-call-site Mach-O fixture and each real provider (verified Kuna 1.986 and Ghidra 12.1.4): analyze_function(_main) and trace_feature(goal) both appeared in get_evidence_bundle({detail:"summary"}). Passing the discovered function record to inspect_analysis_view with view: {kind:"summary"} succeeded; passing the discovered trace record to the same view returned unsupported_target. Both summary entries had the same metadata keys and no explicit view-availability field. The operation name is present, so an agent can consult the view contract, but the summary's unconditional recommendation is misleading. Both provider sessions closed successfully and the target digest was unchanged; this is a workflow control on one macOS arm64 fixture, not a claim about all view types or provider inputs.

@morluto commented on GitHub (Oct 31, 2026): Fresh-session recovery control on the same macOS arm64 host: after the 16,000-reference MCP result exceeded the transport budget, a separate MCP server imported the exported 50,211,965-byte bundle successfully. `import_evidence_bundle` reported 16,001 records and 16,000 Unknowns; a `trace_feature`-filtered summary returned the exact trace Evidence ID from the original overflow, and `list_unknowns({offset:0,limit:5})` returned five of 16,000 heads. The bundle SHA-256 was unchanged. Successful replies were validated by the TypeScript SDK against the full advertised schemas. The 8,000-reference exported bundle also imported and supported a filtered summary and Unknown page. The summary tool description says to use a discovered Evidence ID with `inspect_analysis_view`, but that advice does not apply to this record: after import, both `view: {kind: "summary"}` and `view: {kind: "native-observation", offset: 0, limit: 5}` returned `unsupported_target`, explicitly because `trace_feature` is not a supported parent. Qualifying that guidance by supported parent operation would prevent a dead-end recommendation even before a bounded trace view exists. This verifies export/import as an offline recovery path for these two synthetic Kuna fixtures. It does not establish an efficient way to inspect individual references inside the retained trace, test a fresh provider analysis, or establish behavior for Ghidra or other hosts. Same-session control on REA PR #8368 head `0ebfcabf` with the two-call-site Mach-O fixture and each real provider (verified Kuna 1.986 and Ghidra 12.1.4): `analyze_function(_main)` and `trace_feature(goal)` both appeared in `get_evidence_bundle({detail:"summary"})`. Passing the discovered function record to `inspect_analysis_view` with `view: {kind:"summary"}` succeeded; passing the discovered trace record to the same view returned `unsupported_target`. Both summary entries had the same metadata keys and no explicit view-availability field. The operation name is present, so an agent can consult the view contract, but the summary's unconditional recommendation is misleading. Both provider sessions closed successfully and the target digest was unchanged; this is a workflow control on one macOS arm64 fixture, not a claim about all view types or provider inputs.

bsakaguchi commented

2026-11-01 21:12:21 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Nov 1, 2026):

Proposal for the retained-view part only (source-derived on aa48d494e; I'll implement it unless someone objects). This mirrors #8522 for the JavaScript trace and leaves residual-unknown identity untouched.

One domain owner for the result. The trace_feature result shape currently exists only as literalTraceOutput in toolOutputSchemaGroups.ts. Move it to src/domain/native/literalFeatureTrace.ts; the contract output and the new view then parse with the same schema.
Retained native-trace view (shared by MCP inspect_analysis_view and CLI inspect-analysis-view):
- summary: query, search_mode, truncated, match counts by type, reference counts by owner state (found / not-found / unavailable), residual_unknowns count, and source provider/profile/parameters.
- {kind:"native-trace", collection:"references"|"matches"|"residual_unknowns", where?, offset, limit}: exact rows with source_index in retained order. references filters by exact target_addresses and owner state; matches filters by type.
- Page coverage counts only the retained rows and never claims discovery beyond what the provider reported. truncated and the provider limitations stay visible on every page.
trace_feature gains detail: "summary" through the existing enhanced-tool summary delivery (as get_call_graph does). The default stays complete.
Overflow continuation: an oversized retained trace returns a ready-to-call first references page.

Not changed: one residual Unknown per unavailable reference (the grouping experiment showed why independent identity matters), list_unknowns paging, and get_evidence_bundle summary size (#8528 adds exact record reads separately).

@kaoru0822-kitauji commented on GitHub (Nov 1, 2026): Proposal for the retained-view part only (source-derived on `aa48d494e`; I'll implement it unless someone objects). This mirrors #8522 for the JavaScript trace and leaves residual-unknown identity untouched. 1. **One domain owner for the result.** The `trace_feature` result shape currently exists only as `literalTraceOutput` in `toolOutputSchemaGroups.ts`. Move it to `src/domain/native/literalFeatureTrace.ts`; the contract output and the new view then parse with the same schema. 2. **Retained `native-trace` view** (shared by MCP `inspect_analysis_view` and CLI `inspect-analysis-view`): - `summary`: `query`, `search_mode`, `truncated`, match counts by type, reference counts by owner state (`found` / `not-found` / `unavailable`), `residual_unknowns` count, and source provider/profile/parameters. - `{kind:"native-trace", collection:"references"|"matches"|"residual_unknowns", where?, offset, limit}`: exact rows with `source_index` in retained order. `references` filters by exact `target_addresses` and owner state; `matches` filters by type. - Page coverage counts only the retained rows and never claims discovery beyond what the provider reported. `truncated` and the provider limitations stay visible on every page. 3. **`trace_feature` gains `detail: "summary"`** through the existing enhanced-tool summary delivery (as `get_call_graph` does). The default stays `complete`. 4. **Overflow continuation:** an oversized retained trace returns a ready-to-call first `references` page. Not changed: one residual Unknown per unavailable reference (the grouping experiment showed why independent identity matters), `list_unknowns` paging, and `get_evidence_bundle` summary size (#8528 adds exact record reads separately).

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6572

No description provided.

Rows
Columns