morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8498] [Design] Retrieve selected results from retained batch decompilation #6623

New issue

Closed

opened 2026-11-01 21:12:45 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-11-01 21:12:45 +00:00

Owner

Originally created by @morluto on GitHub (Nov 1, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8498

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

How should an agent retrieve selected results from a completed, retained batch_decompile when the complete response does not fit the MCP transport?

Context and evidence

On REA e5e0dc554 (macOS arm64, Node 24), a public in-memory MCP registration used a controlled synthetic provider returning 64 successful 100 KiB pseudocode strings. At the normal 10 MiB response budget, the call completed all 64 items and returned a transport constraint with a retained Evidence reference and export guidance. The retained batch existed in the ledger. Asking the production analysis-view service for its summary returned unsupported: Parent Evidence operation "batch_decompile" does not support analysis views. No real provider, natural output frequency, or autonomous agent was measured in this control.

An unmerged experiment for #8350 adds per-item provider provenance. In a separate synthetic control, a single 6 MiB raw observation made its response exceed the same budget even though the current output fit; the complete experimental result was retained. This demonstrates a design tradeoff, not a claim that Ghidra or Kuna emitted 6 MiB of metadata. Their tested fixture observations were substantially smaller.

Expected interaction with targets

Read existing files or captured data. A retained view should not rerun analysis.

Boundary details

Keep the retained parent as the source of truth. Preserve ordered duplicates, each item's identity, status, pseudocode and relevant provider caveats; identify omissions and do not claim unavailable source closure or retained records. No target mutation or execution is proposed.

Options and tradeoffs

Consider a selected or paged batch view, or a compact response that explicitly points to complete retained Evidence. Compare agent call count and serialized response cost; retain an export path for an individual result that cannot fit. Avoid silently changing an Evidence record while keeping its original ID.

Verification available so far

I have a prototype or observed results. The overflow control above uses a synthetic provider; current-main source behavior and a public MCP response were checked. Real provider metadata sizing was exercised only on a small fixture.

Success criteria and open questions

A client can access a selected completed item and understand its provenance and omissions without repeating analysis or transferring the entire bundle. Verify full and compact SDK schemas, retention failure, duplicates, large individual items, cancellation and applicable real-provider workflows.

References

Related but distinct: #5930 concerns input workload bounds; #8350 concerns lost item-specific producer provenance; #8307 concerns selected historical annotation Evidence. #8444 addresses retention-failure recovery.

@gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Follow-up: retention failure and the input boundary

In an unmerged combination of an item-view experiment and #8444 e619141b9, a synthetic 64-item batch completed but could not retain its source or parent because the test ledger had zero capacity. The error returned the completed unretained batch with source closure explicitly unestablished. Sending that batch back inline to the experimental view exceeded the default 1,024 combined input elements before the view ran. Writing the returned Evidence to an owned standalone JSON file and selecting source.kind: evidence-file allowed a selected page; with the ledger still full, the page itself was returned as unretained Evidence inside a capacity error. No analysis was repeated.

Separate real Kuna 1.986 and Ghidra 12.1.4 controls used real pseudocode and an injected synthetic source and zero ledger capacity; small inline batch and page recovery succeeded as unretained error payloads. These are controlled boundary experiments, not a claim about natural failure frequency or a shipped view. This handoff may require local file access; a practical agent contract should explain applicable choices without claiming source records were retained or requiring a larger input limit by default.

Originally created by @morluto on GitHub (Nov 1, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8498 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed How should an agent retrieve selected results from a completed, retained `batch_decompile` when the complete response does not fit the MCP transport? ### Context and evidence On REA `e5e0dc554` (macOS arm64, Node 24), a public in-memory MCP registration used a controlled synthetic provider returning 64 successful 100 KiB pseudocode strings. At the normal 10 MiB response budget, the call completed all 64 items and returned a transport constraint with a retained Evidence reference and export guidance. The retained batch existed in the ledger. Asking the production analysis-view service for its summary returned unsupported: `Parent Evidence operation "batch_decompile" does not support analysis views.` No real provider, natural output frequency, or autonomous agent was measured in this control. An unmerged experiment for #8350 adds per-item provider provenance. In a separate synthetic control, a single 6 MiB raw observation made its response exceed the same budget even though the current output fit; the complete experimental result was retained. This demonstrates a design tradeoff, not a claim that Ghidra or Kuna emitted 6 MiB of metadata. Their tested fixture observations were substantially smaller. ### Expected interaction with targets Read existing files or captured data. A retained view should not rerun analysis. ### Boundary details Keep the retained parent as the source of truth. Preserve ordered duplicates, each item's identity, status, pseudocode and relevant provider caveats; identify omissions and do not claim unavailable source closure or retained records. No target mutation or execution is proposed. ### Options and tradeoffs Consider a selected or paged batch view, or a compact response that explicitly points to complete retained Evidence. Compare agent call count and serialized response cost; retain an export path for an individual result that cannot fit. Avoid silently changing an Evidence record while keeping its original ID. ### Verification available so far I have a prototype or observed results. The overflow control above uses a synthetic provider; current-main source behavior and a public MCP response were checked. Real provider metadata sizing was exercised only on a small fixture. ### Success criteria and open questions A client can access a selected completed item and understand its provenance and omissions without repeating analysis or transferring the entire bundle. Verify full and compact SDK schemas, retention failure, duplicates, large individual items, cancellation and applicable real-provider workflows. ### References Related but distinct: #5930 concerns input workload bounds; #8350 concerns lost item-specific producer provenance; #8307 concerns selected historical annotation Evidence. #8444 addresses retention-failure recovery. @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 ### Follow-up: retention failure and the input boundary In an unmerged combination of an item-view experiment and #8444 `e619141b9`, a synthetic 64-item batch completed but could not retain its source or parent because the test ledger had zero capacity. The error returned the completed unretained batch with source closure explicitly unestablished. Sending that batch back inline to the experimental view exceeded the default 1,024 combined input elements before the view ran. Writing the returned Evidence to an owned standalone JSON file and selecting `source.kind: evidence-file` allowed a selected page; with the ledger still full, the page itself was returned as unretained Evidence inside a capacity error. No analysis was repeated. Separate real Kuna 1.986 and Ghidra 12.1.4 controls used real pseudocode and an injected synthetic source and zero ledger capacity; small inline batch and page recovery succeeded as unretained error payloads. These are controlled boundary experiments, not a claim about natural failure frequency or a shipped view. This handoff may require local file access; a practical agent contract should explain applicable choices without claiming source records were retained or requiring a larger input limit by default.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:12:45 +00:00

bsakaguchi

2026-11-03 00:30:03 +00:00

closed this issue
added
enhancement
and removed
enhancement
labels

bsakaguchi commented

2026-11-03 00:30:05 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Nov 2, 2026):

Proposal, based on the shared analysis-view path (I'm starting on this now; objections welcome):

Delivery on batch_decompile

Add opt-in detail: "summary". It retains the complete batch and returns counts plus parent_evidence_id. The default stays complete.
A complete batch that exceeds the MCP budget gets a ready-to-call first index page as its continuation, alongside the existing export guidance.

inspect_analysis_view on a batch_decompile parent (retained, Evidence file or portable bundle; no analysis is repeated)

summary: producer counts, identity counts, total and largest body size, and provider and profile identity.
{kind:"batch-decompile", collection:"items", where:{indices, addresses, procedure_addresses, statuses}, offset, limit, include_pseudocode}: rows in retained order, keyed by original index, so duplicate selectors stay separate. Bodies are opt-in. By default a row carries pseudocode_code_units, or its exact error and limitations.
{kind:"batch-decompile", collection:"pseudocode", index, offset, limit}: UTF-16 pages of one body, reusing the existing text pager, so a single oversized item is still reachable without an export.

Evidence semantics

The parent stays the source of truth and is never rewritten.
Page coverage only describes the selected rows. It says nothing about functions outside the batch.
Batch-level limitations stay visibly aggregated: they are not attributed to individual ok items (that's #8350).

Out of scope: input bounds (#5930), per-item provider provenance (#8350), and retention-failure recovery (#8444). An unretained batch can already use the evidence-file source.

@kaoru0822-kitauji commented on GitHub (Nov 2, 2026): Proposal, based on the shared analysis-view path (I'm starting on this now; objections welcome): **Delivery on `batch_decompile`** - Add opt-in `detail: "summary"`. It retains the complete batch and returns counts plus `parent_evidence_id`. The default stays `complete`. - A complete batch that exceeds the MCP budget gets a ready-to-call first index page as its continuation, alongside the existing export guidance. **`inspect_analysis_view` on a `batch_decompile` parent** (retained, Evidence file or portable bundle; no analysis is repeated) - `summary`: producer counts, identity counts, total and largest body size, and provider and profile identity. - `{kind:"batch-decompile", collection:"items", where:{indices, addresses, procedure_addresses, statuses}, offset, limit, include_pseudocode}`: rows in retained order, keyed by original `index`, so duplicate selectors stay separate. Bodies are opt-in. By default a row carries `pseudocode_code_units`, or its exact `error` and `limitations`. - `{kind:"batch-decompile", collection:"pseudocode", index, offset, limit}`: UTF-16 pages of one body, reusing the existing text pager, so a single oversized item is still reachable without an export. **Evidence semantics** - The parent stays the source of truth and is never rewritten. - Page coverage only describes the selected rows. It says nothing about functions outside the batch. - Batch-level limitations stay visibly aggregated: they are not attributed to individual `ok` items (that's #8350). **Out of scope:** input bounds (#5930), per-item provider provenance (#8350), and retention-failure recovery (#8444). An unretained batch can already use the `evidence-file` source.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6623

No description provided.

Rows
Columns