[GH-ISSUE #8773] [Design] Avoid repeated batch failure diagnostics without losing observations #7539

New issue

Open

opened 2026-11-03 11:23:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 11:23:23 +00:00

Owner

Originally created by @morluto on GitHub (Nov 3, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8773

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Should the agent-facing batch item view represent equivalent identity-resolution and decompilation diagnostics more compactly, while preserving both observations and their provenance in retained Evidence?

Context and evidence

Using the public MCP workflow in the #8768 Ghidra ambiguity-recovery branch, I submitted a three-item batch containing a duplicate-function-name selector and two exact addresses, then requested the retained batch-decompile items page filtered to errors. Both real Ghidra 12.1.4 and Kuna 1.989 returned one failed item and two successes; both output profiles behaved alike. The failed row contains an identity lookup error and a decompilation error with the same ambiguity message and candidate guidance.

For this one Ghidra page, the serialized result was 7,800 bytes, of which the lookup and decompilation error objects were 2,289 and 2,293 bytes respectively. The corresponding Kuna page was 3,061 bytes with two 277-byte error objects. These are JSON byte counts for one small fixture, not token, latency, or representative-workload measurements. The Ghidra objects carry distinct procedure_address and procedure_pseudo_code operation provenance and detailed diagnostics; they are not the same observation.

Source inspection: EnhancedTools.#batchDecompile resolves each identity, retains a failed resolution as unknown, and still attempts pseudocode using the caller's original selector. This fallback is intentional and may succeed even when lookup fails, so treating all lookup failures as terminal would lose useful results. The existing items view projects both errors without field selection. The full parent is retained separately.

Expected interaction with targets

Read existing files or captured data. A projection of a retained result need not rerun analysis.

Boundary details

Preserve original selector and item position, both independently observed outcomes, actionable recovery, provider and operation provenance, and access to the complete parent. Do not imply the failures are interchangeable or identical merely because their messages match.

Options and tradeoffs

Consider an explicitly compact projection with a reference to complete retained diagnostics, or keeping the present output if the extra complexity and calls outweigh the benefit. A different option is avoiding a second call only where a provider contract proves that a particular failure is terminal for that selector; no such cross-provider guarantee was established here. Do not infer equivalence from error text.

Verification available so far

I have observed results. No autonomous agent, token consumption, elapsed-time comparison, or large batch was measured. Test a lookup failure followed by successful decompilation, distinct failures, repeated selectors and real provider results before changing the contract.

References

Related: #8498 (retained batch views), #8350 (per-item provider provenance), #8768 (ambiguity recovery).

@gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Originally created by @morluto on GitHub (Nov 3, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8773 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Should the agent-facing batch item view represent equivalent identity-resolution and decompilation diagnostics more compactly, while preserving both observations and their provenance in retained Evidence? ### Context and evidence Using the public MCP workflow in the #8768 Ghidra ambiguity-recovery branch, I submitted a three-item batch containing a duplicate-function-name selector and two exact addresses, then requested the retained `batch-decompile` items page filtered to errors. Both real Ghidra 12.1.4 and Kuna 1.989 returned one failed item and two successes; both output profiles behaved alike. The failed row contains an identity lookup error and a decompilation error with the same ambiguity message and candidate guidance. For this one Ghidra page, the serialized result was 7,800 bytes, of which the lookup and decompilation error objects were 2,289 and 2,293 bytes respectively. The corresponding Kuna page was 3,061 bytes with two 277-byte error objects. These are JSON byte counts for one small fixture, not token, latency, or representative-workload measurements. The Ghidra objects carry distinct `procedure_address` and `procedure_pseudo_code` operation provenance and detailed diagnostics; they are not the same observation. Source inspection: `EnhancedTools.#batchDecompile` resolves each identity, retains a failed resolution as unknown, and still attempts pseudocode using the caller's original selector. This fallback is intentional and may succeed even when lookup fails, so treating all lookup failures as terminal would lose useful results. The existing items view projects both errors without field selection. The full parent is retained separately. ### Expected interaction with targets Read existing files or captured data. A projection of a retained result need not rerun analysis. ### Boundary details Preserve original selector and item position, both independently observed outcomes, actionable recovery, provider and operation provenance, and access to the complete parent. Do not imply the failures are interchangeable or identical merely because their messages match. ### Options and tradeoffs Consider an explicitly compact projection with a reference to complete retained diagnostics, or keeping the present output if the extra complexity and calls outweigh the benefit. A different option is avoiding a second call only where a provider contract proves that a particular failure is terminal for that selector; no such cross-provider guarantee was established here. Do not infer equivalence from error text. ### Verification available so far I have observed results. No autonomous agent, token consumption, elapsed-time comparison, or large batch was measured. Test a lookup failure followed by successful decompilation, distinct failures, repeated selectors and real provider results before changing the contract. ### References Related: #8498 (retained batch views), #8350 (per-item provider provenance), #8768 (ambiguity recovery). @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

bsakaguchi added the

enhancement

triage

labels

2026-11-03 11:23:23 +00:00

bsakaguchi referenced this issue

2026-11-05 01:47:54 +00:00

[PR #7332] [MERGED] feat(environment): serve query numbers, counted values and made objects #7818