morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8184] [Design] Make feature-trace reference coverage explicit for agents #6468

New issue

Open

opened 2026-11-01 21:11:22 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-11-01 21:11:22 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8184

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Should trace_feature expose structured reference-coverage status separately from truncation and residual execution failures, so an agent can distinguish no returned references from a verified absence without interpreting provider-specific prose?

Context and evidence

On macOS arm64, a self-built, unexecuted arm64 Mach-O contained a Base64 alphabet and a function that indexed it. In real REA MCP sessions, trace_feature({query:"ABCDEFGHIJKLMNOPQRSTUVWXYZ"}) with Ghidra 12.1.4 returned the string and one source reference whose containing procedure was _audit_base64_character. With Kuna 1.986 it returned the string, references: [], truncated: false, and residual_unknowns: []. Kuna's Evidence did include the explicit limitation that its static reference walk may omit edges and an empty result does not prove no reference exists. No claim of exact complete runtime references follows from either provider.

Source inspection shows truncated and the residual list reflect explicit provider-reported truncation or encountered lookup gaps; they are not documented as a proof of exhaustive reference discovery. The question is whether that distinction is sufficiently available as structured data to agent workflows when the observed references are empty. This is one controlled example, not a measured model error rate or a claim that Kuna violates its capability contract.

A second control on current REA 85013da0c and Kuna 1.988 used a self-built x86-64 ELF. Direct Kuna strings with reference walking reported one reference and function _start for both éclair and WIDER; --no-xrefs returned zero counts and empty function lists because it did not walk references. REA deliberately uses the latter inventory. Real MCP trace_feature({query:"éclair"}) then found the exact reference at 0x2011ac, but reported its containing procedure as unavailable because Kuna does not declare the containment operation. Direct Kuna associates that string with _start at 0x201190; this demonstrates available provider evidence, not that its association can always be converted into an exact source-to-function mapping. Ghidra exact xrefs independently returned the same source for this fixture. The target digest remained unchanged and both sessions closed.

Consider whether a bounded analyst workflow can expose this association with its actual provider semantics and provenance, while keeping unsupported exact containment unknown. Compare the cost of Kuna reference walks with the existing separate xrefs path; no performance measurement or general coverage claim was made.

A further controlled string case places the sole observed reference inside the string rather than at its start. REA/Kuna trace_feature returned the string with an empty reference list, truncated:false, and no residual unknowns; direct Kuna reported one reference and its referring function for the string extent. Exact REA xrefs at the interior destination found the reference with both Kuna and Ghidra. Evidence for the trace states the exact-address limitation. See #8569 for fixture, disassembly, addresses and digest. This is a scope-of-lookup gap as well as a reporting question: structured coverage alone would not discover the interior reference.

Expected interaction with targets

Read an opened local binary and query the bound static-analysis provider; no target execution or mutation is proposed.

Boundary details (optional)

Preserve returned rows, per-match outcomes and the existing distinction between actual truncation, unknown discovery coverage, incomplete searches and runtime reachability. Do not promote Ghidra's observed reference to proof of complete ground truth or mark all Kuna answers truncated. Provider limitations and source coverage should originate at their owning provider boundary, not be inferred from text or from another provider's result.

Options and tradeoffs

Compare an explicit per-match or aggregate reference_coverage with unknown/provider-observed semantics, a smaller inline summary plus retained per-match details, and current limitations-only reporting. Reuse existing evidence/relationship coverage contracts where applicable rather than adding another provider-specific tool. Account for extra payload and compatibility, and preserve relevant source limitations.

Verification available so far

I have a prototype or observed results

Success criteria and open questions

Test empty and nonempty references, explicit provider truncation, unsupported containment, lookup failures and missing coverage through real MCP and controlled provider cases. Confirm that truncated stays about observed truncation and that an unavailable coverage fact is not presented as complete. A controlled model trace would be needed to measure any change in agent decisions.

References

Existing truncation fix: https://github.com/morluto/rea/issues/2558

Global-name discovery and typed reference control

On REA main 9ebbe67e and Ghidra 12.1.4, a source-built unexecuted arm64 Mach-O contained global _observed_global at 0x100004000 and a pointer slot at 0x100004008; dyld_info -fixups independently reported the slot's rebase to the global. In both MCP profiles, trace_feature({query:"observed_global"}) matched a string containing the symbol name, but did not match the global label or report the pointer slot as its reference. This follows the advertised trace scope of matching strings and procedures, so it is not a claim that the tool violated its contract.

A separate list_names call did identify the imported global label at 0x100004000. xrefs at that address reported the slot, and inspect_native_instruction returned it as a typed Ghidra DATA reference even though its target status was not-in-listing. The separately requested instruction record and trace record survived export/import unchanged in a fresh server without an open binary; the trace did not link the instruction record. Target SHA-256 167f7b743182b707705cc06a5318222c82d69b4dd4f19c32213e078f717d4aec stayed unchanged.

This control distinguishes the existing documented string/procedure trace scope from data-symbol discovery. Consider whether an agent searching a named global needs a bounded, explicit handoff to the existing name inventory; do not present the matched symbol-name string as the global itself or treat empty trace references as proof of absence. It does not establish agent behavior, general provider coverage, or a need for automatic expansion of every trace.

Unwalked source affects incoming references

On the unexecuted Mach-O used in #8362 (SHA-256 17a184835ff22f86c6398d96b699acdd2eb200ebb3d966d6fb5af7d0cebff4ed), direct Kuna 1.990 xrefs --to 0x1000003c8 returned _direct_goal's call at 0x1000003d0 and reported _middle at 0x1000003e0 in coverage.unwalked. A separate explicitly requested xrefs --from 0x1000003e0 --kind call found its call to _goal. The unwalked list thus describes sources outside the walk, not merely whether the queried target was decoded.

On REA #8813 head 187f3d8cc, compact/full MCP xrefs returned the first call but did not retain the unwalked list. trace_feature for _goal included that reference, omitted _middle's call and set truncated: false; its residual unknown concerned the unavailable containing-procedure lookup, not the unwalked source. Generic provider limitations remain. Successful replies used the SDK and fetched schemas, the target hash was unchanged, and the target was never executed. This is one observed coverage gap, not proof that every unwalked source references the selected target. A producer-owned structured observation must preserve that distinction and should avoid copying a full unbounded source list into every composed match.

Repeated image-level coverage gaps in a prototype

A local, unmerged prototype propagated known Kuna reference-discovery gaps into trace_feature.residual_unknowns, while preserving truncated: false for gaps that were not response caps. On a source-built, unexecuted Mach-O with 20 distinct audit_coverage_* functions, real Kuna 1.990 through REA stdio MCP returned 20 matches and no reference rows in each schema profile. The prototype emitted 20 target-specific questions for the same provider report of no pointer-relocation coverage. Those strings occupied 3,041 serialized UTF-8 bytes, and an exported session bundle contained 20 Unknown records and 41 Evidence records (116,898 bytes). The fixture's SHA-256 bbfac7c7ad3f2e88ff202657557c9e2a4a141fa30ef92d327105ed2b876f52ec was unchanged. These are counts for this prototype and fixture, not token, performance, or prevalence measurements.

Consider representing a shared image-level relocation-metadata limitation once with an explicit scope and retained per-target source links, rather than materializing an independent question for every affected target. Any grouping must preserve which targets were actually covered and must not make resolving one target's distinct unknown resolve another. Compare this with per-match details for cases where coverage really differs; the observation does not establish that all provider gaps are global.

Kuna 1.990 documentation describes data_refs_coverage: "none" as an image listing no pointer relocations, but the loader also emits it for unsupported formats or architectures and some relocation-reading failures. Therefore it does not establish that relocation metadata is absent. See https://github.com/Noelo-Lab/kuna/blob/v1.990/decompiler/crates/kuna-analysis/src/loader/kuna_dataptrs.rs.

The RISC-V control reproducing the upstream ambiguity is reported at https://github.com/Noelo-Lab/kuna/issues/1502. Current REA rejects that RISC-V target as unsupported before invoking Kuna, so the upstream reproduction is not a current REA RISC-V failure. For REA-supported images, none should still be treated as unknown coverage rather than as proof of absent relocation metadata.

A follow-up in draft #8851 gives provider-reported gaps an explicit image or observation scope. The real 20-match control now retained one image-wide relocation-coverage Unknown and all 20 linked xrefs sources in both profiles; the exported bundle was 60,164 bytes, versus 116,898 bytes and 20 Unknowns in the earlier prototype. A separate real unwalked-code control retained distinct questions for the two observed targets. These are serialized sizes for one controlled fixture, not token or latency measurements. Grouping is based on producer-defined scope and code, not matching prose; review of shared-Unknown resolution semantics remains open.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8184 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Should `trace_feature` expose structured reference-coverage status separately from truncation and residual execution failures, so an agent can distinguish no returned references from a verified absence without interpreting provider-specific prose? ### Context and evidence On macOS arm64, a self-built, unexecuted arm64 Mach-O contained a Base64 alphabet and a function that indexed it. In real REA MCP sessions, `trace_feature({query:"ABCDEFGHIJKLMNOPQRSTUVWXYZ"})` with Ghidra 12.1.4 returned the string and one source reference whose containing procedure was `_audit_base64_character`. With Kuna 1.986 it returned the string, `references: []`, `truncated: false`, and `residual_unknowns: []`. Kuna's Evidence **did** include the explicit limitation that its static reference walk may omit edges and an empty result does not prove no reference exists. No claim of exact complete runtime references follows from either provider. Source inspection shows `truncated` and the residual list reflect explicit provider-reported truncation or encountered lookup gaps; they are not documented as a proof of exhaustive reference discovery. The question is whether that distinction is sufficiently available as structured data to agent workflows when the observed references are empty. This is one controlled example, not a measured model error rate or a claim that Kuna violates its capability contract. A second control on current REA `85013da0c` and Kuna 1.988 used a self-built x86-64 ELF. Direct Kuna `strings` with reference walking reported one reference and function `_start` for both `éclair` and `WIDER`; `--no-xrefs` returned zero counts and empty function lists because it did not walk references. REA deliberately uses the latter inventory. Real MCP `trace_feature({query:"éclair"})` then found the exact reference at `0x2011ac`, but reported its containing procedure as unavailable because Kuna does not declare the containment operation. Direct Kuna associates that string with `_start` at `0x201190`; this demonstrates available provider evidence, not that its association can always be converted into an exact source-to-function mapping. Ghidra exact `xrefs` independently returned the same source for this fixture. The target digest remained unchanged and both sessions closed. Consider whether a bounded analyst workflow can expose this association with its actual provider semantics and provenance, while keeping unsupported exact containment unknown. Compare the cost of Kuna reference walks with the existing separate `xrefs` path; no performance measurement or general coverage claim was made. A further controlled string case places the sole observed reference inside the string rather than at its start. REA/Kuna `trace_feature` returned the string with an empty reference list, `truncated:false`, and no residual unknowns; direct Kuna reported one reference and its referring function for the string extent. Exact REA xrefs at the interior destination found the reference with both Kuna and Ghidra. Evidence for the trace states the exact-address limitation. See #8569 for fixture, disassembly, addresses and digest. This is a scope-of-lookup gap as well as a reporting question: structured coverage alone would not discover the interior reference. ### Expected interaction with targets Read an opened local binary and query the bound static-analysis provider; no target execution or mutation is proposed. ### Boundary details (optional) Preserve returned rows, per-match outcomes and the existing distinction between actual truncation, unknown discovery coverage, incomplete searches and runtime reachability. Do not promote Ghidra's observed reference to proof of complete ground truth or mark all Kuna answers truncated. Provider limitations and source coverage should originate at their owning provider boundary, not be inferred from text or from another provider's result. ### Options and tradeoffs Compare an explicit per-match or aggregate `reference_coverage` with `unknown`/provider-observed semantics, a smaller inline summary plus retained per-match details, and current limitations-only reporting. Reuse existing evidence/relationship coverage contracts where applicable rather than adding another provider-specific tool. Account for extra payload and compatibility, and preserve relevant source limitations. ### Verification available so far I have a prototype or observed results ### Success criteria and open questions Test empty and nonempty references, explicit provider truncation, unsupported containment, lookup failures and missing coverage through real MCP and controlled provider cases. Confirm that `truncated` stays about observed truncation and that an unavailable coverage fact is not presented as complete. A controlled model trace would be needed to measure any change in agent decisions. ### References Existing truncation fix: https://github.com/morluto/rea/issues/2558 ### Global-name discovery and typed reference control On REA main `9ebbe67e` and Ghidra 12.1.4, a source-built unexecuted arm64 Mach-O contained global `_observed_global` at `0x100004000` and a pointer slot at `0x100004008`; `dyld_info -fixups` independently reported the slot's rebase to the global. In both MCP profiles, `trace_feature({query:"observed_global"})` matched a string containing the symbol name, but did not match the global label or report the pointer slot as its reference. This follows the advertised trace scope of matching strings and procedures, so it is not a claim that the tool violated its contract. A separate `list_names` call did identify the imported global label at `0x100004000`. `xrefs` at that address reported the slot, and `inspect_native_instruction` returned it as a typed Ghidra DATA reference even though its target status was `not-in-listing`. The separately requested instruction record and trace record survived export/import unchanged in a fresh server without an open binary; the trace did not link the instruction record. Target SHA-256 `167f7b743182b707705cc06a5318222c82d69b4dd4f19c32213e078f717d4aec` stayed unchanged. This control distinguishes the existing documented string/procedure trace scope from data-symbol discovery. Consider whether an agent searching a named global needs a bounded, explicit handoff to the existing name inventory; do not present the matched symbol-name string as the global itself or treat empty trace references as proof of absence. It does not establish agent behavior, general provider coverage, or a need for automatic expansion of every trace. ### Unwalked source affects incoming references On the unexecuted Mach-O used in #8362 (SHA-256 `17a184835ff22f86c6398d96b699acdd2eb200ebb3d966d6fb5af7d0cebff4ed`), direct Kuna 1.990 `xrefs --to 0x1000003c8` returned `_direct_goal`'s call at `0x1000003d0` and reported `_middle` at `0x1000003e0` in `coverage.unwalked`. A separate explicitly requested `xrefs --from 0x1000003e0 --kind call` found its call to `_goal`. The unwalked list thus describes sources outside the walk, not merely whether the queried target was decoded. On REA #8813 head `187f3d8cc`, compact/full MCP `xrefs` returned the first call but did not retain the unwalked list. `trace_feature` for `_goal` included that reference, omitted `_middle`'s call and set `truncated: false`; its residual unknown concerned the unavailable containing-procedure lookup, not the unwalked source. Generic provider limitations remain. Successful replies used the SDK and fetched schemas, the target hash was unchanged, and the target was never executed. This is one observed coverage gap, not proof that every unwalked source references the selected target. A producer-owned structured observation must preserve that distinction and should avoid copying a full unbounded source list into every composed match. ### Repeated image-level coverage gaps in a prototype A local, unmerged prototype propagated known Kuna reference-discovery gaps into `trace_feature.residual_unknowns`, while preserving `truncated: false` for gaps that were not response caps. On a source-built, unexecuted Mach-O with 20 distinct `audit_coverage_*` functions, real Kuna 1.990 through REA stdio MCP returned 20 matches and no reference rows in each schema profile. The prototype emitted 20 target-specific questions for the same provider report of no pointer-relocation coverage. Those strings occupied 3,041 serialized UTF-8 bytes, and an exported session bundle contained 20 Unknown records and 41 Evidence records (116,898 bytes). The fixture's SHA-256 `bbfac7c7ad3f2e88ff202657557c9e2a4a141fa30ef92d327105ed2b876f52ec` was unchanged. These are counts for this prototype and fixture, not token, performance, or prevalence measurements. Consider representing a shared image-level relocation-metadata limitation once with an explicit scope and retained per-target source links, rather than materializing an independent question for every affected target. Any grouping must preserve which targets were actually covered and must not make resolving one target's distinct unknown resolve another. Compare this with per-match details for cases where coverage really differs; the observation does not establish that all provider gaps are global. Kuna 1.990 documentation describes `data_refs_coverage: "none"` as an image listing no pointer relocations, but the loader also emits it for unsupported formats or architectures and some relocation-reading failures. Therefore it does not establish that relocation metadata is absent. See https://github.com/Noelo-Lab/kuna/blob/v1.990/decompiler/crates/kuna-analysis/src/loader/kuna_dataptrs.rs. The RISC-V control reproducing the upstream ambiguity is reported at https://github.com/Noelo-Lab/kuna/issues/1502. Current REA rejects that RISC-V target as unsupported before invoking Kuna, so the upstream reproduction is not a current REA RISC-V failure. For REA-supported images, `none` should still be treated as unknown coverage rather than as proof of absent relocation metadata. A follow-up in draft #8851 gives provider-reported gaps an explicit image or observation scope. The real 20-match control now retained one image-wide relocation-coverage Unknown and all 20 linked xrefs sources in both profiles; the exported bundle was 60,164 bytes, versus 116,898 bytes and 20 Unknowns in the earlier prototype. A separate real unwalked-code control retained distinct questions for the two observed targets. These are serialized sizes for one controlled fixture, not token or latency measurements. Grouping is based on producer-defined scope and code, not matching prose; review of shared-Unknown resolution semantics remains open.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:22 +00:00

bsakaguchi commented

2026-11-01 21:11:22 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

A related real-provider control adds a discovery limitation distinct from reference coverage: on an owned Mach-O with isolated QZ7\0 and QZ7_LONG_MARK!\0, trace_feature({query:"QZ7"}) returned the longer inventory string, empty references, truncated:false, and residual_unknowns:[] with both Ghidra 12.1.4 and Kuna 1.986. The Evidence limitations disclosed each provider's incomplete string inventory; direct Kuna with minimum length 3 and REA/Ghidra exact byte search independently located the shorter literal. Empty exact xrefs did not prove absence of indexed use in this fixture.

This is not a claim that truncated:false promised raw-byte completeness or that either provider violated its documented contract. #8218 discusses the separate task of locating known short/encoded byte candidates; this issue remains about representing reference coverage truthfully.

@morluto commented on GitHub (Oct 30, 2026): A related real-provider control adds a **discovery** limitation distinct from reference coverage: on an owned Mach-O with isolated `QZ7\0` and `QZ7_LONG_MARK!\0`, `trace_feature({query:"QZ7"})` returned the longer inventory string, empty references, `truncated:false`, and `residual_unknowns:[]` with both Ghidra 12.1.4 and Kuna 1.986. The Evidence limitations disclosed each provider's incomplete string inventory; direct Kuna with minimum length 3 and REA/Ghidra exact byte search independently located the shorter literal. Empty exact xrefs did not prove absence of indexed use in this fixture. This is not a claim that `truncated:false` promised raw-byte completeness or that either provider violated its documented contract. #8218 discusses the separate task of locating known short/encoded byte candidates; this issue remains about representing reference coverage truthfully.

bsakaguchi commented

2026-11-01 21:11:23 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi commented

2026-11-01 21:11:23 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

CLI rendering control on the same source-owned, unexecuted Base64-alphabet Mach-O, REA b43c329 and Kuna 1.986: both ordinary rea trace <fixture> ABCDEFGHIJKLMNOPQRSTUVWXYZ --provider kuna output and --format json exited 0 and included the matched string, references: [], truncated: false, residual_unknowns: [], and the provider's explicit limitation that an empty exact-address xref result does not prove absence. Both retained the Evidence subject digest and upstream provider profile. The target SHA-256 was unchanged (6d276076713030d87100f3a98e551b88de799193d5f9e1b57c1774d583aa77e9).

This is a positive check that the existing caveat is visible in these two CLI output formats, not evidence of an omission by CLI rendering. It leaves the original design question: whether machine-readable coverage would help agents distinguish incomplete discovery from complete absence, beyond the available prose. No autonomous agent decisions were measured.

@morluto commented on GitHub (Oct 31, 2026): CLI rendering control on the same source-owned, unexecuted Base64-alphabet Mach-O, REA `b43c329` and Kuna 1.986: both ordinary `rea trace <fixture> ABCDEFGHIJKLMNOPQRSTUVWXYZ --provider kuna` output and `--format json` exited 0 and included the matched string, `references: []`, `truncated: false`, `residual_unknowns: []`, and the provider's explicit limitation that an empty exact-address xref result does not prove absence. Both retained the Evidence subject digest and upstream provider profile. The target SHA-256 was unchanged (`6d276076713030d87100f3a98e551b88de799193d5f9e1b57c1774d583aa77e9`). This is a positive check that the existing caveat is visible in these two CLI output formats, not evidence of an omission by CLI rendering. It leaves the original design question: whether machine-readable coverage would help agents distinguish incomplete discovery from complete absence, beyond the available prose. No autonomous agent decisions were measured.

bsakaguchi commented

2026-11-01 21:11:23 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

A real current-main (60149282) Evidence handoff check adds a provenance tradeoff to this question. On the same source-built Base64 Mach-O (SHA-256 6d276076713030d87100f3a98e551b88de799193d5f9e1b57c1774d583aa77e9), I called only trace_feature("ABCDEFGHIJKLMNOPQRSTUVWXYZ") after opening the target, then exported the bundle. Ghidra 12.1.4 returned one observed source reference and its containing function; Kuna 1.986 returned no references with its existing explicit coverage limitation. Each exported bundle contained one trace_feature record with the result and relevant provider limitations but no linked or separate xrefs record. A fresh target-free server imported each bundle and recovered the trace record structurally unchanged.

In the same live sessions, a subsequent explicit xrefs call for the matched address returned Ghidra's same source and Kuna's empty list and added a separate Evidence record. Kuna's direct xref record carried provider-output digest, byte count and examined/excluded row counts. The existing trace record remained unchanged and did not link to that separate record. In separate fresh sessions I also called xrefs before trace_feature; neither trace linked the earlier xref record. A later independent query is a useful recovery while the target and provider remain available, but its separate record should not be represented as automatically proving the exact earlier source observation.

Source-derived context: the composed-workflow collector can link an AnalysisExecution.evidenceId and retain sourceEvidence when supplied, while the feature trace consumes the xref execution's result and explicit truncation observation. This check does not establish that every source must be retained separately or that the inline trace is invalid: it already records the provider and caveats. The design question is whether offline analysts need a bounded, attributable per-match source/coverage observation and how to retain it without unnecessary duplicated payload or conflating absence with unknown coverage. Target digest was unchanged; no target was executed. One fixture and two providers were tested, not all cache or failure states.

@morluto commented on GitHub (Oct 31, 2026): A real current-main (`60149282`) Evidence handoff check adds a provenance tradeoff to this question. On the same source-built Base64 Mach-O (SHA-256 `6d276076713030d87100f3a98e551b88de799193d5f9e1b57c1774d583aa77e9`), I called only `trace_feature("ABCDEFGHIJKLMNOPQRSTUVWXYZ")` after opening the target, then exported the bundle. Ghidra 12.1.4 returned one observed source reference and its containing function; Kuna 1.986 returned no references with its existing explicit coverage limitation. Each exported bundle contained one `trace_feature` record with the result and relevant provider limitations but no linked or separate `xrefs` record. A fresh target-free server imported each bundle and recovered the trace record structurally unchanged. In the same live sessions, a subsequent explicit `xrefs` call for the matched address returned Ghidra's same source and Kuna's empty list and added a separate Evidence record. Kuna's direct xref record carried provider-output digest, byte count and examined/excluded row counts. The existing trace record remained unchanged and did not link to that separate record. In separate fresh sessions I also called `xrefs` *before* `trace_feature`; neither trace linked the earlier xref record. A later independent query is a useful recovery while the target and provider remain available, but its separate record should not be represented as automatically proving the exact earlier source observation. Source-derived context: the composed-workflow collector can link an `AnalysisExecution.evidenceId` and retain `sourceEvidence` when supplied, while the feature trace consumes the xref execution's result and explicit truncation observation. This check does not establish that every source must be retained separately or that the inline trace is invalid: it already records the provider and caveats. The design question is whether offline analysts need a bounded, attributable per-match source/coverage observation and how to retain it without unnecessary duplicated payload or conflating absence with unknown coverage. Target digest was unchanged; no target was executed. One fixture and two providers were tested, not all cache or failure states.

bsakaguchi commented

2026-11-01 21:11:23 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

A second real-provider control shows this is not confined to Kuna or empty xref results. On current main 6e9ee97a69cb16c7150d8b550e705cba055ee4f0, I traced audit_initial in a self-built, unexecuted Mach-O, using real Ghidra 12.1.4/JDK 21 on macOS arm64 and SDK 2.3.1 stdio MCP with advertised compact output schemas.

A deliberately incomplete selected project has both _audit_initial and _audit_dispatch function entries but no decoded instructions. Its trace returned the matching procedure at 0x1000003c0 and an observed metadata reference at 0x1000080c0, with truncated: false and residual_unknowns: []. Its surrounding Evidence explicitly said that original project analysis completeness and settings are unknown.
A fresh Ghidra import of the same binary additionally returned the direct call site 0x1000003e8 in _audit_dispatch, and independent disassembly showed that call. Direct xrefs reproduced the same difference. These establish one missing reference in the deliberately incomplete project, not a general statement of exhaustive reference discovery for either session.
On the same target, real Kuna 1.986 returned the call site and an explicit unavailable owner with an owner residual, exercising the merged #8368 behavior in both full and compact advertised schemas. Its Evidence separately says reference discovery may omit edges. A no-match control returned no matches and no residual for each session; this is not proof the queried text is absent from an artifact.
Each trace and no-match record survived export and reimport into a fresh target-free MCP server with equal normalized results and limitations. Target, original project file hashes, and exported bundle hashes remained unchanged.

This extends the original coverage design question to nonempty selected-project references with known incompleteness. truncated correctly does not assert a provider cap, and an empty residual list is not documented as a completeness guarantee. If adding structured coverage, derive it from provider-owned observation scope and keep reference discovery, project history, explicit truncation, and per-source owner resolution distinct; avoid inferring it from limitations prose or treating #8391's caller/callee change as a fix for direct xrefs.

A separate real CLI rea trace ... --format json control reproduced the same fresh Ghidra, selected Ghidra, and Kuna reference/owner outcomes and selected-project Evidence caveat. The target hash was unchanged. The CLI control did not independently hash the selected project tree; the MCP runs above did.

@morluto commented on GitHub (Oct 31, 2026): A second real-provider control shows this is not confined to Kuna or empty xref results. On current main `6e9ee97a69cb16c7150d8b550e705cba055ee4f0`, I traced `audit_initial` in a self-built, unexecuted Mach-O, using real Ghidra 12.1.4/JDK 21 on macOS arm64 and SDK 2.3.1 stdio MCP with advertised compact output schemas. - A deliberately incomplete selected project has both `_audit_initial` and `_audit_dispatch` function entries but no decoded instructions. Its trace returned the matching procedure at `0x1000003c0` and an observed metadata reference at `0x1000080c0`, with `truncated: false` and `residual_unknowns: []`. Its surrounding Evidence explicitly said that original project analysis completeness and settings are unknown. - A fresh Ghidra import of the same binary additionally returned the direct call site `0x1000003e8` in `_audit_dispatch`, and independent disassembly showed that call. Direct `xrefs` reproduced the same difference. These establish one missing reference in the deliberately incomplete project, not a general statement of exhaustive reference discovery for either session. - On the same target, real Kuna 1.986 returned the call site and an explicit unavailable owner with an owner residual, exercising the merged #8368 behavior in both full and compact advertised schemas. Its Evidence separately says reference discovery may omit edges. A no-match control returned no matches and no residual for each session; this is not proof the queried text is absent from an artifact. - Each trace and no-match record survived export and reimport into a fresh target-free MCP server with equal normalized results and limitations. Target, original project file hashes, and exported bundle hashes remained unchanged. This extends the original coverage design question to **nonempty** selected-project references with known incompleteness. `truncated` correctly does not assert a provider cap, and an empty residual list is not documented as a completeness guarantee. If adding structured coverage, derive it from provider-owned observation scope and keep reference discovery, project history, explicit truncation, and per-source owner resolution distinct; avoid inferring it from limitations prose or treating #8391's caller/callee change as a fix for direct `xrefs`. A separate real CLI `rea trace ... --format json` control reproduced the same fresh Ghidra, selected Ghidra, and Kuna reference/owner outcomes and selected-project Evidence caveat. The target hash was unchanged. The CLI control did not independently hash the selected project tree; the MCP runs above did.

bsakaguchi referenced this issue

2026-11-01 21:47:31 +00:00

[PR #6690] test(ghidra): verify installation metadata budget through doctor #7390

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 09:20:06 +00:00