morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8245] [Design] Expose loaded and metadata provenance for string matches #6512

New issue

Open

opened 2026-11-01 21:11:48 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-11-01 21:11:48 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8245

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

How should an analyst or agent distinguish loaded-program strings from defined strings in non-loaded metadata when using search_strings or trace_feature, without dropping either kind of evidence?

Context and evidence

On REA 53a4ab78451a0654e9c3beb3d43a5e371a288fcc with Ghidra 12.1.4, a self-built, unexecuted x86-64 ELF has a defined UTF-16LE WIDE at 0x20017d and ELF symbol names in a separate non-loaded .strtab. search_strings({pattern:"WIDE"}) returned the defined value plus .strtab:0x43 / wide_ascii and .strtab:0x4e / wide_japanese. The address spaces are preserved, but the rows have only address and value, without a loaded/non-loaded field. trace_feature({query:"wide_ascii"}) returned .strtab:0x43 as a string match with no references. Its result also preserves the qualified address; it does not explicitly claim runtime use.

In the same session, Ghidra search_bytes reported .strtab as skipped_not_loaded; LLVM readelf independently showed .strtab outside the load segments. A known appended file-only marker was also absent from initialized-memory search and appeared in Ghidra's skipped non-loaded unallocated_0 block. The executable digest was unchanged. This is one Ghidra/ELF fixture, not a universal characterization of metadata or an observed agent mistake.

Expected interaction with targets

Read an existing local binary and query the selected static analysis session; no target execution or mutation.

Boundary details

Non-loaded metadata may be valuable evidence and should not be silently removed. A loaded block is not proof of runtime use; overlays, source-only data, address spaces, missing mappings and differing provider capabilities must retain their distinctions. Do not infer loaded status from address spelling or assume every provider can report it.

Options and tradeoffs

Consider bounded selection or optional row-level source/mapping facts from an existing provider-owned observation, and make it discoverable how to inspect the mapping when absent. Compare result size and latency against the existing inspect_native_load_image and search_bytes coverage; preserve unknown when unavailable. Avoid duplicating the raw provider API.

Verification available so far

I have a prototype or observed results.

Success criteria and open questions

Exercise loaded and non-loaded matches, overlays, absent or ambiguous mappings and other providers. Verify CLI and MCP output and the derived trace_feature handoff, including evidence provenance. A controlled agent study would be needed to assess discoverability or selection impact.

References

Related #8218 covers strings absent from inventories and #8184 covers trace reference coverage. #8233 concerns a separate loaded-overlay search classification.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8245 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed How should an analyst or agent distinguish loaded-program strings from defined strings in non-loaded metadata when using `search_strings` or `trace_feature`, without dropping either kind of evidence? ### Context and evidence On REA `53a4ab78451a0654e9c3beb3d43a5e371a288fcc` with Ghidra 12.1.4, a self-built, unexecuted x86-64 ELF has a defined UTF-16LE `WIDE` at `0x20017d` and ELF symbol names in a separate non-loaded `.strtab`. `search_strings({pattern:"WIDE"})` returned the defined value plus `.strtab:0x43` / `wide_ascii` and `.strtab:0x4e` / `wide_japanese`. The address spaces are preserved, but the rows have only `address` and `value`, without a loaded/non-loaded field. `trace_feature({query:"wide_ascii"})` returned `.strtab:0x43` as a string match with no references. Its result also preserves the qualified address; it does not explicitly claim runtime use. In the same session, Ghidra `search_bytes` reported `.strtab` as `skipped_not_loaded`; LLVM `readelf` independently showed `.strtab` outside the load segments. A known appended file-only marker was also absent from initialized-memory search and appeared in Ghidra's skipped non-loaded `unallocated_0` block. The executable digest was unchanged. This is one Ghidra/ELF fixture, not a universal characterization of metadata or an observed agent mistake. ### Expected interaction with targets Read an existing local binary and query the selected static analysis session; no target execution or mutation. ### Boundary details Non-loaded metadata may be valuable evidence and should not be silently removed. A loaded block is not proof of runtime use; overlays, source-only data, address spaces, missing mappings and differing provider capabilities must retain their distinctions. Do not infer loaded status from address spelling or assume every provider can report it. ### Options and tradeoffs Consider bounded selection or optional row-level source/mapping facts from an existing provider-owned observation, and make it discoverable how to inspect the mapping when absent. Compare result size and latency against the existing `inspect_native_load_image` and `search_bytes` coverage; preserve unknown when unavailable. Avoid duplicating the raw provider API. ### Verification available so far I have a prototype or observed results. ### Success criteria and open questions Exercise loaded and non-loaded matches, overlays, absent or ambiguous mappings and other providers. Verify CLI and MCP output and the derived `trace_feature` handoff, including evidence provenance. A controlled agent study would be needed to assess discoverability or selection impact. ### References Related [#8218](https://github.com/morluto/rea/issues/8218) covers strings absent from inventories and [#8184](https://github.com/morluto/rea/issues/8184) covers trace reference coverage. [#8233](https://github.com/morluto/rea/issues/8233) concerns a separate loaded-overlay search classification.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:48 +00:00

bsakaguchi commented

2026-11-01 21:11:48 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi commented

2026-11-01 21:11:48 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

A second control shows why a loaded flag alone is not a semantic classification. On a Clang -fno-rtti, symbol-table-stripped arm64 Mach-O, real REA/Ghidra search_strings("Doubler") returned residual fragments; its load-image mapping placed a selected fragment in __LINKEDIT with loaded: true, and address_to_file_offset mapped it to retained file bytes. xrefs for that exact fragment returned no rows. REA/Kuna returned no matches in its scanned inventory. This does not prove the string is runtime-used or that other references do not exist; the source-built control and limits are recorded in #3624. The target was not executed and its digest stayed unchanged.

@morluto commented on GitHub (Oct 31, 2026): A second control shows why a loaded flag alone is not a semantic classification. On a Clang `-fno-rtti`, symbol-table-stripped arm64 Mach-O, real REA/Ghidra `search_strings("Doubler")` returned residual fragments; its load-image mapping placed a selected fragment in `__LINKEDIT` with `loaded: true`, and `address_to_file_offset` mapped it to retained file bytes. `xrefs` for that exact fragment returned no rows. REA/Kuna returned no matches in its scanned inventory. This does not prove the string is runtime-used or that other references do not exist; the source-built control and limits are recorded in #3624. The target was not executed and its digest stayed unchanged.

bsakaguchi referenced this issue

2026-11-01 21:47:25 +00:00

[PR #6512] [CLOSED] fix(pcap): report capture summaries unsupported off their verified hosts #7375

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6512

No description provided.

Rows
Columns