morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8240] [Design] Find the defined data object containing an address #6510

New issue

Open

opened 2026-11-01 21:11:46 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-11-01 21:11:46 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8240

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Should an analyst inspecting an interior address be able to find the containing defined data object and applicable component candidates, then use the existing type-layout tool, without assuming that a byte identifies one unique field?

Context and evidence

I used direct Ghidra 12.1.4 headless scripts to create a caller-authored seven-byte packed structure in a new disposable project. It has a tag, two bit fields sharing offset 1, a four-byte union at offset 2 with whole and bytes members, and an enum at offset 6. I defined this structure at 0x100004000 in the project. These are project-authored analysis facts, not source or runtime layout claims.

A separate read-only headless Ghidra inspection found the parent data object at offsets 0 through 6. At offset 1 its getComponentsContaining returned both bit fields; at offsets 2 and 3 it returned the containing union, and the union returned both member candidates. Offset 7 had no defined containing object.

Real REA/Ghidra MCP on the selected project returned the structure and its fields from inspect_native_data_type({"address":"0x100004000"}). The same tool returned unavailable for addresses at offsets 1, 2, 3 and 6 because it uses an exact defined-data start. inspect_native_instruction reported status: data for those interior addresses but did not return their parent data address or type identity. With the independently known parent address, the agent could inspect the whole structure and then query the exact union type; without it, the tested public path did not provide the parent. Offset 7 returned not-in-listing and no exact type. The exact-start behavior is consistent with the current implementation and documentation; this is a workflow question, not a claim that it promises containment.

The target digest and all nine selected project regular-file digests stayed unchanged during REA's session and the read-only control. The target was not executed. Initial import reported unrelated unrecoverable DWARF expressions; these manually authored types do not establish debug-info completeness.

Expected interaction with targets

Read an existing analysis program and its type database; no new data definition, target execution or source-project mutation is proposed. The direct scripts above wrote only the explicitly owned experimental project before REA opened it.

Boundary details (optional)

Preserve the requested address, containing data start/range, exact type/source/project identity, and byte offset. Return overlapping bit-field and union interpretations as distinct candidates rather than selecting one. Separate a component's representation from an exact inspectable database type (#8224); preserve unknown or unavailable results for missing types, padding, overlays, conflicting observations, recursive/dynamic layouts and providers lacking the capability. Do not infer original source types or runtime accesses.

Options and tradeoffs

Consider an explicit containing-data mode on the existing type inspection, a small address-context addition linked to existing exact inspect_native_data_type, or documented bounded composition if another existing tool can provide the parent. Avoid expanding every full type layout or inventing a Ghidra-only wrapper. Preserve today's exact lookup behavior unless an explicit selection requests containment; define bounded traversal/paging for recursive or large component sets.

Verification available so far

I have a prototype or observed results

Success criteria and open questions

Exercise object start, interior byte, component boundaries, union overlaps, packed bit fields, padding, arrays, recursive pointers, uninitialized and unmapped addresses, source archives and address spaces; check real Ghidra and unavailable providers, SDK schemas, retained Evidence and per-result coverage. Do not claim a single active union member or complete source layout. No CLI, Windows, large layout, agent-selection, or general performance test was performed.

References

Existing exact-type design #492; related #8219 for function-parameter type identity and #8224 for non-resolvable bit-field identities. REA revision 53a4ab78451a0654e9c3beb3d43a5e371a288fcc; Ghidra exact lookup source.

Byte-search handoff with a project-authored data object

A separate Ghidra 12.1.4 headless script added a five-byte initialized block and a caller-authored structure to a disposable project. The structure at 0x200001000 has a byte tag followed by a four-byte payload; its project-authored bytes are a1 b2 c3 d4 e5.

With REA 9ebbe67e05af08b567e215340da993c08dae378f and full-profile public MCP, search_bytes for c3 d4 over that block returned 0x200001002 with file_offset: null and no procedure. inspect_native_instruction at the hit returned status: data, but no containing object identity. Exact inspect_native_data_type at the hit returned unavailable; the same call at the independently known parent 0x200001000 returned the five-byte REA_ByteSearchRecord and its two fields. The executable SHA-256 remained ae0f2a349b22332113ab60ab4c78024eb18153c76bdeefb01c4b1d89a26c0003.

This demonstrates the search-to-data handoff, not a false claim by either existing exact-address tool. The bytes and type were created in the experimental project, and the null file offset must not be interpreted as a target-file mapping. This simple non-overlapping structure does not replace the union and bit-field controls above.

A subsequent read-only Ghidra headless check of that project reported the root getDefinedDataContaining object at 0x200001000, length 5, type path /REA_ByteSearchRecord, and its payload component at 0x200001001. In real REA MCP, exact inspect_native_data_type by /REA_ByteSearchRecord returned the structure, and by /dword returned the payload's scalar type. This establishes one viable handoff for this fixture: a bounded parent identity could lead to the existing exact type tool. It does not show that every Ghidra component type path is independently resolvable (see #8224), or that a single component is correct for union and bit-field cases. Preserve the requested address and candidate ambiguity, and explicitly distinguish parent discovery from deeper component selection.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8240 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Should an analyst inspecting an interior address be able to find the containing defined data object and applicable component candidates, then use the existing type-layout tool, without assuming that a byte identifies one unique field? ### Context and evidence I used direct Ghidra 12.1.4 headless scripts to create a caller-authored seven-byte packed structure in a new disposable project. It has a tag, two bit fields sharing offset 1, a four-byte union at offset 2 with `whole` and `bytes` members, and an enum at offset 6. I defined this structure at `0x100004000` in the project. These are project-authored analysis facts, not source or runtime layout claims. A separate read-only headless Ghidra inspection found the parent data object at offsets 0 through 6. At offset 1 its `getComponentsContaining` returned both bit fields; at offsets 2 and 3 it returned the containing union, and the union returned both member candidates. Offset 7 had no defined containing object. Real REA/Ghidra MCP on the selected project returned the structure and its fields from `inspect_native_data_type({"address":"0x100004000"})`. The same tool returned `unavailable` for addresses at offsets 1, 2, 3 and 6 because it uses an exact defined-data start. `inspect_native_instruction` reported `status: data` for those interior addresses but did not return their parent data address or type identity. With the independently known parent address, the agent could inspect the whole structure and then query the exact union type; without it, the tested public path did not provide the parent. Offset 7 returned `not-in-listing` and no exact type. The exact-start behavior is consistent with the current implementation and documentation; this is a workflow question, not a claim that it promises containment. The target digest and all nine selected project regular-file digests stayed unchanged during REA's session and the read-only control. The target was not executed. Initial import reported unrelated unrecoverable DWARF expressions; these manually authored types do not establish debug-info completeness. ### Expected interaction with targets Read an existing analysis program and its type database; no new data definition, target execution or source-project mutation is proposed. The direct scripts above wrote only the explicitly owned experimental project before REA opened it. ### Boundary details (optional) Preserve the requested address, containing data start/range, exact type/source/project identity, and byte offset. Return overlapping bit-field and union interpretations as distinct candidates rather than selecting one. Separate a component's representation from an exact inspectable database type (#8224); preserve unknown or unavailable results for missing types, padding, overlays, conflicting observations, recursive/dynamic layouts and providers lacking the capability. Do not infer original source types or runtime accesses. ### Options and tradeoffs Consider an explicit containing-data mode on the existing type inspection, a small address-context addition linked to existing exact `inspect_native_data_type`, or documented bounded composition if another existing tool can provide the parent. Avoid expanding every full type layout or inventing a Ghidra-only wrapper. Preserve today's exact lookup behavior unless an explicit selection requests containment; define bounded traversal/paging for recursive or large component sets. ### Verification available so far I have a prototype or observed results ### Success criteria and open questions Exercise object start, interior byte, component boundaries, union overlaps, packed bit fields, padding, arrays, recursive pointers, uninitialized and unmapped addresses, source archives and address spaces; check real Ghidra and unavailable providers, SDK schemas, retained Evidence and per-result coverage. Do not claim a single active union member or complete source layout. No CLI, Windows, large layout, agent-selection, or general performance test was performed. ### References Existing exact-type design #492; related #8219 for function-parameter type identity and #8224 for non-resolvable bit-field identities. REA revision `53a4ab78451a0654e9c3beb3d43a5e371a288fcc`; [Ghidra exact lookup source](https://github.com/morluto/rea/blob/53a4ab78451a0654e9c3beb3d43a5e371a288fcc/bridge/ghidra/ReaGhidraBridge.java#L1180-L1195). ### Byte-search handoff with a project-authored data object A separate Ghidra 12.1.4 headless script added a five-byte initialized block and a caller-authored structure to a disposable project. The structure at `0x200001000` has a byte `tag` followed by a four-byte `payload`; its project-authored bytes are `a1 b2 c3 d4 e5`. With REA `9ebbe67e05af08b567e215340da993c08dae378f` and full-profile public MCP, `search_bytes` for `c3 d4` over that block returned `0x200001002` with `file_offset: null` and no procedure. `inspect_native_instruction` at the hit returned `status: data`, but no containing object identity. Exact `inspect_native_data_type` at the hit returned `unavailable`; the same call at the independently known parent `0x200001000` returned the five-byte `REA_ByteSearchRecord` and its two fields. The executable SHA-256 remained `ae0f2a349b22332113ab60ab4c78024eb18153c76bdeefb01c4b1d89a26c0003`. This demonstrates the search-to-data handoff, not a false claim by either existing exact-address tool. The bytes and type were created in the experimental project, and the null file offset must not be interpreted as a target-file mapping. This simple non-overlapping structure does not replace the union and bit-field controls above. A subsequent read-only Ghidra headless check of that project reported the root `getDefinedDataContaining` object at `0x200001000`, length 5, type path `/REA_ByteSearchRecord`, and its `payload` component at `0x200001001`. In real REA MCP, exact `inspect_native_data_type` by `/REA_ByteSearchRecord` returned the structure, and by `/dword` returned the payload's scalar type. This establishes one viable handoff for this fixture: a bounded parent identity could lead to the existing exact type tool. It does not show that every Ghidra component type path is independently resolvable (see #8224), or that a single component is correct for union and bit-field cases. Preserve the requested address and candidate ambiguity, and explicitly distinguish parent discovery from deeper component selection.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:46 +00:00

bsakaguchi commented

2026-11-01 21:11:47 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

A TypeScript MCP SDK 2.3.1 control exercised this selected-project fixture against both REA's full and compact advertised output-schema profiles. For each profile the client listed the 100 tools in the selected four families and explicitly supplied each advertised tool definition to callTool for output validation. Ghidra returned the available parent type, unavailable interior type, data-status instruction, graph summary and retained graph page as successful structured results without SDK validation exceptions. The read-only project correctly returned a tool-level capability_unavailable for a name edit; SDK 2.3.1 skips output-schema validation for tool-error results, so that is not schema validation of the error.

An independent negative control supplied a deliberately incompatible client-side test schema for a successful binary_session call; the SDK rejected the returned structured result. The same incompatible test schema on a tool-level error did not trigger schema validation. This checks that validation was active in the selected path, not that REA's catalog or every client/error case is valid. Target and nine selected project file digests remained unchanged during the Ghidra sessions.

@morluto commented on GitHub (Oct 30, 2026): A TypeScript MCP SDK 2.3.1 control exercised this selected-project fixture against both REA's full and compact advertised output-schema profiles. For each profile the client listed the 100 tools in the selected four families and explicitly supplied each advertised tool definition to `callTool` for output validation. Ghidra returned the available parent type, unavailable interior type, data-status instruction, graph summary and retained graph page as successful structured results without SDK validation exceptions. The read-only project correctly returned a tool-level `capability_unavailable` for a name edit; SDK 2.3.1 skips output-schema validation for tool-error results, so that is not schema validation of the error. An independent negative control supplied a deliberately incompatible **client-side test schema** for a successful `binary_session` call; the SDK rejected the returned structured result. The same incompatible test schema on a tool-level error did not trigger schema validation. This checks that validation was active in the selected path, not that REA's catalog or every client/error case is valid. Target and nine selected project file digests remained unchanged during the Ghidra sessions.

bsakaguchi commented

2026-11-01 21:11:47 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi commented

2026-11-01 21:11:47 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Bounded manual recovery and CLI control

On REA b43c329 / Ghidra 12.1.4 with the same caller-authored packed-data project selected read-only, I tried a deliberately bounded fallback starting at interior address 0x100004003. address_name returned null; the full name inventory did contain two names at 0x100004000, but a nearby name alone does not establish its data extent. Exact inspect_native_data_type queries at offsets 3, 2 and 1 were unavailable; offset 0 returned /audit/layout/Packed, size 7, with a complete five-field layout.

Using that observed start and size as a candidate covering offset 3, its payload field at offsets 2–5 led to exact type /audit/layout/Payload. The union layout retained both whole and bytes candidates; querying /byte[4] separately reported four elements with stride 1, so offset 3 corresponds to byte-array index 1 if that union interpretation is selected. At offset 1 the parent layout retained both overlapping bit fields. The size check excludes offset 7 from this seven-byte object. These are analyst-composed database observations, not a claim about the active union member, original source type, runtime value, or a unique object for arbitrary addresses. A scan bound must be caller-selected; crossing unmapped regions, overlays, huge objects, truncated layouts and multiple candidates remain unresolved.

Real rea inspect-native-data-type CLI calls on this same selected project produced the same normalized unavailable interior, available parent and available union results as MCP; all exited 0, reflecting a typed unavailable result rather than a transport failure. Target and original project file hashes remained unchanged, and the target was not executed. This demonstrates an expensive bounded fallback when a plausible lower bound is already known; it does not remove the value of a provider-owned containing-data operation or justify an unbounded backwards scan.

@morluto commented on GitHub (Oct 31, 2026): ### Bounded manual recovery and CLI control On REA `b43c329` / Ghidra 12.1.4 with the same caller-authored packed-data project selected read-only, I tried a deliberately bounded fallback starting at interior address `0x100004003`. `address_name` returned null; the full name inventory did contain two names at `0x100004000`, but a nearby name alone does not establish its data extent. Exact `inspect_native_data_type` queries at offsets 3, 2 and 1 were unavailable; offset 0 returned `/audit/layout/Packed`, size 7, with a complete five-field layout. Using that observed start and size as a **candidate** covering offset 3, its `payload` field at offsets 2–5 led to exact type `/audit/layout/Payload`. The union layout retained both `whole` and `bytes` candidates; querying `/byte[4]` separately reported four elements with stride 1, so offset 3 corresponds to byte-array index 1 if that union interpretation is selected. At offset 1 the parent layout retained both overlapping bit fields. The size check excludes offset 7 from this seven-byte object. These are analyst-composed database observations, not a claim about the active union member, original source type, runtime value, or a unique object for arbitrary addresses. A scan bound must be caller-selected; crossing unmapped regions, overlays, huge objects, truncated layouts and multiple candidates remain unresolved. Real `rea inspect-native-data-type` CLI calls on this same selected project produced the same normalized unavailable interior, available parent and available union results as MCP; all exited 0, reflecting a typed unavailable result rather than a transport failure. Target and original project file hashes remained unchanged, and the target was not executed. This demonstrates an expensive bounded fallback when a plausible lower bound is already known; it does not remove the value of a provider-owned containing-data operation or justify an unbounded backwards scan.

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 11:23:19 +00:00

bsakaguchi referenced this issue

2026-11-03 13:38:49 +00:00

[PR #7013] [MERGED] fix(targets): keep original DOS header padding out of e_lfanew #7574