morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8219] [Design] Link native API boundary types to inspectable type identities #6493

New issue

Closed

opened 2026-11-01 21:11:38 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-11-01 21:11:38 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8219

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Can an agent move from a recovered function parameter or return type to the correct inspectable database type without guessing from a display name, while keeping inferred decompiler types and database type identities distinct?

Context and evidence

On macOS arm64, Ghidra 12.1.4 and REA 53a4ab78451a0654e9c3beb3d43a5e371a288fcc, a headless script created two different 4-byte structures named Record in a self-owned project: /audit/wire/Record with wire_id, and /audit/memory/Record with memory_flags. It assigned a pointer to the wire type as an existing function parameter and the memory type to a defined global data object. These are caller-authored project facts, not claims about original source types.

In an REA MCP session using the read-only project copy, procedure_info and inspect_native_api reported the parameter as Record *; the latter retained signature_source: user_defined, high-function provenance and the type-lock observation, but no exact category identity. inspect_native_data_type for Record or /Record returned unavailable, as documented for exact lookup. Both full structure paths succeeded with their distinct fields. The known global address successfully returned /audit/memory/Record, illustrating an existing discovery path that does not identify the function parameter's type.

A separate read-only Ghidra headless script inspected the same decompiler HighFunction parameter: display Record *, decompiler type path /audit/wire/Record *, pointee path /audit/wire/Record, locked true. The database manager separately contained pointer type /audit/wire/Record *64. Through REA, lookup of the high-function pointer path returned unavailable, whereas the exact saved pointer path returned its pointee /audit/wire/Record. Thus attaching the high-function display or path as if it were always a lookup-ready database ID would be incorrect, even though the underlying pointee can be established on this fixture.

REA closed successfully; the source-project regular-file digests and target digest were unchanged by its session. The target was not executed. One Ghidra fixture is not evidence that every decompiler type is a persistent database type or that an agent would choose the wrong one.

Expected interaction with targets

Read the selected project's existing analysis and typed observations; no new edit or target execution is proposed.

Boundary details (optional)

Preserve the exact function/parameter ordinal, source and type-lock facts, pointer layers, category and archive identity where the producer establishes them. Distinguish an ephemeral decompiler type from an exact database entry, keep absent or ambiguous identity unknown, and do not infer that caller-applied types describe original source or runtime layouts. Never choose one of multiple same-named types by display spelling.

Options and tradeoffs

Consider a bounded type-identity link on the existing native API boundary or a follow-up focused on one selected function parameter, returning lookup-ready database references only when verified. Separately evaluate a paged type search for cases without a known function or typed address; enumeration alone does not solve the parameter-to-type association. Reuse inspect_native_data_type for layouts rather than duplicating all fields in every API result.

Verification available so far

I have a prototype or observed results

Success criteria and open questions

Exercise duplicate names/categories, database-backed and decompiler-only types, pointer/typedef/array layers, missing types, archives, renamed types, selected-project state, and recursion. Validate source identity and SDK schemas; test real provider and unavailable providers. Ensure bounded output and no automatic claims of source-level type authority. A controlled agent trace would be needed to measure discoverability.

References

Existing exact-type contract #492; related analyst-hypothesis question #8166 and Ghidra project continuity #8165. The bridge currently uses DataType.getDisplayName() for boundary data_type in ReaGhidraBridge.java.

Imported dSYM type handoff

On Ghidra 12.1.4 with an explicitly selected disposable project created from an owned arm64 Mach-O and its adjacent dSYM, real REA analyze_function reported the parameter of audit_record as Sample *, with signature_source: imported, a type-lock observation, and ghidra-high-function provenance. Its pseudocode named fields count and scale. The exposed display spelling did not include an exact database identity; inspect_native_data_type for Sample, /Sample, and /DWARF/fixture.c/Sample returned unavailable.

A separate read-only headless inspection of a copy of the project found the saved struct's actual category path, which encoded the full compilation source path with escaped slashes. Supplying that exact identity to REA returned an 8-byte structure with count at offset 0 and scale at offset 4. Thus this was a lookup-identity/discoverability gap, not absence of the type. The exact path depended on this fixture's source path; it is not a path the agent can safely infer from the display string. Ghidra's import reported unrecoverable frame/register DWARF expressions, so not all debug facts were established. REA's initial project sessions in both profiles left all ten original project-file digests and the executable unchanged; the target SHA-256 was c244f7d7d9fa9b7f4cfcb1f69d69cc1f83ad98fe38b0293b223b3ca7d9076363, and it was not executed. This extends the earlier user-authored same-name control to an imported-debug case; it does not imply all decompiler types have resolvable database identities. Related dSYM selection: #1193.

Existing value-flow path on the imported dSYM fixture

The previous control has an important existing workaround: the complete analyze_function dossier's native_value_flow.parameters[0].data_type reported the HighFunction pointer path, even though native_api.parameters[0].data_type contained only Sample *. In fresh real REA/Ghidra sessions in both compact and full MCP profiles, I passed that value-flow path unchanged to inspect_native_data_type. It resolved as a pointer and supplied referenced_type; querying that returned the Sample structure and both count and scale fields. No external path discovery was needed for this particular chain, and the target SHA-256 remained unchanged.

This corrects any implication that the complete dossier offered no usable route on this fixture. It does not turn every HighFunction path into a lookup-ready database ID: the earlier user-authored same-name control above had a HighFunction pointer path that exact lookup could not resolve. The agent also needs the native-value-flow facet to be available, and a complete dossier can be larger than the API-boundary question requires. A design should preserve the distinction between the observed HighFunction type and a verified inspectable identity; where no link can be established, report that explicitly rather than guessing or claiming the type is absent. No autonomous-agent or latency measurement was made.

Ghidra producer ownership control

On a read-only copy of the selected dSYM project, a headless Ghidra 12.1.4 script inspected the actual HighFunction parameter type and its pointee. The decompiler's Sample * was a PointerDataType associated with the program manager but manager.contains(type) was false. Lookup of its path returned a different database pointer object that Ghidra considered equivalent. Its pointee, by contrast, was a StructureDB the manager contained, and exact lookup returned that same object. This explains why the previous value-flow path happened to resolve without establishing that the decompiler pointer itself was a persisted database object.

For an explicit negative control, the script constructed an unattached 12-byte structure with the same category path and name as the 8-byte project structure. Exact path lookup returned the existing project object, but isEquivalent returned false and the manager did not contain the constructed object. This is an intentionally synthetic collision, not evidence of a natural decompiler failure. It demonstrates that path equality and a non-null lookup alone cannot establish that a decompiler type is the database object. Conversely, this single positive isEquivalent result does not establish a general equivalence or provenance rule. A production link should distinguish a manager-owned identity from any independently justified equivalent-type relationship, and leave unresolved cases unknown; it must not call Ghidra's resolve if that would alter project data. Target SHA-256 remained c244f7d7d9fa9b7f4cfcb1f69d69cc1f83ad98fe38b0293b223b3ca7d9076363 and the target was not executed.

Bounded view and compact catalog

On the imported-dSYM control above, real Ghidra MCP in compact and full input/output profiles returned an analyze_function summary whose retained parent could be queried with inspect_analysis_view facet value_flow_parameters. The one-row page exposed the same observed HighFunction path; passing it to inspect_native_data_type returned a pointer and then, via referenced_type, the structure and both fields. This avoids delivering the complete dossier for that workflow, though it does not avoid constructing or retaining it. No token or performance measurement was made. The earlier non-resolving HighFunction pointer control and the synthetic same-path collision still apply.

The compact advertised input schema did not enumerate this facet, and the existing description mentioned high p-code without naming value_flow_parameters. Draft #8782 names the usable bounded view and warns about lookup/identity; it does not supply a verified type identity or resolve this issue.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8219 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Can an agent move from a recovered function parameter or return type to the correct inspectable database type without guessing from a display name, while keeping inferred decompiler types and database type identities distinct? ### Context and evidence On macOS arm64, Ghidra 12.1.4 and REA `53a4ab78451a0654e9c3beb3d43a5e371a288fcc`, a headless script created two different 4-byte structures named `Record` in a self-owned project: `/audit/wire/Record` with `wire_id`, and `/audit/memory/Record` with `memory_flags`. It assigned a pointer to the wire type as an existing function parameter and the memory type to a defined global data object. These are caller-authored project facts, not claims about original source types. In an REA MCP session using the read-only project copy, `procedure_info` and `inspect_native_api` reported the parameter as `Record *`; the latter retained `signature_source: user_defined`, high-function provenance and the type-lock observation, but no exact category identity. `inspect_native_data_type` for `Record` or `/Record` returned unavailable, as documented for exact lookup. Both full structure paths succeeded with their distinct fields. The known global address successfully returned `/audit/memory/Record`, illustrating an existing discovery path that does **not** identify the function parameter's type. A separate read-only Ghidra headless script inspected the same decompiler `HighFunction` parameter: display `Record *`, decompiler type path `/audit/wire/Record *`, pointee path `/audit/wire/Record`, locked true. The database manager separately contained pointer type `/audit/wire/Record *64`. Through REA, lookup of the high-function pointer path returned unavailable, whereas the exact saved pointer path returned its pointee `/audit/wire/Record`. Thus attaching the high-function display or path as if it were always a lookup-ready database ID would be incorrect, even though the underlying pointee can be established on this fixture. REA closed successfully; the source-project regular-file digests and target digest were unchanged by its session. The target was not executed. One Ghidra fixture is not evidence that every decompiler type is a persistent database type or that an agent would choose the wrong one. ### Expected interaction with targets Read the selected project's existing analysis and typed observations; no new edit or target execution is proposed. ### Boundary details (optional) Preserve the exact function/parameter ordinal, source and type-lock facts, pointer layers, category and archive identity where the producer establishes them. Distinguish an ephemeral decompiler type from an exact database entry, keep absent or ambiguous identity unknown, and do not infer that caller-applied types describe original source or runtime layouts. Never choose one of multiple same-named types by display spelling. ### Options and tradeoffs Consider a bounded type-identity link on the existing native API boundary or a follow-up focused on one selected function parameter, returning lookup-ready database references only when verified. Separately evaluate a paged type search for cases without a known function or typed address; enumeration alone does not solve the parameter-to-type association. Reuse `inspect_native_data_type` for layouts rather than duplicating all fields in every API result. ### Verification available so far I have a prototype or observed results ### Success criteria and open questions Exercise duplicate names/categories, database-backed and decompiler-only types, pointer/typedef/array layers, missing types, archives, renamed types, selected-project state, and recursion. Validate source identity and SDK schemas; test real provider and unavailable providers. Ensure bounded output and no automatic claims of source-level type authority. A controlled agent trace would be needed to measure discoverability. ### References Existing exact-type contract #492; related analyst-hypothesis question #8166 and Ghidra project continuity #8165. The bridge currently uses `DataType.getDisplayName()` for boundary `data_type` in [`ReaGhidraBridge.java`](https://github.com/morluto/rea/blob/53a4ab78451a0654e9c3beb3d43a5e371a288fcc/bridge/ghidra/ReaGhidraBridge.java). ### Imported dSYM type handoff On Ghidra 12.1.4 with an explicitly selected disposable project created from an owned arm64 Mach-O and its adjacent dSYM, real REA `analyze_function` reported the parameter of `audit_record` as `Sample *`, with `signature_source: imported`, a type-lock observation, and `ghidra-high-function` provenance. Its pseudocode named fields `count` and `scale`. The exposed display spelling did not include an exact database identity; `inspect_native_data_type` for `Sample`, `/Sample`, and `/DWARF/fixture.c/Sample` returned unavailable. A separate read-only headless inspection of a copy of the project found the saved struct's actual category path, which encoded the full compilation source path with escaped slashes. Supplying that exact identity to REA returned an 8-byte structure with `count` at offset 0 and `scale` at offset 4. Thus this was a lookup-identity/discoverability gap, not absence of the type. The exact path depended on this fixture's source path; it is not a path the agent can safely infer from the display string. Ghidra's import reported unrecoverable frame/register DWARF expressions, so not all debug facts were established. REA's initial project sessions in both profiles left all ten original project-file digests and the executable unchanged; the target SHA-256 was `c244f7d7d9fa9b7f4cfcb1f69d69cc1f83ad98fe38b0293b223b3ca7d9076363`, and it was not executed. This extends the earlier user-authored same-name control to an imported-debug case; it does not imply all decompiler types have resolvable database identities. Related dSYM selection: #1193. ### Existing value-flow path on the imported dSYM fixture The previous control has an important existing workaround: the complete `analyze_function` dossier's `native_value_flow.parameters[0].data_type` reported the HighFunction pointer path, even though `native_api.parameters[0].data_type` contained only `Sample *`. In fresh real REA/Ghidra sessions in both compact and full MCP profiles, I passed that value-flow path unchanged to `inspect_native_data_type`. It resolved as a pointer and supplied `referenced_type`; querying that returned the `Sample` structure and both `count` and `scale` fields. No external path discovery was needed for this particular chain, and the target SHA-256 remained unchanged. This corrects any implication that the complete dossier offered no usable route on this fixture. It does not turn every HighFunction path into a lookup-ready database ID: the earlier user-authored same-name control above had a HighFunction pointer path that exact lookup could not resolve. The agent also needs the native-value-flow facet to be available, and a complete dossier can be larger than the API-boundary question requires. A design should preserve the distinction between the observed HighFunction type and a verified inspectable identity; where no link can be established, report that explicitly rather than guessing or claiming the type is absent. No autonomous-agent or latency measurement was made. ### Ghidra producer ownership control On a read-only copy of the selected dSYM project, a headless Ghidra 12.1.4 script inspected the actual `HighFunction` parameter type and its pointee. The decompiler's `Sample *` was a `PointerDataType` associated with the program manager but `manager.contains(type)` was false. Lookup of its path returned a different database pointer object that Ghidra considered equivalent. Its pointee, by contrast, was a `StructureDB` the manager contained, and exact lookup returned that same object. This explains why the previous value-flow path happened to resolve without establishing that the decompiler pointer itself was a persisted database object. For an explicit negative control, the script constructed an unattached 12-byte structure with the same category path and name as the 8-byte project structure. Exact path lookup returned the existing project object, but `isEquivalent` returned false and the manager did not contain the constructed object. This is an intentionally synthetic collision, not evidence of a natural decompiler failure. It demonstrates that path equality and a non-null lookup alone cannot establish that a decompiler type is the database object. Conversely, this single positive `isEquivalent` result does not establish a general equivalence or provenance rule. A production link should distinguish a manager-owned identity from any independently justified equivalent-type relationship, and leave unresolved cases unknown; it must not call Ghidra's `resolve` if that would alter project data. Target SHA-256 remained `c244f7d7d9fa9b7f4cfcb1f69d69cc1f83ad98fe38b0293b223b3ca7d9076363` and the target was not executed. ### Bounded view and compact catalog On the imported-dSYM control above, real Ghidra MCP in compact and full input/output profiles returned an `analyze_function` summary whose retained parent could be queried with `inspect_analysis_view` facet `value_flow_parameters`. The one-row page exposed the same observed HighFunction path; passing it to `inspect_native_data_type` returned a pointer and then, via `referenced_type`, the structure and both fields. This avoids delivering the complete dossier for that workflow, though it does not avoid constructing or retaining it. No token or performance measurement was made. The earlier non-resolving HighFunction pointer control and the synthetic same-path collision still apply. The compact advertised input schema did not enumerate this facet, and the existing description mentioned high p-code without naming `value_flow_parameters`. Draft #8782 names the usable bounded view and warns about lookup/identity; it does not supply a verified type identity or resolve this issue.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:38 +00:00

bsakaguchi commented

2026-11-01 21:11:39 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

A related but separate component case is in #8224: a real packed Ghidra structure exposed bit-field component type_id: /uint:3, but an exact database query for that returned path was unavailable. The headless Ghidra control reported the field's base type as /uint, which REA could inspect separately. Union, enum and array category paths in the same project did resolve. This reinforces distinguishing ephemeral/component type representations from inspectable database identities; it does not assume the HighFunction pointer-path issue here has the same cause.

@morluto commented on GitHub (Oct 30, 2026): A related but separate component case is in #8224: a real packed Ghidra structure exposed bit-field component `type_id: /uint:3`, but an exact database query for that returned path was unavailable. The headless Ghidra control reported the field's base type as `/uint`, which REA could inspect separately. Union, enum and array category paths in the same project did resolve. This reinforces distinguishing ephemeral/component type representations from inspectable database identities; it does not assume the HighFunction pointer-path issue here has the same cause.

bsakaguchi commented

2026-11-01 21:11:39 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 12:24:56 +00:00

bsakaguchi closed this issue

2026-11-04 21:27:39 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6493

No description provided.

Rows
Columns