morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8460] [Design] Distinguish function correspondence from analysis metadata changes #6624

New issue

Open

opened 2026-11-01 21:12:45 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-11-01 21:12:45 +00:00

Owner

Originally created by @morluto on GitHub (Nov 1, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8460

Before submitting

I searched the open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Should a function comparison distinguish function correspondence from differences in the provider's names, prototype and local-variable annotations, and identify when a static change candidate comes from different analysis state rather than a changed artifact?

Context and evidence

Using public stdio MCP on REA main e18594b24754ab95ebd119311cc32a1d1ad6fa13, Node 24.22.1, Ghidra 12.1.4/JDK 21 and compact advertised schemas, I compared two selected-project analyses of the same self-built, unexecuted x86-64 Mach-O (SHA-256 16be6017e676a9bbaf3f346b659f8ae5c7f788a1184199b6dee8a5a8efbe833e). Before REA ran, I copied the project and used headless Ghidra to change one user-defined function prototype from int audit_variadic(int count, ...) to int audit_variadic(int count); the latter is intentionally inaccurate for the source function. No executable byte was changed. The target and both project trees remained unchanged during REA sessions.

I called analyze_function for the callee and its caller in each selected project, exported each session and imported both bundles into a fresh target-free MCP server. Original normalized dossiers matched after import. For the callee:

Both subjects report the same artifact SHA-256, entry 0x1000003d0 and name audit_variadic; function_match is matched by symbol.
The identity dimension nevertheless reports changed with no limitation. The signature differs; name and local descriptions are equal. Assembly and CFG are unchanged; pseudocode changes. The selected-project analysis-profile digests differ, and both source records retain the selected-project caveat.
find_changed_behavior preserves behavior_status: "unknown" and zero observed changes, but includes identity and pseudocode as changed static_candidate findings. It does not claim observed runtime behavior changed. The question is whether an agent can readily distinguish an analysis-annotation difference from a candidate artifact change using the comparison result itself.

The same comparison on #8343 head 84f66cc672f4bf7725e21bd3b5c73c2c5c4b35d0 produced the same statuses. Its structured parameter_list separately reflects the two supplied prototype states; it does not cause this comparison behavior. The caller comparison has unchanged identity but changed pseudocode. Both comparisons leave calls/references unknown because common provider coordinate commitments were not established. No autonomous-agent trial or runtime behavior was measured.

Source-derived: identityProjection currently hashes {name, signature, locals} as one identity dimension; function_match is a separate correspondence result. The opaque dimension digests and retained source Evidence preserve the inputs, but the dimension does not say which part changed or whether the selected analyses differed.

Expected interaction with targets

Read existing binaries and caller-selected project copies; launch existing Ghidra analysis. No target execution, new mutation authority, or automatic replay of annotations is proposed.

Boundary details (optional)

Keep exact source Evidence, project/profile identity and uncertainty. Identical executable bytes do not prove identical runtime behavior; different analysis profiles do not prove every difference was caused by an annotation. Do not suppress useful text or signature changes merely because artifacts match.

Options and tradeoffs

Consider clearer machine-readable labeling/limitations for the existing dimension, separating correspondence from observed metadata changes, or exposing selected field-level differences alongside provenance. Any contract change must account for strict schemas, historical Evidence, reconstruction verification, and static/runtime correlation. Avoid prescribing a new tool or changing behavior classifications without checking their consumers.

Verification available so far

I have observed results. This is one fixture, two deliberately authored selected projects and one provider/host; no other provider, natural artifact version change, or GUI workflow was tested.

Success criteria and open questions

An analyst can tell matched-function metadata changes from correspondence and from evidence of changed executable behavior, while preserving unknown cases and existing useful comparisons. Check same/different target digests, genuine function changes, renamed symbols, signature-only and local-only annotation changes, absent metadata, old Evidence, and both CLI/MCP schema profiles.

References

Related: #8314 (local-variable provenance), #8367 (same-record matching), #5523 (structured variadic prototype), #8358 (unmatched call operands). Source: identityProjection. Fixture/prototype control: https://github.com/morluto/rea/issues/8358#issuecomment-6555740037.

Maintainers: @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

Originally created by @morluto on GitHub (Nov 1, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8460 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Should a function comparison distinguish **function correspondence** from differences in the provider's names, prototype and local-variable annotations, and identify when a static change candidate comes from different analysis state rather than a changed artifact? ### Context and evidence Using public stdio MCP on REA main `e18594b24754ab95ebd119311cc32a1d1ad6fa13`, Node 24.22.1, Ghidra 12.1.4/JDK 21 and compact advertised schemas, I compared two selected-project analyses of the same self-built, unexecuted x86-64 Mach-O (SHA-256 `16be6017e676a9bbaf3f346b659f8ae5c7f788a1184199b6dee8a5a8efbe833e`). Before REA ran, I copied the project and used headless Ghidra to change one user-defined function prototype from `int audit_variadic(int count, ...)` to `int audit_variadic(int count)`; the latter is intentionally inaccurate for the source function. No executable byte was changed. The target and both project trees remained unchanged during REA sessions. I called `analyze_function` for the callee and its caller in each selected project, exported each session and imported both bundles into a fresh target-free MCP server. Original normalized dossiers matched after import. For the callee: - Both subjects report the same artifact SHA-256, entry `0x1000003d0` and name `audit_variadic`; `function_match` is `matched` by symbol. - The `identity` dimension nevertheless reports `changed` with no limitation. The signature differs; name and local descriptions are equal. Assembly and CFG are unchanged; pseudocode changes. The selected-project analysis-profile digests differ, and both source records retain the selected-project caveat. - `find_changed_behavior` preserves `behavior_status: "unknown"` and zero observed changes, but includes `identity` and `pseudocode` as changed `static_candidate` findings. It does not claim observed runtime behavior changed. The question is whether an agent can readily distinguish an analysis-annotation difference from a candidate artifact change using the comparison result itself. The same comparison on #8343 head `84f66cc672f4bf7725e21bd3b5c73c2c5c4b35d0` produced the same statuses. Its structured `parameter_list` separately reflects the two supplied prototype states; it does not cause this comparison behavior. The caller comparison has unchanged identity but changed pseudocode. Both comparisons leave calls/references unknown because common provider coordinate commitments were not established. No autonomous-agent trial or runtime behavior was measured. Source-derived: `identityProjection` currently hashes `{name, signature, locals}` as one `identity` dimension; `function_match` is a separate correspondence result. The opaque dimension digests and retained source Evidence preserve the inputs, but the dimension does not say which part changed or whether the selected analyses differed. ### Expected interaction with targets Read existing binaries and caller-selected project copies; launch existing Ghidra analysis. No target execution, new mutation authority, or automatic replay of annotations is proposed. ### Boundary details (optional) Keep exact source Evidence, project/profile identity and uncertainty. Identical executable bytes do not prove identical runtime behavior; different analysis profiles do not prove every difference was caused by an annotation. Do not suppress useful text or signature changes merely because artifacts match. ### Options and tradeoffs Consider clearer machine-readable labeling/limitations for the existing dimension, separating correspondence from observed metadata changes, or exposing selected field-level differences alongside provenance. Any contract change must account for strict schemas, historical Evidence, reconstruction verification, and static/runtime correlation. Avoid prescribing a new tool or changing behavior classifications without checking their consumers. ### Verification available so far I have observed results. This is one fixture, two deliberately authored selected projects and one provider/host; no other provider, natural artifact version change, or GUI workflow was tested. ### Success criteria and open questions An analyst can tell matched-function metadata changes from correspondence and from evidence of changed executable behavior, while preserving unknown cases and existing useful comparisons. Check same/different target digests, genuine function changes, renamed symbols, signature-only and local-only annotation changes, absent metadata, old Evidence, and both CLI/MCP schema profiles. ### References Related: #8314 (local-variable provenance), #8367 (same-record matching), #5523 (structured variadic prototype), #8358 (unmatched call operands). Source: [`identityProjection`](https://github.com/morluto/rea/blob/e18594b24754ab95ebd119311cc32a1d1ad6fa13/src/domain/functionComparisonNormalization.ts). Fixture/prototype control: https://github.com/morluto/rea/issues/8358#issuecomment-6555740037. Maintainers: @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:12:45 +00:00

bsakaguchi commented

2026-11-01 21:12:46 +00:00

Author

Owner

@morluto commented on GitHub (Nov 1, 2026):

A public, target-free recovery path exists for this particular comparison. I reran the real selected-project Ghidra sessions, exported both bundles, and imported them into a fresh stdio MCP server with no open binary. On main e18594b24754ab95ebd119311cc32a1d1ad6fa13, I followed each of the callee comparison's two top-level evidence_links with inspect_analysis_view({source:{kind:"retained-evidence",evidence_id},view:{kind:"native",facet:"procedure"}}). The two returned procedure records preserved the differing displayed signatures and identical name/locals, so this manual comparison identifies the changed part of the opaque identity dimension. A separate native_api facet returned signature_source: "user_defined" for both; on this main revision it did not contain a structured variadic field.

The same target-free handoff on #8343 head 84f66cc672f4bf7725e21bd3b5c73c2c5c4b35d0 additionally returned parameter_list signature/true and signature/false from the two native_api facets. The second fact is the deliberately inaccurate analyst-supplied prototype, not source truth. Each view's item was checked against its corresponding original dossier; target and both selected project trees stayed unchanged in the REA sessions. Both runs used pinned Node 24.22.1, Ghidra 12.1.4, compact SDK stdio schemas and the same unexecuted fixture. The fallback requires following both source IDs and comparing their facets; it doesn't establish why arbitrary metadata differs or whether an actual agent would find the route. No CLI or other-provider claim.

@morluto commented on GitHub (Nov 1, 2026): A public, target-free recovery path exists for this particular comparison. I reran the real selected-project Ghidra sessions, exported both bundles, and imported them into a fresh stdio MCP server with no open binary. On main `e18594b24754ab95ebd119311cc32a1d1ad6fa13`, I followed each of the callee comparison's two top-level `evidence_links` with `inspect_analysis_view({source:{kind:"retained-evidence",evidence_id},view:{kind:"native",facet:"procedure"}})`. The two returned procedure records preserved the differing displayed signatures and identical name/locals, so this manual comparison identifies the changed part of the opaque identity dimension. A separate `native_api` facet returned `signature_source: "user_defined"` for both; on this main revision it did not contain a structured variadic field. The same target-free handoff on #8343 head `84f66cc672f4bf7725e21bd3b5c73c2c5c4b35d0` additionally returned `parameter_list` signature/true and signature/false from the two `native_api` facets. The second fact is the deliberately inaccurate analyst-supplied prototype, not source truth. Each view's item was checked against its corresponding original dossier; target and both selected project trees stayed unchanged in the REA sessions. Both runs used pinned Node 24.22.1, Ghidra 12.1.4, compact SDK stdio schemas and the same unexecuted fixture. The fallback requires following both source IDs and comparing their facets; it doesn't establish why arbitrary metadata differs or whether an actual agent would find the route. No CLI or other-provider claim.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6624

No description provided.

Rows
Columns