[GH-ISSUE #4395] [Feature] Generate unique masked byte signatures and relocate them in another build #1749

New issue

Open

opened 2026-10-16 21:09:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:09:48 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4395

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Native analysis or decompilation

Current limitation

REA can search for a masked pattern but cannot produce one. search_bytes (src/contracts/hopperMemoryContracts.ts, Ghidra only) finds hex patterns with ?? wildcards and an optional bit mask and reports coverage, and docs/native-investigation.md ("Locate a byte signature") documents that use. The signature itself must come from the agent.

Building one by hand is error-prone: the agent has to read instruction bytes, decide which bytes are relocations, absolute addresses, or PC-relative displacements, wildcard them, grow the pattern until it is unique, and then check uniqueness across every initialized block. inspect_native_instruction exposes bytes and operand kinds for one address but not per-operand byte offsets, read_function_instructions returns text without bytes, and nothing combines them.

Cross-build correspondence is also left to the agent. compare_functions compares two functions the caller has already paired (symbol or explicit match only), compare_application_versions as a tool handles JavaScript application graphs, and list_procedures exposes no fingerprints. Address-normalized comparison (#1947, PR #2124) and evidence-supported callee matching (#3506) make a known pair comparable but do not locate the counterpart.

Use cases

Patch diffing: after a vendor update, find the new address of each function named in the previous build's analysis before comparing behavior.
Malware variant analysis: re-locate known routines in a new sample to carry names and comments forward.
Re-verifying earlier findings: confirm that the code an earlier report cited still exists, unchanged or changed, in the current build.

Desired outcome and success criteria

Add a read-only way to derive the shortest unique masked byte signature for a procedure or address in an analyzed binary, prove its uniqueness with search coverage, and relocate it in a second build of the same program, so names, annotations, and findings can be carried across versions whose addresses shift.

Given an opened binary and a procedure or address, a tool returns a byte pattern and mask, the anchor offset, and for each wildcarded byte the reason (relocation, absolute address operand, PC-relative displacement, or an immediate that resolves to an address). Small non-address immediates stay concrete, consistent with the address-normalization rule discussed in #1947.
Uniqueness is backed by a complete search over the same eligible blocks and cites that Evidence. If coverage is incomplete (uninitialized or not-loaded blocks skipped), the result says uniqueness is unknown rather than unique.
If no unique signature exists within a caller-supplied maximum length, the result is a typed ambiguous outcome listing competing matches, not a non-unique pattern presented as unique.
Relocating a signature in a second opened build reports unique (with containing procedure), ambiguous (all candidates), or absent (scoped to the coverage searched), and never applies names automatically; renaming stays an explicit set_addresses_names call citing both Evidence sets.
Unsupported providers return typed capability unavailability. The tool never executes or modifies the target.
A fixture pair with known unchanged and changed functions (for example the Wallace v1/v2 version-tracking exercise referenced in #1947) relocates unchanged functions uniquely and returns ambiguous or absent for changed ones, never a wrong unique match.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Static analysis of already-opened images only; Ghidra is the natural first provider because REA's bridge already returns instruction bytes and the relocation table. Non-goals: live process memory scanning, patching bytes, or applying signatures to running processes; fuzzy or similarity-based function matching (a separate problem). Documentation examples should use version porting and variant analysis.

Verification available so far

I can suggest a sample or manual check

Verification details (optional)

No response

Alternatives considered

Hand-built patterns with search_bytes: possible today but unreliable for the reasons above, and uniqueness is easy to over-claim when coverage is partial.
Folding matching into compare_functions: conflicts with its explicit-pairing contract.
Exposing Rizin zignatures through execute_rizin_command: untyped, mutating-capable surface, no coverage-backed uniqueness Evidence.
External tools (IDA signature-maker plugins, Ghidra Version Tracking, BinDiff): useful, but their results do not arrive as REA Evidence with coverage and limitations.

Possible direction

Reuse the existing masked-search engine (src/domain/native/bytePattern.ts) for the uniqueness proof and relocation search. Ghidra's instruction prototype masks could supply exact per-operand bit masks; the bridge does not call them yet.

References

Checked against main at commit 8ab5024 (rea-agents 6.5.0).
PR #2341 (merged) added search_bytes.
Related: #1947, PR #2124, #3506.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4395 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Native analysis or decompilation ### Current limitation REA can search for a masked pattern but cannot produce one. `search_bytes` (`src/contracts/hopperMemoryContracts.ts`, Ghidra only) finds hex patterns with `??` wildcards and an optional bit mask and reports coverage, and `docs/native-investigation.md` ("Locate a byte signature") documents that use. The signature itself must come from the agent. Building one by hand is error-prone: the agent has to read instruction bytes, decide which bytes are relocations, absolute addresses, or PC-relative displacements, wildcard them, grow the pattern until it is unique, and then check uniqueness across every initialized block. `inspect_native_instruction` exposes bytes and operand kinds for one address but not per-operand byte offsets, `read_function_instructions` returns text without bytes, and nothing combines them. Cross-build correspondence is also left to the agent. `compare_functions` compares two functions the caller has already paired (`symbol` or `explicit` match only), `compare_application_versions` as a tool handles JavaScript application graphs, and `list_procedures` exposes no fingerprints. Address-normalized comparison (#1947, PR #2124) and evidence-supported callee matching (#3506) make a known pair comparable but do not locate the counterpart. ### Use cases - Patch diffing: after a vendor update, find the new address of each function named in the previous build's analysis before comparing behavior. - Malware variant analysis: re-locate known routines in a new sample to carry names and comments forward. - Re-verifying earlier findings: confirm that the code an earlier report cited still exists, unchanged or changed, in the current build. ### Desired outcome and success criteria Add a read-only way to derive the shortest unique masked byte signature for a procedure or address in an analyzed binary, prove its uniqueness with search coverage, and relocate it in a second build of the same program, so names, annotations, and findings can be carried across versions whose addresses shift. - Given an opened binary and a procedure or address, a tool returns a byte pattern and mask, the anchor offset, and for each wildcarded byte the reason (relocation, absolute address operand, PC-relative displacement, or an immediate that resolves to an address). Small non-address immediates stay concrete, consistent with the address-normalization rule discussed in #1947. - Uniqueness is backed by a complete search over the same eligible blocks and cites that Evidence. If coverage is incomplete (uninitialized or not-loaded blocks skipped), the result says uniqueness is unknown rather than unique. - If no unique signature exists within a caller-supplied maximum length, the result is a typed ambiguous outcome listing competing matches, not a non-unique pattern presented as unique. - Relocating a signature in a second opened build reports `unique` (with containing procedure), `ambiguous` (all candidates), or `absent` (scoped to the coverage searched), and never applies names automatically; renaming stays an explicit `set_addresses_names` call citing both Evidence sets. - Unsupported providers return typed capability unavailability. The tool never executes or modifies the target. - A fixture pair with known unchanged and changed functions (for example the Wallace v1/v2 version-tracking exercise referenced in #1947) relocates unchanged functions uniquely and returns ambiguous or absent for changed ones, never a wrong unique match. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) Static analysis of already-opened images only; Ghidra is the natural first provider because REA's bridge already returns instruction bytes and the relocation table. Non-goals: live process memory scanning, patching bytes, or applying signatures to running processes; fuzzy or similarity-based function matching (a separate problem). Documentation examples should use version porting and variant analysis. ### Verification available so far I can suggest a sample or manual check ### Verification details (optional) _No response_ ### Alternatives considered - Hand-built patterns with `search_bytes`: possible today but unreliable for the reasons above, and uniqueness is easy to over-claim when coverage is partial. - Folding matching into `compare_functions`: conflicts with its explicit-pairing contract. - Exposing Rizin zignatures through `execute_rizin_command`: untyped, mutating-capable surface, no coverage-backed uniqueness Evidence. - External tools (IDA signature-maker plugins, Ghidra Version Tracking, BinDiff): useful, but their results do not arrive as REA Evidence with coverage and limitations. ### Possible direction Reuse the existing masked-search engine (`src/domain/native/bytePattern.ts`) for the uniqueness proof and relocation search. Ghidra's instruction prototype masks could supply exact per-operand bit masks; the bridge does not call them yet. ### References - Checked against `main` at commit 8ab5024 (`rea-agents` 6.5.0). - PR #2341 (merged) added `search_bytes`. - Related: #1947, PR #2124, #3506.

bsakaguchi referenced this issue

2026-10-31 00:37:27 +00:00

[GH-ISSUE #5160] [Bug] Codex + kimi-k3: older REA schemas trigger misleading recursion errors; recovery guidance missing #3170

bsakaguchi referenced this issue

2026-10-31 00:55:06 +00:00

[PR #1749] [MERGED] fix(mcp): keep advertised inputs within client reference budgets #4791