morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8194] [Design] Investigate evidence-backed algorithm-constant discovery and references #6475

New issue

Open

opened 2026-11-01 21:11:27 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-11-01 21:11:27 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8194

Before submitting

I searched open and closed issues and pull requests for a related discussion.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Can REA help answer “where are known algorithm-related byte constants, and what code references them?” as a bounded evidence-backed investigation, without claiming the target implements or uses an algorithm merely because a table occurs?

Context and evidence

I generated the standard 256-byte AES substitution table from its field arithmetic, placed it in a source-owned arm64 Mach-O with a function that indexes it, and analyzed but did not execute the binary. The fixture deliberately contains no AES implementation (SHA-256 b671645aeb1b53e2f29539973fcc0837d9a4ff571d211fc489534a95d89344d1).

Direct Kuna 1.986 crypto ... --algorithm aes --json reported one AES / S-box candidate at 0x1000003c0, 256 bytes, and no referencing functions. The source does contain a lookup function, so zero reported references is not proof of no use.
REA/Ghidra 12.1.4 search_bytes with the caller-supplied exact table found the same address with initialized-memory coverage. REA xrefs returned 0x100000390; procedure_references for _audit_lookup reported a DATA edge from that site to the table.
REA/Kuna reported search_bytes and procedure_references unavailable; its exact-address xrefs returned an empty list. trace_feature({"query":"AES"}) is a literal name/string search, not a byte-signature classifier: in the Ghidra run its only match was the fixture filename, and Kuna returned no match.

These are single-fixture observations, not measured accuracy, comprehensive xref coverage, or proof of algorithm behavior. A table can have other uses, be embedded as data, be transformed, or be absent even when an algorithm exists.

Kuna 1.988 follow-up on the same fixture

The released Kuna 1.988 changes the reference observation above. Direct crypto --algorithm aes --json now reports the same AES S-box candidate and one referencing function, _audit_lookup at 0x100000378. Direct xrefs --to 0x1000003c0 reports a data reference at 0x10000038c; REA/Kuna 1.988 xrefs in compact stdio MCP returns that address too. The original REA/Kuna 1.986 negative remains a historical version-specific observation, not a current 1.988 result. search_bytes and procedure_references remain unavailable for REA/Kuna in this control, and trace_feature({query:"AES"}) still returned no matches.

An independent llvm-objdump shows the 0x10000038c instruction forms the table base with add, followed by an indexed byte load at 0x100000390. The earlier Ghidra control reported 0x100000390; these are distinct provider reference-site observations and should not be silently conflated. The fixture deliberately has no AES implementation, so even the positive table and referencing-function evidence does not prove algorithm use. With direct Kuna crypto --no-xrefs, the same candidate reports zero references and top-level xrefs: false; this means the reference walk was skipped, not that no references exist.

This is one unchanged, unexecuted arm64 Mach-O and one release on macOS arm64, not a general 1.988 accuracy claim. Its SHA-256 remained b671645aeb1b53e2f29539973fcc0837d9a4ff571d211fc489534a95d89344d1.

Negative and scope controls

Using Kuna 1.988 directly with --algorithm aes --no-xrefs, two unexecuted copies of the same fixture produced zero candidates: one changed a single byte of the 256-byte table (SHA-256 db012d1016b260858aa8a050215a1f089fb33ebf5f5e6a726992a6e119ff425e), and the other XOR-transformed the table (SHA-256 4ec5a3a055f2533afc672acf8823aca33f7b73a36483842f2aba20fba84e50d4). Only the table bytes differ from the original; neither fixture implements AES. This demonstrates the exact-signature limitation on these cases, not failure to detect a real transformed AES implementation.

On the original positive fixture, selecting the valid __text section returned zero AES candidates, while __const returned one; selecting a nonexistent section exited with an error naming the available sections. The successful outputs reported the selected section and xrefs: false because the reference walk was explicitly skipped. An empty result must therefore retain the selected scan scope and reference-walk state; it cannot establish algorithm absence or absence of references outside that scope.

Interior table-reference control

A separate owned, unexecuted arm64 Mach-O places the same 256-byte S-box at 0x100000398 and reads byte 16 directly. On Kuna 1.988, crypto --algorithm aes reported the table and _audit_lookup; direct xrefs --to found no references at the table start but found the load at 0x10000037c when queried at 0x1000003a8. Real REA compact MCP with both Kuna 1.988 and Ghidra 12.1.4 produced the same exact-address distinction. Independent llvm-objdump showed the indexed-offset ldrb at that source instruction. Target SHA-256 597c08e1d2d8732cdfad7170f61b28cf9d25be4be65d7bc9e7486faf00f740ad remained unchanged.

This shows that xrefs for a signature's starting address alone cannot establish the references to the entire matched range. A composed task needs range-aware reference coverage or an explicit unknown, while preserving exact source and destination addresses and provider-specific gaps. This control is one direct static reference and does not prove runtime execution or AES use.

Expected interaction with targets

Read selected local files and static analysis observations; optionally launch the selected static provider. No target execution or binary mutation is proposed.

Boundary details (optional)

Preserve exact candidate bytes or signature identity/version, target/slice and address mapping, searched ranges, omitted/uninitialized regions, candidate encoding/layout, and observation coverage. Keep table matches, static reference observations, and behavioral inferences separate. An empty reference list is unknown when reference coverage is incomplete.

Options and tradeoffs

Investigate composing a bounded signature catalog with existing search_bytes, xrefs and retained Evidence, versus accepting a provider's explicitly scoped candidate observations. Reuse existing analyst workflows where feasible; do not add a raw kuna crypto wrapper or equate provider results. A caller-supplied signature might be enough for some tasks, while curated signatures introduce corpus ownership, updates and false-positive tradeoffs.

Verification available so far

I have a prototype or observed results.

Success criteria and open questions

Test genuine and coincidental matches, absent and transformed tables, multiple encodings/architectures, range limits, incomplete and negative xrefs, and reproducible signature identities. Verify that output calls results candidates and never asserts encryption or runtime use without independent evidence.

References

Related: #8170 (original byte reads), #8184 (reference coverage), #8190 (outgoing reference semantics). REA source d48e472fa plus unrelated Kuna baseline changes in #8187; macOS arm64.

Kuna 1.989 range and skipped-reference control

On macOS arm64, a new unexecuted arm64 Mach-O contains the standard 256-byte AES S-box and two lookup functions, including one that reads byte 16 directly; it does not implement AES. Direct Kuna 1.989 crypto --algorithm aes --json reported one 256-byte candidate at 0x1000003c4 and both _look and _interior as referencing functions. With --no-xrefs, the same candidate had zero references but top-level xrefs: false; this is a skipped walk, not a negative reference finding. Restricting the scan to __text returned zero candidates for that selected scope.

Direct Kuna exact-address xrefs reported _look's address formation at 0x100000380 for the table start and _interior's read at 0x100000390 for 0x1000003d4. Real REA MCP/Kuna returned those distinct exact-address results in compact and full profiles. REA/Ghidra returned 0x100000384 for the start and 0x100000390 for the interior. REA/Ghidra search_bytes with the caller-provided 256-byte signature found the table and reported scanned and skipped ranges; REA/Kuna reported search_bytes unavailable. Successful MCP outputs validated against the advertised schemas. This is one fixture and one host, not a coverage claim or evidence of AES execution. The unchanged target SHA-256 was db652d426f596d311d91d5e20f1e3d714784f0512faa70dca8a2a788d99b15a7.

This reinforces the analyst task's boundary: a signature candidate, whether the reference walk ran, the scanned scope, and references to a range are distinct facts. Querying only the table's first byte misses the independently observed interior reference; differing provider source locations should remain separate observations.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8194 ### Before submitting - [x] I searched open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Can REA help answer “where are known algorithm-related byte constants, and what code references them?” as a bounded evidence-backed investigation, without claiming the target implements or uses an algorithm merely because a table occurs? ### Context and evidence I generated the standard 256-byte AES substitution table from its field arithmetic, placed it in a source-owned arm64 Mach-O with a function that indexes it, and analyzed but did not execute the binary. The fixture deliberately contains no AES implementation (SHA-256 `b671645aeb1b53e2f29539973fcc0837d9a4ff571d211fc489534a95d89344d1`). - Direct Kuna 1.986 `crypto ... --algorithm aes --json` reported one `AES` / `S-box` candidate at `0x1000003c0`, 256 bytes, and no referencing functions. The source does contain a lookup function, so zero reported references is not proof of no use. - REA/Ghidra 12.1.4 `search_bytes` with the caller-supplied exact table found the same address with initialized-memory coverage. REA `xrefs` returned `0x100000390`; `procedure_references` for `_audit_lookup` reported a `DATA` edge from that site to the table. - REA/Kuna reported `search_bytes` and `procedure_references` unavailable; its exact-address `xrefs` returned an empty list. `trace_feature({"query":"AES"})` is a literal name/string search, not a byte-signature classifier: in the Ghidra run its only match was the fixture filename, and Kuna returned no match. These are single-fixture observations, not measured accuracy, comprehensive xref coverage, or proof of algorithm behavior. A table can have other uses, be embedded as data, be transformed, or be absent even when an algorithm exists. ### Kuna 1.988 follow-up on the same fixture The released Kuna 1.988 changes the reference observation above. Direct `crypto --algorithm aes --json` now reports the same AES S-box candidate and one referencing function, `_audit_lookup` at `0x100000378`. Direct `xrefs --to 0x1000003c0` reports a `data` reference at `0x10000038c`; REA/Kuna 1.988 `xrefs` in compact stdio MCP returns that address too. The original REA/Kuna 1.986 negative remains a historical version-specific observation, not a current 1.988 result. `search_bytes` and `procedure_references` remain unavailable for REA/Kuna in this control, and `trace_feature({query:"AES"})` still returned no matches. An independent `llvm-objdump` shows the `0x10000038c` instruction forms the table base with `add`, followed by an indexed byte load at `0x100000390`. The earlier Ghidra control reported `0x100000390`; these are distinct provider reference-site observations and should not be silently conflated. The fixture deliberately has no AES implementation, so even the positive table and referencing-function evidence does not prove algorithm use. With direct Kuna `crypto --no-xrefs`, the same candidate reports zero references **and** top-level `xrefs: false`; this means the reference walk was skipped, not that no references exist. This is one unchanged, unexecuted arm64 Mach-O and one release on macOS arm64, not a general 1.988 accuracy claim. Its SHA-256 remained `b671645aeb1b53e2f29539973fcc0837d9a4ff571d211fc489534a95d89344d1`. ### Negative and scope controls Using Kuna 1.988 directly with `--algorithm aes --no-xrefs`, two unexecuted copies of the same fixture produced zero candidates: one changed a single byte of the 256-byte table (SHA-256 `db012d1016b260858aa8a050215a1f089fb33ebf5f5e6a726992a6e119ff425e`), and the other XOR-transformed the table (SHA-256 `4ec5a3a055f2533afc672acf8823aca33f7b73a36483842f2aba20fba84e50d4`). Only the table bytes differ from the original; neither fixture implements AES. This demonstrates the exact-signature limitation on these cases, not failure to detect a real transformed AES implementation. On the original positive fixture, selecting the valid `__text` section returned zero AES candidates, while `__const` returned one; selecting a nonexistent section exited with an error naming the available sections. The successful outputs reported the selected section and `xrefs: false` because the reference walk was explicitly skipped. An empty result must therefore retain the selected scan scope and reference-walk state; it cannot establish algorithm absence or absence of references outside that scope. ### Interior table-reference control A separate owned, unexecuted arm64 Mach-O places the same 256-byte S-box at `0x100000398` and reads byte 16 directly. On Kuna 1.988, `crypto --algorithm aes` reported the table and `_audit_lookup`; direct `xrefs --to` found no references at the table start but found the load at `0x10000037c` when queried at `0x1000003a8`. Real REA compact MCP with both Kuna 1.988 and Ghidra 12.1.4 produced the same exact-address distinction. Independent `llvm-objdump` showed the indexed-offset `ldrb` at that source instruction. Target SHA-256 `597c08e1d2d8732cdfad7170f61b28cf9d25be4be65d7bc9e7486faf00f740ad` remained unchanged. This shows that `xrefs` for a signature's starting address alone cannot establish the references to the entire matched range. A composed task needs range-aware reference coverage or an explicit unknown, while preserving exact source and destination addresses and provider-specific gaps. This control is one direct static reference and does not prove runtime execution or AES use. ### Expected interaction with targets Read selected local files and static analysis observations; optionally launch the selected static provider. No target execution or binary mutation is proposed. ### Boundary details (optional) Preserve exact candidate bytes or signature identity/version, target/slice and address mapping, searched ranges, omitted/uninitialized regions, candidate encoding/layout, and observation coverage. Keep table matches, static reference observations, and behavioral inferences separate. An empty reference list is unknown when reference coverage is incomplete. ### Options and tradeoffs Investigate composing a bounded signature catalog with existing `search_bytes`, `xrefs` and retained Evidence, versus accepting a provider's explicitly scoped candidate observations. Reuse existing analyst workflows where feasible; do not add a raw `kuna crypto` wrapper or equate provider results. A caller-supplied signature might be enough for some tasks, while curated signatures introduce corpus ownership, updates and false-positive tradeoffs. ### Verification available so far I have a prototype or observed results. ### Success criteria and open questions Test genuine and coincidental matches, absent and transformed tables, multiple encodings/architectures, range limits, incomplete and negative xrefs, and reproducible signature identities. Verify that output calls results candidates and never asserts encryption or runtime use without independent evidence. ### References Related: #8170 (original byte reads), #8184 (reference coverage), #8190 (outgoing reference semantics). REA source `d48e472fa` plus unrelated Kuna baseline changes in #8187; macOS arm64. ### Kuna 1.989 range and skipped-reference control On macOS arm64, a new unexecuted arm64 Mach-O contains the standard 256-byte AES S-box and two lookup functions, including one that reads byte 16 directly; it does not implement AES. Direct Kuna 1.989 `crypto --algorithm aes --json` reported one 256-byte candidate at `0x1000003c4` and both `_look` and `_interior` as referencing functions. With `--no-xrefs`, the same candidate had zero references but top-level `xrefs: false`; this is a skipped walk, not a negative reference finding. Restricting the scan to `__text` returned zero candidates for that selected scope. Direct Kuna exact-address xrefs reported `_look`'s address formation at `0x100000380` for the table start and `_interior`'s read at `0x100000390` for `0x1000003d4`. Real REA MCP/Kuna returned those distinct exact-address results in compact and full profiles. REA/Ghidra returned `0x100000384` for the start and `0x100000390` for the interior. REA/Ghidra `search_bytes` with the caller-provided 256-byte signature found the table and reported scanned and skipped ranges; REA/Kuna reported `search_bytes` unavailable. Successful MCP outputs validated against the advertised schemas. This is one fixture and one host, not a coverage claim or evidence of AES execution. The unchanged target SHA-256 was `db652d426f596d311d91d5e20f1e3d714784f0512faa70dca8a2a788d99b15a7`. This reinforces the analyst task's boundary: a signature candidate, whether the reference walk ran, the scanned scope, and references to a range are distinct facts. Querying only the table's first byte misses the independently observed interior reference; differing provider source locations should remain separate observations.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:27 +00:00

bsakaguchi commented

2026-11-01 21:11:27 +00:00

Author

Owner

@morluto commented on GitHub (Oct 30, 2026):

A separate immediate-value control on a self-built x86-64 ELF illustrates why candidate kind and attribution matter. One ordinary function returns 0x9e3779b9; a separate data variable holds the same number. There is no TEA implementation. Direct Kuna 1.986 crypto --algorithm TEA --json reported a TEA/XTEA immediate candidate at the four-byte operand inside the return-constant instruction, with the containing function listed and xrefs_count: 0. With --no-xrefs, the same candidate remained but its functions list was empty and top-level xrefs was false. A one-byte-changed control returned no TEA candidate. None of these results establishes runtime use or algorithm implementation.

REA/Ghidra search_bytes found the byte sequence both within .text and in .data. The code hit retained its containing procedure; following the procedure's instruction listing and querying the instruction start showed a five-byte MOV with a typed immediate. Exact xrefs to the operand byte were empty, while the data address had separate references including a typed read. REA/Kuna returned capability unavailable for byte search and typed instruction/reference tools, but did return the data-address xref in this example. This is one immediate/control pair, not a classifier accuracy study. #8241 covers the byte-hit-to-instruction handoff, not a request to wrap the crypto command. Both target digests stayed unchanged and neither was executed.

@morluto commented on GitHub (Oct 30, 2026): A separate immediate-value control on a self-built x86-64 ELF illustrates why candidate kind and attribution matter. One ordinary function returns `0x9e3779b9`; a separate data variable holds the same number. There is no TEA implementation. Direct Kuna 1.986 `crypto --algorithm TEA --json` reported a `TEA/XTEA` immediate candidate at the four-byte operand inside the return-constant instruction, with the containing function listed and `xrefs_count: 0`. With `--no-xrefs`, the same candidate remained but its functions list was empty and top-level `xrefs` was false. A one-byte-changed control returned no TEA candidate. None of these results establishes runtime use or algorithm implementation. REA/Ghidra `search_bytes` found the byte sequence both within `.text` and in `.data`. The code hit retained its containing procedure; following the procedure's instruction listing and querying the instruction start showed a five-byte `MOV` with a typed immediate. Exact `xrefs` to the operand byte were empty, while the data address had separate references including a typed read. REA/Kuna returned capability unavailable for byte search and typed instruction/reference tools, but did return the data-address xref in this example. This is one immediate/control pair, not a classifier accuracy study. #8241 covers the byte-hit-to-instruction handoff, not a request to wrap the crypto command. Both target digests stayed unchanged and neither was executed.

bsakaguchi commented

2026-11-01 21:11:27 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 13:27:02 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6475

No description provided.

Rows
Columns