morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #748] [MERGED] feat(ghidra): integrate optional NativeAOT metadata recovery #2210

New issue

Closed

opened 2026-10-16 21:15:44 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:15:44 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/748
Author: @N0zoM1z0
Created: 10/6/2026
Status: ✅ Merged
Merged: 10/6/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/ghidra-nativeaot-metadata

📝 Commits (10+)

cf39e30 feat(ghidra): integrate optional NativeAOT metadata recovery
2aecb2a fix(ghidra): bind extension snapshots to configured artifacts
0154760 test(ghidra): verify lazy NativeAOT startup failures
4ca846c fix(ghidra): preserve NativeAOT failure reasons and coverage
b7df14c fix(ghidra): preserve NativeAOT x64 calling conventions
2ab8455 test(ghidra): guard recovered NativeAOT method conventions
fead4be fix(ghidra): reject blocking extension file inputs
0da6ef0 fix(ghidra): diagnose incompatible NativeAOT extension classes
23202a8 test(ghidra): verify real extension initialization failures
e427f0a fix(ghidra): keep extension descriptor fields optional

📊 Changes

43 files changed (+2695 additions, -22 deletions)

View changed files

➕ .github/workflows/nativeaot-fixtures.yml (+44 -0)
📝 .gitmodules (+3 -0)
📝 README.md (+2 -0)
📝 bridge/ghidra/ReaGhidraBridge.java (+22 -6)
➕ bridge/ghidra/ReaGhidraExtensions.java (+113 -0)
➕ bridge/ghidra/extensions/nativeaot/NativeAotDirectory.java (+70 -0)
➕ bridge/ghidra/extensions/nativeaot/NativeAotExtension.java (+183 -0)
➕ bridge/ghidra/extensions/nativeaot/NativeAotMethodConventions.java (+38 -0)
➕ bridge/ghidra/extensions/nativeaot/NativeAotTypeMetadata.java (+70 -0)
➕ docs/ghidra-nativeaot.md (+99 -0)
📝 docs/product-catalog.json (+2 -2)
📝 docs/testing.md (+30 -0)
📝 docs/verification/managed-conformance-manifest.json (+2 -2)
📝 package.json (+5 -1)
➕ scripts/build-ghidra-nativeaot.mjs (+140 -0)
➕ scripts/generate-nativeaot-fixtures.mjs (+188 -0)
➕ scripts/verify-real-ghidra-nativeaot.mjs (+403 -0)
📝 skills/reverse-engineer-anything/SKILL.md (+4 -1)
📝 skills/reverse-engineer-anything/references/native-and-artifacts.md (+24 -0)
📝 src/config/environment.ts (+5 -0)

...and 23 more files

📄 Description

Changes

Integrate optional NativeAOT metadata recovery through a registered Ghidra extension ABI, with separate loader, NativeAOT producer validation and metadata projection modules. Preserve unchanged MIT upstream sources in an optional submodule pinned to effeb734fc570c32650f88b159608979dc7b423e.
Opt in with REA_GHIDRA_NATIVEAOT_JAR. Bind the actual JAR digest to the analysis profile, require matching explicit local configuration, snapshot/recheck its bytes, reject nonregular/blocking inputs and own classloader/project cleanup.
Extend existing inspect_native_load_image with inline format discovery, type identities, measured coverage and derived-memory provenance; extend inspect_native_data_type with related types, interfaces and virtual slots. CLI and MCP use the same workflows; no new tools or approval flags.
Validate RTR directory bounds/flags/version before recovery. Run metadata rehydration and annotations in one ephemeral Program transaction with rollback on fatal failure/cancellation. Preserve stage diagnostics and unsupported reasons; original executable bytes remain unchanged.
Preserve loaded compiler calling conventions instead of upstream's universal __thiscall assignment, which duplicated the receiver in Windows x64 pseudocode.
Add optional source-built fixture/real verification lanes and a separate Windows PE fixture workflow. Installed packages include the generic loader, with no upstream source, compiled extension, fixture binaries or private reference data.
Teach agents to continue NativeAOT native analysis and distinguish recovered native logic, generated/inferred metadata and unknown original C# names/fields/prototypes.

Refs #743, #744. The initial supported/tested boundary is Linux x64 analysis of .NET runtime 8.0.22 / RTR 9.1 x64 ELF and native Windows PE targets, Ghidra 12.1.4 / JDK 21. #743 remains open for broader format/shared-library/sidecar coverage; #744's Windows-host workflow remains open: Windows Ghidra P0 has no mutation authority. Other runtime layouts/hosts are not claimed.

Verification

npm run check:fast, npm run docs:check, npm run knip — passed. Focused domain, provider, extension, parser and MCP schema/catalog tests passed; the final extension suite contains 18 tests, including FIFO rejection without blocking, configuration/profile identity, digest substitution, malformed reports and coverage validation. CI runs the full deterministic gate.
npm run build:ghidra:nativeaot — passed with existing JDK 21 and Ghidra 12.1.4, clean pinned upstream. Repeated build produced identical JAR SHA 8d0e9005e8ce0c3d932b27078a2afaf21de96fec428d7fa4a1a0af24070d2324.
Real symbols and stripped ELF lanes — passed, including independent compiler-symbol base/interface/virtual-target oracles, slot byte readback, frozen strings, representative pseudocode, original SHA and owned cleanup. Original corpus: 1528 method tables / 737 recovered frozen-object candidates. Final reduced source fixture: 1364 / 618; receiver duplication regression passes.
Real source-built Windows x64 PE on Linux, using an installed npm package — passed: 1360 method tables / 649 frozen-object candidates, linker-MAP relationship/slot oracles, string/readback checks and correct two-parameter x64 pseudocode return param_2 * 0xb + 5;. Independently produced fixture workflow: https://github.com/morluto/rea/actions/runs/37493439226 . Original installed Windows-target proof also exercised equivalent CLI metadata and MCP.
Real ordinary native / unsupported RTR version / reversed range / ambiguous directory lanes — passed, with explicit applicability/failure reasons and cleanup.
Final npm pack / offline installation manifest — passed; 761 files, generic loader present and upstream/fixture/JAR/reference artifacts absent. Real installed-package initializer failure preserves both ExceptionInInitializerError and its underlying cause; original SHA and cleanup pass. A final ordinary CLI import with the extension disabled also passes; a new default-native MCP lane caught and guards the optional-descriptor compatibility regression.

NativeAOT fixtures are benign source-built programs and are never executed. Builds are bounded; Ghidra imports run sequentially with a 768 MiB heap. The final Windows target needed two CPUs: its one-CPU run hit the existing 330-second startup deadline. No host defaults/global toolchains changed. The local GNU demangler executable could not run due to its existing file permissions; these proofs establish metadata/native pseudocode, not GNU demangler coverage.

Coverage counts refer to recovered candidates, not every possible object/type. Custom fields and original C# identities remain unknown; recovered prototypes are inferences.

MCP examples

{"name":"inspect_native_load_image","arguments":{}}
{"name":"inspect_native_data_type","arguments":{"type":"/NativeAOT/MethodTables/Class_ADDRESS_MT"}}
{"name":"analyze_function","arguments":{"procedure":"0xADDRESS"}}

Use the actual type category paths/addresses returned inline by the load-image inspection in place of the illustrative placeholders. Existing inline Evidence contains the complete normalized result and original artifact identity; rehydrated memory has its own SHA and no original file offset.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/748 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/6/2026 **Status:** ✅ Merged **Merged:** 10/6/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/ghidra-nativeaot-metadata` --- ### 📝 Commits (10+) - [`cf39e30`](https://github.com/morluto/rea/commit/cf39e30447bbd027342230e441c9c851aaeecd4f) feat(ghidra): integrate optional NativeAOT metadata recovery - [`2aecb2a`](https://github.com/morluto/rea/commit/2aecb2aaac7da4d9800dcde1d6c540f980f600d5) fix(ghidra): bind extension snapshots to configured artifacts - [`0154760`](https://github.com/morluto/rea/commit/0154760ce134bc3e7a9124e0e3333400612f0f88) test(ghidra): verify lazy NativeAOT startup failures - [`4ca846c`](https://github.com/morluto/rea/commit/4ca846c73f8ecc783807c04e5b5816882a9d865d) fix(ghidra): preserve NativeAOT failure reasons and coverage - [`b7df14c`](https://github.com/morluto/rea/commit/b7df14ca9efd3a7d1fd645ee27d036de02940823) fix(ghidra): preserve NativeAOT x64 calling conventions - [`2ab8455`](https://github.com/morluto/rea/commit/2ab8455a8c854574c093e364631677477b2e1cce) test(ghidra): guard recovered NativeAOT method conventions - [`fead4be`](https://github.com/morluto/rea/commit/fead4bee8901920dbfbb4043c19b3e87ff54950b) fix(ghidra): reject blocking extension file inputs - [`0da6ef0`](https://github.com/morluto/rea/commit/0da6ef01262f25703b0834f09e4a665222932d9e) fix(ghidra): diagnose incompatible NativeAOT extension classes - [`23202a8`](https://github.com/morluto/rea/commit/23202a8b001fdd459ea6ba0c40be31a15c3f880f) test(ghidra): verify real extension initialization failures - [`e427f0a`](https://github.com/morluto/rea/commit/e427f0a77a7da241d9ac23208a41b864cdb6301e) fix(ghidra): keep extension descriptor fields optional ### 📊 Changes **43 files changed** (+2695 additions, -22 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/nativeaot-fixtures.yml` (+44 -0) 📝 `.gitmodules` (+3 -0) 📝 `README.md` (+2 -0) 📝 `bridge/ghidra/ReaGhidraBridge.java` (+22 -6) ➕ `bridge/ghidra/ReaGhidraExtensions.java` (+113 -0) ➕ `bridge/ghidra/extensions/nativeaot/NativeAotDirectory.java` (+70 -0) ➕ `bridge/ghidra/extensions/nativeaot/NativeAotExtension.java` (+183 -0) ➕ `bridge/ghidra/extensions/nativeaot/NativeAotMethodConventions.java` (+38 -0) ➕ `bridge/ghidra/extensions/nativeaot/NativeAotTypeMetadata.java` (+70 -0) ➕ `docs/ghidra-nativeaot.md` (+99 -0) 📝 `docs/product-catalog.json` (+2 -2) 📝 `docs/testing.md` (+30 -0) 📝 `docs/verification/managed-conformance-manifest.json` (+2 -2) 📝 `package.json` (+5 -1) ➕ `scripts/build-ghidra-nativeaot.mjs` (+140 -0) ➕ `scripts/generate-nativeaot-fixtures.mjs` (+188 -0) ➕ `scripts/verify-real-ghidra-nativeaot.mjs` (+403 -0) 📝 `skills/reverse-engineer-anything/SKILL.md` (+4 -1) 📝 `skills/reverse-engineer-anything/references/native-and-artifacts.md` (+24 -0) 📝 `src/config/environment.ts` (+5 -0) _...and 23 more files_ </details> ### 📄 Description ## Changes - Integrate optional NativeAOT metadata recovery through a registered Ghidra extension ABI, with separate loader, NativeAOT producer validation and metadata projection modules. Preserve unchanged MIT upstream sources in an optional submodule pinned to `effeb734fc570c32650f88b159608979dc7b423e`. - Opt in with `REA_GHIDRA_NATIVEAOT_JAR`. Bind the actual JAR digest to the analysis profile, require matching explicit local configuration, snapshot/recheck its bytes, reject nonregular/blocking inputs and own classloader/project cleanup. - Extend existing `inspect_native_load_image` with inline format discovery, type identities, measured coverage and derived-memory provenance; extend `inspect_native_data_type` with related types, interfaces and virtual slots. CLI and MCP use the same workflows; no new tools or approval flags. - Validate RTR directory bounds/flags/version before recovery. Run metadata rehydration and annotations in one ephemeral Program transaction with rollback on fatal failure/cancellation. Preserve stage diagnostics and unsupported reasons; original executable bytes remain unchanged. - Preserve loaded compiler calling conventions instead of upstream's universal `__thiscall` assignment, which duplicated the receiver in Windows x64 pseudocode. - Add optional source-built fixture/real verification lanes and a separate Windows PE fixture workflow. Installed packages include the generic loader, with no upstream source, compiled extension, fixture binaries or private reference data. - Teach agents to continue NativeAOT native analysis and distinguish recovered native logic, generated/inferred metadata and unknown original C# names/fields/prototypes. Refs #743, #744. The initial supported/tested boundary is **Linux x64 analysis of .NET runtime 8.0.22 / RTR 9.1 x64 ELF and native Windows PE targets**, Ghidra 12.1.4 / JDK 21. **#743 remains open for broader format/shared-library/sidecar coverage; #744's Windows-host workflow remains open**: Windows Ghidra P0 has no mutation authority. Other runtime layouts/hosts are not claimed. ## Verification - `npm run check:fast`, `npm run docs:check`, `npm run knip` — passed. Focused domain, provider, extension, parser and MCP schema/catalog tests passed; the final extension suite contains 18 tests, including FIFO rejection without blocking, configuration/profile identity, digest substitution, malformed reports and coverage validation. CI runs the full deterministic gate. - `npm run build:ghidra:nativeaot` — passed with existing JDK 21 and Ghidra 12.1.4, clean pinned upstream. Repeated build produced identical JAR SHA `8d0e9005e8ce0c3d932b27078a2afaf21de96fec428d7fa4a1a0af24070d2324`. - Real `symbols` and `stripped` ELF lanes — passed, including independent compiler-symbol base/interface/virtual-target oracles, slot byte readback, frozen strings, representative pseudocode, original SHA and owned cleanup. Original corpus: 1528 method tables / 737 recovered frozen-object candidates. Final reduced source fixture: 1364 / 618; receiver duplication regression passes. - Real source-built **Windows x64 PE on Linux**, using an installed npm package — passed: 1360 method tables / 649 frozen-object candidates, linker-MAP relationship/slot oracles, string/readback checks and correct two-parameter x64 pseudocode `return param_2 * 0xb + 5;`. Independently produced fixture workflow: https://github.com/morluto/rea/actions/runs/37493439226 . Original installed Windows-target proof also exercised equivalent CLI metadata and MCP. - Real ordinary native / unsupported RTR version / reversed range / ambiguous directory lanes — passed, with explicit applicability/failure reasons and cleanup. - Final `npm pack` / offline installation manifest — passed; 761 files, generic loader present and upstream/fixture/JAR/reference artifacts absent. Real installed-package initializer failure preserves both `ExceptionInInitializerError` and its underlying cause; original SHA and cleanup pass. A final ordinary CLI import with the extension disabled also passes; a new `default-native` MCP lane caught and guards the optional-descriptor compatibility regression. NativeAOT fixtures are benign source-built programs and are never executed. Builds are bounded; Ghidra imports run sequentially with a 768 MiB heap. The final Windows target needed two CPUs: its one-CPU run hit the existing 330-second startup deadline. No host defaults/global toolchains changed. The local GNU demangler executable could not run due to its existing file permissions; these proofs establish metadata/native pseudocode, not GNU demangler coverage. Coverage counts refer to recovered candidates, not every possible object/type. Custom fields and original C# identities remain unknown; recovered prototypes are inferences. ## MCP examples ```json {"name":"inspect_native_load_image","arguments":{}} {"name":"inspect_native_data_type","arguments":{"type":"/NativeAOT/MethodTables/Class_ADDRESS_MT"}} {"name":"analyze_function","arguments":{"procedure":"0xADDRESS"}} ``` Use the actual type category paths/addresses returned inline by the load-image inspection in place of the illustrative placeholders. Existing inline Evidence contains the complete normalized result and original artifact identity; rehydrated memory has its own SHA and no original file offset. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:15:44 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 01:03:35 +00:00

[PR #2210] [MERGED] ci: keep validation fresh and preserve failed-shard reports #5085

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2210

No description provided.

Rows
Columns