[PR #4112] [MERGED] fix(ghidra): use resolved type addresses for NativeAOT metadata #6062

New issue

Closed

opened 2026-10-31 01:11:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:11:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4112
Author: @N0zoM1z0
Created: 10/15/2026
Status: ✅ Merged
Merged: 10/15/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/nativeaot-canonical-type-address-3327

📝 Commits (1)

2e56c3c fix(ghidra): use resolved type addresses for NativeAOT metadata

📊 Changes

3 files changed (+193 additions, -124 deletions)

View changed files

📝 docs/ghidra-nativeaot.md (+6 -0)
📝 src/ghidra/GhidraProvider.behavior.test.ts (+171 -105)
📝 src/ghidra/GhidraProviderClient.ts (+16 -19)

📄 Description

Summary

Fixes #3327. NativeAOT type inspection now enriches the Ghidra-resolved type address, so bare hex, a default-space selector, its percent-encoded spelling and prefixed controls return the same recovered MethodTable. Original caller spelling remains in Evidence parameters; foreign spaces and unresolved identities are not matched by numeric offset alone.

Problem and approach

The Java bridge already resolves address spelling and emits a canonical address, even when no database DataType exists. The TypeScript consumer discarded that identity and passed the original request string to a PE metadata reader that accepts flat canonical identities. This silently removed otherwise available metadata.

Use the already parsed bridge result. Keep the object/identity guard at that boundary and remove the later redundant checks and unsafe cast. Preserve engine-supplied metadata before attempting an independent snapshot read; an already enriched type must not fail because an unnecessary extra snapshot read fails. No second address parser, alias table, new cache or new tool call is introduced.

Three drifting native-type stubs previously returned a null address for address queries, unlike the real Java producer. They now share a fixture with an explicit resolved address. Regression cases cover all reported spellings, a different resolved object, unknown identity, distinct/encoded spaces, interior addresses, original request parameters, one snapshot shared by load-image plus repeated type queries, and preservation of engine-provided metadata without another snapshot read. Existing identity-conflict, cancellation, close and parser-capacity coverage remains.

Contract and boundary impact

Owner: Ghidra bridge's parsed type identity; supplemental interpretation stays in the existing session-owned NativeAOT snapshot owner.
Scope: ordinary fix within the current provider/metadata capability; no new maintainer scope decision is applicable. No new target execution, injection, naming, database or executable mutation.
CLI/MCP: request/result schemas and caller choices stay compatible. Database status and raw observations retain their meaning; read-only-derived-overlay metadata is distinct from a Ghidra DataType. Existing engine metadata remains authoritative.
The validated snapshot cache is still keyed by target SHA-256 and loaded image base, shared within one session, invalidated on close and rejected on a conflicting base. No cached live database result is reported as current.
No provider, dependency, installation, release-version, bridge protocol or generated-file change. The owning NativeAOT guide documents the identity boundary.

Executed evidence

Actual Linux Ghidra 12.1.4 / Temurin 21.0.12.1 on initial main c7697595: all five spellings resolve to 0x140002280; the three non-prefixed/selected forms omit metadata. This uses the original synthetic PE bytes and a real Java process, not an injected client factory.
Actual compiled NativeAOT PE: .NET SDK 8.0.416 / runtime 8.0.22 artifact from source-built fixture run 37834266655, source f8647faf, executable SHA-256 1fdd1250b4fb22125b461f4a626dc42af449fed2a822b192c935b49a3cde8722. GitHub artifact archive digest and the independent byte oracle agree. Its RTR 9.1 metadata contains 324 recovered MethodTables within the parser's stated coverage.
Public stdio MCP, unmodified parent fc41b24d, actual Ghidra: all spellings resolve to 0x14009a000; canonical/uppercase controls include metadata while bare/default-space/encoded-default-space forms omit it. The first analysis takes about 90 s under two CPU cores; subsequent queries take 8–15 ms. Target bytes remain unchanged and the session closes.
On unfixed production source, final regression cases produce 10 failures / 73 passing cases and controls; corrected source passes. Mock-backed cases establish the specific identity/cache/engine-metadata semantics, not real addon behavior.
Fixed real stdio MCP: every reported spelling returns the same full normalized MethodTable result in one call, with original Evidence parameters preserved. The public CLI's bare-hex control on the synthetic PE changes from missing metadata to recovered metadata; both use actual Ghidra and preserve target bytes.

Measured completed tasks

Actual public stdio MCP + real Ghidra, same compiled PE and limits, parent fc41b24d / runtime 2e56c3cc. Each version completes one cold type query and ten warm queries (two per spelling); baseline retries the returned canonical identity when needed. Every final normalized type result is identical across both versions.

Metric	Before	After
Warm noncanonical query calls	2	1
Warm noncanonical response tokens, median	5489	3011
Warm noncanonical request + response tokens, median	5529	3032
Warm noncanonical elapsed, median ms (n=6)	28.39	16.75
Type calls for all 11 completed goals	18	11
Whole recorded workflow payload tokens	862928	845367
Whole workflow wall / reaped CPU, s	230.07 / 243.44	161.35 / 167.72
Sampled peak family RSS, MiB	1598.8	1887.2

The full SDK catalog is byte-identical: 207 tools / 811,880 proxy tokens; the three selected tool definitions are 5,272 tokens in each version. Whole recorded payload counts include that catalog, open/close and all query requests/replies. Counts use cl100k_base over actual SDK JSON bodies; initialize/framing, actual model consumption and billing are unmeasured. Per-request CPU is unmeasured; CPU/RSS above cover the full completed workflow.

Cold goal timings were 214.84 / 143.62 s (one observation per version), dominated by Ghidra startup. OS caches and shared-host load changed substantially; these cold/CPU/RSS differences establish no causal speedup or memory improvement. Warm noncanonical samples combine three spellings, two repetitions each. Node compile caching was disabled; warm means the same live Ghidra session and validated snapshot. Family RSS is sampled at 200 ms and may double-count shared pages/miss brief children. Heavy jobs ran serially on two CPU cores, JVM heap 2 GiB, own-family guard 7 GiB within the approved 8 GiB overall budget; validation peaked at 4.46 GiB, with no guard/OOM event.

Validation performed

npm run test:focused -- src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraFunctionValues.test.ts src/domain/native/nativeAotPe.test.ts src/domain/native/nativeAotPe.budget.test.ts tests/boundary/mcp/ghidraNativeApiMcp.test.ts — 174 passed.
npm run check — passed typecheck, lint, formatting, unused-code/metadata/module checks and architecture guards.
npm run docs:check — passed, including five documentation consistency tests.

Coverage limits

Local execution is Linux x86_64 and actual headless Ghidra; no Windows/macOS runtime, attached GUI, optional NativeAOT addon, or target-program execution is claimed. The compiled fixture is a genuine compiler-produced C# test program, not a production application parity claim. Engine-provided-metadata I/O avoidance is verified at the production boundary with a controlled client.

The larger Microsoft ILCompiler 8.0.0 binary (11.56 MB) exceeded the existing 330 s Ghidra startup deadline under two CPU cores / 2 GiB JVM, before metadata lookup. No successful larger-application or cold-analysis speedup claim is made. Failed attempts and unrelated unsupported-layout observations are retained in the local test recap.

Author readiness

Human owner: @N0zoM1z0

The human owner explicitly authorized delegated review, proxy completion of these declarations, ready-for-review publication and normal merge for this batch. The agent reviewed the final diff and evidence; this records that delegation and does not assert a separate manual review occurred.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4112 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/15/2026 **Status:** ✅ Merged **Merged:** 10/15/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/nativeaot-canonical-type-address-3327` --- ### 📝 Commits (1) - [`2e56c3c`](https://github.com/morluto/rea/commit/2e56c3ccaf9ae17af5cb07eea33668f77c065034) fix(ghidra): use resolved type addresses for NativeAOT metadata ### 📊 Changes **3 files changed** (+193 additions, -124 deletions) <details> <summary>View changed files</summary> 📝 `docs/ghidra-nativeaot.md` (+6 -0) 📝 `src/ghidra/GhidraProvider.behavior.test.ts` (+171 -105) 📝 `src/ghidra/GhidraProviderClient.ts` (+16 -19) </details> ### 📄 Description ## Summary Fixes #3327. NativeAOT type inspection now enriches the Ghidra-resolved type address, so bare hex, a default-space selector, its percent-encoded spelling and prefixed controls return the same recovered MethodTable. Original caller spelling remains in Evidence parameters; foreign spaces and unresolved identities are not matched by numeric offset alone. ## Problem and approach The Java bridge already resolves address spelling and emits a canonical address, even when no database DataType exists. The TypeScript consumer discarded that identity and passed the original request string to a PE metadata reader that accepts flat canonical identities. This silently removed otherwise available metadata. Use the already parsed bridge result. Keep the object/identity guard at that boundary and remove the later redundant checks and unsafe cast. Preserve engine-supplied metadata before attempting an independent snapshot read; an already enriched type must not fail because an unnecessary extra snapshot read fails. No second address parser, alias table, new cache or new tool call is introduced. Three drifting native-type stubs previously returned a null address for address queries, unlike the real Java producer. They now share a fixture with an explicit resolved address. Regression cases cover all reported spellings, a different resolved object, unknown identity, distinct/encoded spaces, interior addresses, original request parameters, one snapshot shared by load-image plus repeated type queries, and preservation of engine-provided metadata without another snapshot read. Existing identity-conflict, cancellation, close and parser-capacity coverage remains. ## Contract and boundary impact - Owner: Ghidra bridge's parsed type identity; supplemental interpretation stays in the existing session-owned NativeAOT snapshot owner. - Scope: ordinary fix within the current provider/metadata capability; no new maintainer scope decision is applicable. No new target execution, injection, naming, database or executable mutation. - CLI/MCP: request/result schemas and caller choices stay compatible. Database status and raw observations retain their meaning; `read-only-derived-overlay` metadata is distinct from a Ghidra DataType. Existing engine metadata remains authoritative. - The validated snapshot cache is still keyed by target SHA-256 and loaded image base, shared within one session, invalidated on close and rejected on a conflicting base. No cached live database result is reported as current. - No provider, dependency, installation, release-version, bridge protocol or generated-file change. The owning NativeAOT guide documents the identity boundary. ## Executed evidence - Actual Linux Ghidra 12.1.4 / Temurin 21.0.12.1 on initial main `c7697595`: all five spellings resolve to `0x140002280`; the three non-prefixed/selected forms omit metadata. This uses the original synthetic PE bytes and a real Java process, not an injected client factory. - **Actual compiled NativeAOT PE:** .NET SDK 8.0.416 / runtime 8.0.22 artifact from [source-built fixture run 37834266655](https://github.com/morluto/rea/actions/runs/37834266655), source `f8647faf`, executable SHA-256 `1fdd1250b4fb22125b461f4a626dc42af449fed2a822b192c935b49a3cde8722`. GitHub artifact archive digest and the independent byte oracle agree. Its RTR 9.1 metadata contains 324 recovered MethodTables within the parser's stated coverage. - Public **stdio MCP**, unmodified parent `fc41b24d`, actual Ghidra: all spellings resolve to `0x14009a000`; canonical/uppercase controls include metadata while bare/default-space/encoded-default-space forms omit it. The first analysis takes about 90 s under two CPU cores; subsequent queries take 8–15 ms. Target bytes remain unchanged and the session closes. - On unfixed production source, final regression cases produce **10 failures / 73 passing cases and controls**; corrected source passes. Mock-backed cases establish the specific identity/cache/engine-metadata semantics, not real addon behavior. - Fixed real stdio MCP: every reported spelling returns the same full normalized MethodTable result in one call, with original Evidence parameters preserved. The public CLI's bare-hex control on the synthetic PE changes from missing metadata to recovered metadata; both use actual Ghidra and preserve target bytes. ### Measured completed tasks Actual public stdio MCP + real Ghidra, same compiled PE and limits, parent `fc41b24d` / runtime `2e56c3cc`. Each version completes one cold type query and ten warm queries (two per spelling); baseline retries the returned canonical identity when needed. **Every final normalized type result is identical across both versions.** | Metric | Before | After | | --- | ---: | ---: | | Warm noncanonical query calls | 2 | 1 | | Warm noncanonical response tokens, median | 5489 | 3011 | | Warm noncanonical request + response tokens, median | 5529 | 3032 | | Warm noncanonical elapsed, median ms (n=6) | 28.39 | 16.75 | | Type calls for all 11 completed goals | 18 | 11 | | Whole recorded workflow payload tokens | 862928 | 845367 | | Whole workflow wall / reaped CPU, s | 230.07 / 243.44 | 161.35 / 167.72 | | Sampled peak family RSS, MiB | 1598.8 | 1887.2 | The full SDK catalog is byte-identical: **207 tools / 811,880 proxy tokens**; the three selected tool definitions are 5,272 tokens in each version. Whole recorded payload counts include that catalog, open/close and all query requests/replies. Counts use cl100k_base over actual SDK JSON bodies; initialize/framing, actual model consumption and billing are unmeasured. Per-request CPU is unmeasured; CPU/RSS above cover the full completed workflow. Cold goal timings were 214.84 / 143.62 s (one observation per version), dominated by Ghidra startup. OS caches and shared-host load changed substantially; these cold/CPU/RSS differences establish no causal speedup or memory improvement. Warm noncanonical samples combine three spellings, two repetitions each. Node compile caching was disabled; warm means the same live Ghidra session and validated snapshot. Family RSS is sampled at 200 ms and may double-count shared pages/miss brief children. Heavy jobs ran serially on two CPU cores, JVM heap 2 GiB, own-family guard 7 GiB within the approved 8 GiB overall budget; validation peaked at **4.46 GiB**, with no guard/OOM event. ## Validation performed - `npm run test:focused -- src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraFunctionValues.test.ts src/domain/native/nativeAotPe.test.ts src/domain/native/nativeAotPe.budget.test.ts tests/boundary/mcp/ghidraNativeApiMcp.test.ts` — **174 passed**. - `npm run check` — passed typecheck, lint, formatting, unused-code/metadata/module checks and architecture guards. - `npm run docs:check` — passed, including five documentation consistency tests. ## Coverage limits Local execution is Linux x86_64 and actual headless Ghidra; no Windows/macOS runtime, attached GUI, optional NativeAOT addon, or target-program execution is claimed. The compiled fixture is a genuine compiler-produced C# test program, not a production application parity claim. Engine-provided-metadata I/O avoidance is verified at the production boundary with a controlled client. The larger Microsoft ILCompiler 8.0.0 binary (11.56 MB) exceeded the existing 330 s Ghidra startup deadline under two CPU cores / 2 GiB JVM, before metadata lookup. No successful larger-application or cold-analysis speedup claim is made. Failed attempts and unrelated unsupported-layout observations are retained in the local test recap. ## Author readiness Human owner: @N0zoM1z0 The human owner explicitly authorized delegated review, proxy completion of these declarations, ready-for-review publication and normal merge for this batch. The agent reviewed the final diff and evidence; this records that delegation and does not assert a separate manual review occurred. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:11:34 +00:00