[PR #1991] [MERGED] feat(ghidra): admit verified MIPS16 shared ELF images #4954

Closed
opened 2026-10-31 00:57:19 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1991
Author: @N0zoM1z0
Created: 10/11/2026
Status: ✅ Merged
Merged: 10/11/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/1851-mips16-shared-elf


📝 Commits (2)

  • 556cb66 feat(ghidra): admit verified MIPS16 shared ELF images
  • 303a7fa chore: sync main into MIPS16 shared ELF work

📊 Changes

19 files changed (+1460 additions, -37 deletions)

View changed files

📝 bridge/ghidra/ReaGhidraBridge.java (+13 -0)
📝 docs/ghidra-mips.md (+61 -3)
📝 docs/native-investigation.md (+8 -2)
📝 package.json (+1 -0)
➕ scripts/verify-real-ghidra-mips-shared.mjs (+338 -0)
📝 src/application/BinaryTargetResolver.ts (+12 -7)
📝 src/application/MipsElfMetadata.ts (+55 -9)
➕ src/application/MipsElfSharedLoad.ts (+220 -0)
📝 src/domain/binaryTargetTypes.ts (+9 -0)
📝 src/domain/native/nativeInstruction.ts (+17 -1)
📝 src/ghidra/GhidraAnalysisProfile.ts (+31 -6)
📝 src/ghidra/GhidraMipsProfile.test.ts (+125 -0)
📝 src/ghidra/GhidraMipsProfile.ts (+82 -6)
📝 src/ghidra/GhidraProviderClient.ts (+7 -1)
📝 tests/boundary/filesystem/mipsAbiFlags.test.ts (+2 -2)
➕ tests/boundary/filesystem/mipsSharedLoad.test.ts (+260 -0)
➕ tests/conformance/ghidra/mips16-recordless.ld (+22 -0)
➕ tests/conformance/ghidra/mips16-shared-fixture.mjs (+156 -0)
➕ tests/conformance/ghidra/mips16-shared.S (+41 -0)

📄 Description

Summary

REA can analyze unchanged ELF32 o32 MIPS32r2 MIPS16 shared images when their complete source declarations fit a verified Ghidra load profile. Mixed MIPS32/MIPS16 instructions retain their per-address decoder context.

Fixes #1851

Problem and expected behavior

The standard MIPS lane refuses ET_DYN and compressed encodings, preventing native investigation of shared ELF artifacts. Admission must establish the source load, relocation and mode semantics rather than edit headers or infer compatibility from the architecture name.

Change and scope

  • Extend the existing source metadata reader to examine complete program/section declarations and stream dynamic REL records in 64 KiB chunks. Preserve unknown ABI declarations and partial-table reasons.
  • Admit the separate ELF32 ET_DYN/o32/MIPS32r2/MIPS16 profile with uniquely mapped load ranges, R_MIPS_NONE and symbol-index-zero R_MIPS_REL32. Refuse symbol-bearing REL32, extra section relocation tables, RELA/PLT, other relocation types and incomplete declarations.
  • Pin the endian-matched generic MIPS32 language and default compiler specification. Preserve matching caller overrides and explicitly refuse incompatible choices.
  • Expose Ghidra's per-address ISA_MODE separately from the program-wide mode label, preserving null when no context value is available.
  • Add source-owned GNU controls and an optional real CLI/MCP verifier covering both byte orders, mixed modes, declared/absent ABI records and relocation refusals.

Contract and boundary impact

  • Semantic owner and earliest changed stage: application ELF source inspection owns declaration coverage; the Ghidra adapter owns admission, language selection and decoder interpretation.
  • CLI and MCP/tool-catalog contract: existing operations remain unchanged; native instruction results gain optional decoder_context, e.g. [{"register":"ISA_MODE","value":"1","source":"ghidra-program-context"}]. A null value preserves unavailable context; omitted fields remain valid for older captures/providers.
  • Provider, bridge, target-format, or platform compatibility: the bounded profile applies to Ghidra on Linux/macOS. Existing ET_EXEC/ET_REL, other providers and Windows P0 retain their admission boundaries.
  • Evidence, artifact, provenance, or reconstruction contract: preserve original bytes/digests, ABI facts and relocation declarations. Profiles commit the shared-load and context policy; addresses remain analysis coordinates. Loaded-image relocation bytes/status and external-binding limitations remain visible.
  • Process execution, authorization, cleanup, or containment impact: existing owned ephemeral Ghidra lifecycle; no target execution, byte modification or GUI authority. The verifier checks process cleanup and source digests.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: one optional verify:ghidra:mips-shared package script; ignored catalogs regenerate from contracts. No binary, generated catalog or dependency changes. Tools are caller-supplied.

Evidence and regression coverage

  • Tests added or updated: endian-aware source declaration parsing, late relocation records, malformed/duplicate/incomplete tables, overlapping mappings, extra section relocations, cancellation identity, missing declarations, configured-language preservation and profile/session mismatch.
  • Base reproduction or other evidence: main's provider admission refuses the original unchanged shared image. On the patch, real CLI/MCP analysis of that image recovered 412 procedures, selected instruction/pseudocode evidence and a retained Evidence export, with unchanged source digest. Independent source/load comparison verified 169 nonzero relocation/GOT words; unresolved external entries remain unknown.
  • Real negative control: Ghidra 12.1.4 normalizes a symbol-bearing MIPS16 REL32 function pointer to an even address in both byte orders. The profile refuses that shape instead of silently repairing its mode tag.
  • User-visible CLI/MCP output: mode retains default; compressed instructions report ISA_MODE 1, while the standard-mode control preserves an observed null. Source-owned local pointer tags survive relocation; snapshot round trips preserve decoder context.
  • Remaining proof gaps: compressed floating-point behavioral conformance, other relocation families, external GOT resolution, other Ghidra versions, macOS real execution, and other native providers were not verified.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run test:focused -- eight relevant source/boundary files — 152 passed after main integration, including existing MIPS ABI and WASM diagnostics.
  • npm run verify:ghidra:mips-shared — eight real public CLI/MCP controls passed using Ghidra 12.1.4, JDK 21 and GNU cross-binutils 2.40 on Linux x64. Covered four positive and four refused controls, CLI/MCP parity, source symbols/instruction bytes, local pointer tags, snapshot restore, service responsiveness and zero surviving owned descendants.
  • Original-artifact public CLI/MCP replay — passed with advertised schemas, instruction/pseudocode/export parity and unchanged source bytes.
  • The eight-control run used one CPU, peaked at approximately 1.50 GiB, and recorded zero memory-limit, OOM or OOM-kill events.
  • npm run check:fast and npm run docs:check — passed on the integrated head. npm run check:ci passed the full static gate before the main sync; CI validates the final published head.

Compatibility, safety, and release

  • Breaking changes or migration steps: none; additive result field and narrowly expanded admission.
  • Real Hopper/Ghidra, browser, or OS coverage: genuine Linux x64 Ghidra 12.1.4; other providers/platforms unverified for the new lane.
  • Package or release metadata impact: optional developer verifier script only; no installer or runtime dependency additions.
  • Security, privacy, process, or containment review: source-file identity and bytes are preserved; temporary read-only analysis uses existing process ownership and cleanup. Source bounds and complete declaration coverage are checked before admission.

Review checklist

  • This PR addresses a concrete problem or an agreed enhancement.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1991 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/11/2026 **Status:** ✅ Merged **Merged:** 10/11/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/1851-mips16-shared-elf` --- ### 📝 Commits (2) - [`556cb66`](https://github.com/morluto/rea/commit/556cb66090dfeff6199a32da4dbd2d8b7bdcb37c) feat(ghidra): admit verified MIPS16 shared ELF images - [`303a7fa`](https://github.com/morluto/rea/commit/303a7fa50d1d00745eabcce78f76bb2113dcc33f) chore: sync main into MIPS16 shared ELF work ### 📊 Changes **19 files changed** (+1460 additions, -37 deletions) <details> <summary>View changed files</summary> 📝 `bridge/ghidra/ReaGhidraBridge.java` (+13 -0) 📝 `docs/ghidra-mips.md` (+61 -3) 📝 `docs/native-investigation.md` (+8 -2) 📝 `package.json` (+1 -0) ➕ `scripts/verify-real-ghidra-mips-shared.mjs` (+338 -0) 📝 `src/application/BinaryTargetResolver.ts` (+12 -7) 📝 `src/application/MipsElfMetadata.ts` (+55 -9) ➕ `src/application/MipsElfSharedLoad.ts` (+220 -0) 📝 `src/domain/binaryTargetTypes.ts` (+9 -0) 📝 `src/domain/native/nativeInstruction.ts` (+17 -1) 📝 `src/ghidra/GhidraAnalysisProfile.ts` (+31 -6) 📝 `src/ghidra/GhidraMipsProfile.test.ts` (+125 -0) 📝 `src/ghidra/GhidraMipsProfile.ts` (+82 -6) 📝 `src/ghidra/GhidraProviderClient.ts` (+7 -1) 📝 `tests/boundary/filesystem/mipsAbiFlags.test.ts` (+2 -2) ➕ `tests/boundary/filesystem/mipsSharedLoad.test.ts` (+260 -0) ➕ `tests/conformance/ghidra/mips16-recordless.ld` (+22 -0) ➕ `tests/conformance/ghidra/mips16-shared-fixture.mjs` (+156 -0) ➕ `tests/conformance/ghidra/mips16-shared.S` (+41 -0) </details> ### 📄 Description ## Summary REA can analyze unchanged ELF32 o32 MIPS32r2 MIPS16 shared images when their complete source declarations fit a verified Ghidra load profile. Mixed MIPS32/MIPS16 instructions retain their per-address decoder context. Fixes #1851 ## Problem and expected behavior The standard MIPS lane refuses ET_DYN and compressed encodings, preventing native investigation of shared ELF artifacts. Admission must establish the source load, relocation and mode semantics rather than edit headers or infer compatibility from the architecture name. ## Change and scope - Extend the existing source metadata reader to examine complete program/section declarations and stream dynamic REL records in 64 KiB chunks. Preserve unknown ABI declarations and partial-table reasons. - Admit the separate ELF32 ET_DYN/o32/MIPS32r2/MIPS16 profile with uniquely mapped load ranges, R_MIPS_NONE and symbol-index-zero R_MIPS_REL32. Refuse symbol-bearing REL32, extra section relocation tables, RELA/PLT, other relocation types and incomplete declarations. - Pin the endian-matched generic MIPS32 language and default compiler specification. Preserve matching caller overrides and explicitly refuse incompatible choices. - Expose Ghidra's per-address ISA_MODE separately from the program-wide mode label, preserving null when no context value is available. - Add source-owned GNU controls and an optional real CLI/MCP verifier covering both byte orders, mixed modes, declared/absent ABI records and relocation refusals. ## Contract and boundary impact - Semantic owner and earliest changed stage: application ELF source inspection owns declaration coverage; the Ghidra adapter owns admission, language selection and decoder interpretation. - CLI and MCP/tool-catalog contract: existing operations remain unchanged; native instruction results gain optional `decoder_context`, e.g. `[{"register":"ISA_MODE","value":"1","source":"ghidra-program-context"}]`. A null value preserves unavailable context; omitted fields remain valid for older captures/providers. - Provider, bridge, target-format, or platform compatibility: the bounded profile applies to Ghidra on Linux/macOS. Existing ET_EXEC/ET_REL, other providers and Windows P0 retain their admission boundaries. - Evidence, artifact, provenance, or reconstruction contract: preserve original bytes/digests, ABI facts and relocation declarations. Profiles commit the shared-load and context policy; addresses remain analysis coordinates. Loaded-image relocation bytes/status and external-binding limitations remain visible. - Process execution, authorization, cleanup, or containment impact: existing owned ephemeral Ghidra lifecycle; no target execution, byte modification or GUI authority. The verifier checks process cleanup and source digests. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: one optional `verify:ghidra:mips-shared` package script; ignored catalogs regenerate from contracts. No binary, generated catalog or dependency changes. Tools are caller-supplied. ## Evidence and regression coverage - Tests added or updated: endian-aware source declaration parsing, late relocation records, malformed/duplicate/incomplete tables, overlapping mappings, extra section relocations, cancellation identity, missing declarations, configured-language preservation and profile/session mismatch. - Base reproduction or other evidence: main's provider admission refuses the original unchanged shared image. On the patch, real CLI/MCP analysis of that image recovered 412 procedures, selected instruction/pseudocode evidence and a retained Evidence export, with unchanged source digest. Independent source/load comparison verified 169 nonzero relocation/GOT words; unresolved external entries remain unknown. - Real negative control: Ghidra 12.1.4 normalizes a symbol-bearing MIPS16 REL32 function pointer to an even address in both byte orders. The profile refuses that shape instead of silently repairing its mode tag. - User-visible CLI/MCP output: `mode` retains `default`; compressed instructions report ISA_MODE `1`, while the standard-mode control preserves an observed null. Source-owned local pointer tags survive relocation; snapshot round trips preserve decoder context. - Remaining proof gaps: compressed floating-point behavioral conformance, other relocation families, external GOT resolution, other Ghidra versions, macOS real execution, and other native providers were not verified. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:focused --` eight relevant source/boundary files — 152 passed after main integration, including existing MIPS ABI and WASM diagnostics. - `npm run verify:ghidra:mips-shared` — eight real public CLI/MCP controls passed using Ghidra 12.1.4, JDK 21 and GNU cross-binutils 2.40 on Linux x64. Covered four positive and four refused controls, CLI/MCP parity, source symbols/instruction bytes, local pointer tags, snapshot restore, service responsiveness and zero surviving owned descendants. - Original-artifact public CLI/MCP replay — passed with advertised schemas, instruction/pseudocode/export parity and unchanged source bytes. - The eight-control run used one CPU, peaked at approximately 1.50 GiB, and recorded zero memory-limit, OOM or OOM-kill events. - `npm run check:fast` and `npm run docs:check` — passed on the integrated head. `npm run check:ci` passed the full static gate before the main sync; CI validates the final published head. ## Compatibility, safety, and release - Breaking changes or migration steps: none; additive result field and narrowly expanded admission. - Real Hopper/Ghidra, browser, or OS coverage: genuine Linux x64 Ghidra 12.1.4; other providers/platforms unverified for the new lane. - Package or release metadata impact: optional developer verifier script only; no installer or runtime dependency additions. - Security, privacy, process, or containment review: source-file identity and bytes are preserved; temporary read-only analysis uses existing process ownership and cleanup. Source bounds and complete declaration coverage are checked before admission. ## Review checklist - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 00:57:19 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#4954
No description provided.