[PR #556] [MERGED] feat(ghidra): add DOS MZ analysis and complete function extents #2072

New issue

Closed

opened 2026-10-16 21:13:56 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:13:56 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/556
Author: @N0zoM1z0
Created: 10/5/2026
Status: ✅ Merged
Merged: 10/5/2026
Merged by: @N0zoM1z0

Base: main ← Head: feat/ghidra-dos-analysis

📝 Commits (2)

157415a feat(ghidra): add DOS MZ analysis and complete function extents
ef7281b chore: merge latest main

📊 Changes

55 files changed (+2206 additions, -92 deletions)

View changed files

📝 .gitignore (+3 -0)
📝 .prettierignore (+3 -0)
📝 README.md (+2 -0)
📝 bridge/ghidra/ReaGhidraBridge.java (+67 -1)
➕ docs/ghidra-dos.md (+105 -0)
📝 docs/product-catalog.json (+2 -2)
📝 docs/testing.md (+12 -0)
📝 docs/verification/managed-conformance-manifest.json (+2 -2)
📝 package.json (+1 -0)
➕ scripts/verify-real-ghidra-dos.mjs (+502 -0)
📝 skills/reverse-engineer-anything/SKILL.md (+1 -1)
📝 src/application/ArtifactGraphConstruction.ts (+31 -6)
📝 src/application/ArtifactInventory/classify.ts (+7 -14)
📝 src/application/ArtifactInventory/hash.ts (+8 -2)
📝 src/application/ArtifactInventory/scanReader.ts (+1 -1)
📝 src/application/BinaryTargetResolver.ts (+35 -5)
📝 src/contracts/functionWorkflowToolContracts.ts (+1 -1)
📝 src/contracts/toolContracts.ts (+1 -1)
📝 src/contracts/toolOutputSchemaPrimitives.ts (+6 -0)
📝 src/domain/analysisSnapshot.test.ts (+44 -0)

...and 35 more files

📄 Description

Summary

Admit validated DOS MZ images through the existing Ghidra read-only adapter, using an explicit 16-bit real-mode loader/language/compiler profile and authenticated session checks.
Return complete observed function body ranges, inclusive endpoints, owned byte counts, enclosing spans, contiguity and entry membership through existing function and reference contracts. Providers without range evidence report an explicit unknown. Commit the body representation policy in the Ghidra profile so older length-only snapshot profiles do not collide.
Preserve DOS identity through artifact inventory, runtime identification, Evidence and snapshots. Keep PE classification and the Windows native PE boundary intact; Hopper declines DOS MZ.
Add a source-generated real DOS verification lane covering near/far relocation, actual decompilation, disjoint function bodies, containing-procedure queries, CLI/MCP observations and cleanup.

Example: observed function body

A generic source-generated fixture returns this body through procedure_info and analyze_function:

{
  "available": true,
  "provenance": "ghidra-function-body-address-set",
  "ranges": [
    { "start": "0x10000", "end": "0x1000d" },
    { "start": "0x10060", "end": "0x10065" }
  ],
  "total_bytes": 20,
  "span_bytes": 102,
  "non_contiguous": true,
  "contains_entry": true
}

Ranges are Ghidra database observations. Gaps are not owned bytes. No new tool or script execution authority is added.

Verification

npm run check:pr passed: 376 test files, 2056 tests passed, 2 skipped; typecheck, lint, formatting, generated contracts and document checks passed.
npm run verify:ghidra:dos passed on Linux x64 with actual Ghidra 12.1.4 / JDK 21: near/far calls and relocation, nonempty decompilation, complete disjoint ranges, inclusive containing-procedure endpoints and gap rejection, stable CLI/MCP dossiers, unchanged input and session/process cleanup.
env -u GHIDRA_INSTALL_DIR TMPDIR=/tmp npm run verify:package passed with the isolated Node 24 toolchain: packaged CLI/MCP, optional PTY absence, complete catalog/schema validation and lifecycle cleanup. The verifier uses a fake Hopper seam; removing the local Ghidra configuration avoids an unrelated auto-provider ambiguity, and using an outside-repository temporary directory prevents ancestor dependency resolution from defeating the optional-dependency absence control.
git diff --check passed; package manifest review found no local reference inputs or analysis captures.

Existing host Ghidra lane failure

npm run verify:ghidra stops at the existing dense jump-table case-label assertion. A clean, unmodified e793e124 reproduces the same failure with GCC 12.2 and the supported REA_CC=clang alternative (Clang 14.0.6). Ghidra emits 41 destinations but 40 labels: the extra destination is the default branch. The existing adapter requires equal lengths before matching labels, so it retains explicit unknowns. Target mappings, inferred table candidate, limitations and residual unknowns match the baseline; the relevant adapter logic and fixture are unchanged in this PR. The broader host lane is therefore not reported as passing. Its metadata correlation gap needs a separate correction.

Coverage

Linux x64 with Ghidra 12.1.4 / JDK 21. DOS MZ on macOS remains unverified; DOS is outside Windows P0. Static analysis does not emulate BIOS/DOS/device behavior. Raw COM and unsupported DOS extenders are excluded. Packed targets require a separately identified unpacked artifact. The DOS lane compares stable dossier observations and retains provider p-code values; it does not assert cross-process identity of raw high-pcode selectors. Real Hopper verification was not performed.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/556 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/5/2026 **Status:** ✅ Merged **Merged:** 10/5/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `feat/ghidra-dos-analysis` --- ### 📝 Commits (2) - [`157415a`](https://github.com/morluto/rea/commit/157415a3dfa0819eb4d196e9dec0252a8ce7ad1d) feat(ghidra): add DOS MZ analysis and complete function extents - [`ef7281b`](https://github.com/morluto/rea/commit/ef7281bae46a33a411c7794996f2119690d3223b) chore: merge latest main ### 📊 Changes **55 files changed** (+2206 additions, -92 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+3 -0) 📝 `.prettierignore` (+3 -0) 📝 `README.md` (+2 -0) 📝 `bridge/ghidra/ReaGhidraBridge.java` (+67 -1) ➕ `docs/ghidra-dos.md` (+105 -0) 📝 `docs/product-catalog.json` (+2 -2) 📝 `docs/testing.md` (+12 -0) 📝 `docs/verification/managed-conformance-manifest.json` (+2 -2) 📝 `package.json` (+1 -0) ➕ `scripts/verify-real-ghidra-dos.mjs` (+502 -0) 📝 `skills/reverse-engineer-anything/SKILL.md` (+1 -1) 📝 `src/application/ArtifactGraphConstruction.ts` (+31 -6) 📝 `src/application/ArtifactInventory/classify.ts` (+7 -14) 📝 `src/application/ArtifactInventory/hash.ts` (+8 -2) 📝 `src/application/ArtifactInventory/scanReader.ts` (+1 -1) 📝 `src/application/BinaryTargetResolver.ts` (+35 -5) 📝 `src/contracts/functionWorkflowToolContracts.ts` (+1 -1) 📝 `src/contracts/toolContracts.ts` (+1 -1) 📝 `src/contracts/toolOutputSchemaPrimitives.ts` (+6 -0) 📝 `src/domain/analysisSnapshot.test.ts` (+44 -0) _...and 35 more files_ </details> ### 📄 Description ## Summary - Admit validated DOS MZ images through the existing Ghidra read-only adapter, using an explicit 16-bit real-mode loader/language/compiler profile and authenticated session checks. - Return complete observed function body ranges, inclusive endpoints, owned byte counts, enclosing spans, contiguity and entry membership through existing function and reference contracts. Providers without range evidence report an explicit unknown. Commit the body representation policy in the Ghidra profile so older length-only snapshot profiles do not collide. - Preserve DOS identity through artifact inventory, runtime identification, Evidence and snapshots. Keep PE classification and the Windows native PE boundary intact; Hopper declines DOS MZ. - Add a source-generated real DOS verification lane covering near/far relocation, actual decompilation, disjoint function bodies, containing-procedure queries, CLI/MCP observations and cleanup. ## Example: observed function body A generic source-generated fixture returns this body through `procedure_info` and `analyze_function`: ```json { "available": true, "provenance": "ghidra-function-body-address-set", "ranges": [ { "start": "0x10000", "end": "0x1000d" }, { "start": "0x10060", "end": "0x10065" } ], "total_bytes": 20, "span_bytes": 102, "non_contiguous": true, "contains_entry": true } ``` Ranges are Ghidra database observations. Gaps are not owned bytes. No new tool or script execution authority is added. ## Verification - `npm run check:pr` passed: 376 test files, 2056 tests passed, 2 skipped; typecheck, lint, formatting, generated contracts and document checks passed. - `npm run verify:ghidra:dos` passed on Linux x64 with actual Ghidra 12.1.4 / JDK 21: near/far calls and relocation, nonempty decompilation, complete disjoint ranges, inclusive containing-procedure endpoints and gap rejection, stable CLI/MCP dossiers, unchanged input and session/process cleanup. - `env -u GHIDRA_INSTALL_DIR TMPDIR=/tmp npm run verify:package` passed with the isolated Node 24 toolchain: packaged CLI/MCP, optional PTY absence, complete catalog/schema validation and lifecycle cleanup. The verifier uses a fake Hopper seam; removing the local Ghidra configuration avoids an unrelated auto-provider ambiguity, and using an outside-repository temporary directory prevents ancestor dependency resolution from defeating the optional-dependency absence control. - `git diff --check` passed; package manifest review found no local reference inputs or analysis captures. ### Existing host Ghidra lane failure `npm run verify:ghidra` stops at the existing dense jump-table case-label assertion. A clean, unmodified `e793e124` reproduces the same failure with GCC 12.2 and the supported `REA_CC=clang` alternative (Clang 14.0.6). Ghidra emits 41 destinations but 40 labels: the extra destination is the default branch. The existing adapter requires equal lengths before matching labels, so it retains explicit unknowns. Target mappings, inferred table candidate, limitations and residual unknowns match the baseline; the relevant adapter logic and fixture are unchanged in this PR. The broader host lane is therefore **not reported as passing**. Its metadata correlation gap needs a separate correction. ## Coverage Linux x64 with Ghidra 12.1.4 / JDK 21. DOS MZ on macOS remains unverified; DOS is outside Windows P0. Static analysis does not emulate BIOS/DOS/device behavior. Raw COM and unsupported DOS extenders are excluded. Packed targets require a separately identified unpacked artifact. The DOS lane compares stable dossier observations and retains provider p-code values; it does not assert cross-process identity of raw high-pcode selectors. Real Hopper verification was not performed. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:13:56 +00:00