[PR #3818] feat(ghidra): apply a caller-selected PDB that matches the target #5923

Open
opened 2026-10-31 01:10:28 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3818
Author: @whalemenace
Created: 10/15/2026
Status: 🔄 Open

Base: main ← Head: feat/ghidra-pdb-companion


📝 Commits (4)

  • 262c3c6 feat(ghidra): apply a caller-selected PDB that matches the target
  • 90c653b test(ghidra): verify caller-selected PDBs against real Ghidra
  • a6d69e0 docs(ghidra): document caller-selected PDBs
  • 5197a44 fix(ghidra): keep caller PDBs out of existing projects

📊 Changes

24 files changed (+1482 additions, -17 deletions)

View changed files

➕ bridge/ghidra/ReaGhidraApplyPdb.java (+107 -0)
📝 bridge/ghidra/ReaGhidraBridge.java (+5 -0)
📝 docs/ghidra-provider.md (+39 -3)
📝 docs/installation.md (+9 -0)
📝 docs/testing.md (+12 -11)
📝 docs/user-visible-errors.csv (+1 -0)
📝 docs/windows-ghidra-p0.md (+6 -0)
📝 package.json (+2 -0)
➕ scripts/verify-real-ghidra-pdb.mjs (+305 -0)
📝 src/config/environment.ts (+22 -0)
📝 src/config/parseConfig.ts (+3 -0)
📝 src/config/types.ts (+2 -0)
📝 src/ghidra/GhidraLauncher.ts (+31 -0)
➕ src/ghidra/GhidraPdbCompanion.test.ts (+451 -0)
➕ src/ghidra/GhidraPdbCompanion.ts (+343 -0)
📝 src/ghidra/GhidraPeProfile.ts (+1 -1)
📝 src/ghidra/GhidraProjectHistory.ts (+3 -0)
📝 src/ghidra/GhidraProvider.behavior.test.ts (+9 -0)
📝 src/ghidra/GhidraProvider.ts (+14 -1)
📝 src/ghidra/GhidraProviderClient.ts (+30 -0)

...and 4 more files

📄 Description

Summary

Let a caller hand the Ghidra provider the PDB that belongs to the target (REA_GHIDRA_PDB_FILE). Today REA states "REA does not admit a caller-selected PDB companion", so a shipped Windows binary whose vendor left its PDB in the install (it happens) is analysed without its own symbols even though Ghidra can apply them. Related to the use case in #3751 (large game binaries where names are the expensive part).

Problem and expected behavior

  • Today: the PDB Universal analyzer only finds a PDB through Ghidra's configured symbol paths; a PDB sitting beside the target, or one the caller moved into a reference folder, is never applied, and REA reports "Original names and types may be missing".
  • Expected: with REA_GHIDRA_PDB_FILE=<absolute path>, REA verifies before Ghidra starts that the PDB's GUID and age equal the target's RSDS CodeView record (fail closed, unsupported_target naming both identities), then a pre-script verifies again with Ghidra's own PDB reader and hands the file to the analyzer. The profile records pdb_companion and the session limitations say what was applied and that the symbols are caller-supplied debug information, not Ghidra discoveries.

Change and scope

  • src/ghidra/GhidraPdbCompanion.ts (new): reads the MSF 7.00 superblock, stream directory, PDB info stream (1) and DBI header (3) to get GUID and age without loading the PDB (DBI age wins when non-zero, Ghidra's rule); hashes the file through the same no-follow handle; builds the profile commitment {format: rea-ghidra-pdb-v1, path, configured_path, sha256, bytes, guid, age}; compares against target.codeView (REA's existing PE parse of the RSDS record). Refusals: GUID/age mismatch, PE without an RSDS GUID, non-PE target (unsupported_target); unreadable or non-MSF file (ProviderAdapterError, setting: REA_GHIDRA_PDB_FILE).
  • bridge/ghidra/ReaGhidraApplyPdb.java (new pre-script, modelled on ReaGhidraApplySeeds.java): re-hashes the file, reads GUID/age with PdbParser / getIdentifiers(), compares with the committed values and the Program's PdbProgramAttributes, and only then calls PdbUniversalAnalyzer.setPdbFileOption. Any mismatch throws and analyzeHeadless aborts before analysis. Outcome stored in the Program Info option "REA PDB companion"; the bridge reports it as pdb_companion in the handshake. (Ghidra 12.1.4's PdbAnalyzerCommon.findPdb applies a forced PDB without any GUID check, so REA's check is the only gate.)
  • Session (GhidraProviderClient.ts, GhidraProvider.ts, GhidraSessionValues.ts): fail-closed outcome check — report must match the commitment's sha256/GUID/age, the Program's GUID/age must match, and Ghidra must report PDB Loaded true; otherwise ProviderAdapterError and the session closes. New limitation line "PDB applied: … caller-supplied debug information, not Ghidra discoveries; a matching GUID and age identify the build, not the completeness of the PDB". The old "does not admit a caller-selected PDB companion" sentence now reads "To apply a PDB from this exact build, set REA_GHIDRA_PDB_FILE; REA does not search for one."
  • Config (environment.ts, parseConfig.ts, types.ts): the setting; REA_GHIDRA_DISABLED_ANALYZERS may not contain PDB Universal while it is set (ConfigurationError).
  • package.json: ReaGhidraApplyPdb.java added to files (the package would otherwise ship without it); verify:ghidra:pdb script.
  • Not included: symbol-server lookup, PDB search paths, applying a PDB to a persistent project (if #3817 lands first, REA_GHIDRA_PDB_FILE should join its refused import-time settings). Scope decision: not applicable — static analysis input, no execution, injection or mutation.

Contract and boundary impact

  • Semantic owner and earliest changed stage: Ghidra provider; analysis-profile resolution.
  • Contribution scope fit and linked maintainer decision: in scope; not applicable.
  • CLI and MCP/tool-catalog contract: no new tools; analysis_profile.parameters.pdb_companion added when the setting is present; error contract row 90.8 in docs/user-visible-errors.csv (no new code).
  • Provider, bridge, target-format, or platform compatibility: PE targets only; Ghidra 12.1.x; verified Linux x64 with an LLD-linked PE+PDB pair. MSVC-produced PDBs, very large PDBs and Windows P0 unverified (documented).
  • Evidence, artifact, provenance, or reconstruction contract: the PDB's identity (sha256, GUID, age) is part of the profile; symbols from it are labelled caller-supplied in the limitations.
  • Target selection, authority, upstream behavior, and cleanup details: the caller selects the file; it is read only (no-follow handle; Ghidra reads it through the analyzer). Nothing is written next to the target.
  • Generated metadata, package, or installation impact: files list + one npm script; no catalog change.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: src/ghidra/GhidraPdbCompanion.test.ts (MSF parsing incl. truncated/invalid containers, GUID normalisation, DBI-age rule, commitment and refusal paths, outcome check), tests/boundary/providers/ghidra/ghidraLauncher.test.ts (pre-script argument order after seeds and before analyzer options; the disabled-analyzer conflict), tests/conformance/ghidra/pdb-companion.c (fixture source).

  • Base reproduction or other evidence: npm run verify:ghidra:pdb (new, scripts/verify-real-ghidra-pdb.mjs, needs REA_CLANG / REA_LLD_LINK as the cross-format lane does): links two variants of the fixture into x86-64 PE + PDB pairs (/debug /pdb:), moves each PDB away from the path recorded in the image, then through rea mcp: control without the setting → rea_pdb_scale absent from list_procedures; matching PDB → rea_pdb_scale and rea_pdb_entry present, pseudocode int __cdecl rea_pdb_entry(void) { … iVar1 = rea_pdb_scale(4,5); … }, pdb_companion GUID/age equal the RSDS record and sha256 equals the file; the other link's PDB → unsupported_target "The PDB …/other-renamed.pdb has GUID 13445a73-… and age 1, but the target's CodeView record declares GUID 1a2550ec-… and age 1. REA refuses to apply debug information from another build."; direct analyzeHeadless with a wrong committed sha → pre-script refusal, "Processing aborted as a result of pre-script."

  • User-visible CLI/MCP output: as quoted above.

  • Remaining proof gaps: MSVC-linked PDBs (the lane uses LLD's PDB writer); PDBs whose stream-directory block map spans more than one block are refused with a format error rather than parsed (32 KiB blocks reach that above 2 TiB); Windows P0 and macOS not run; time and heap for multi-GB PDBs (hashed twice: TypeScript at profile time, Java in the pre-script) not measured.

  • Observed, derived, and inferred claims remain distinguishable.

  • Artifact identity, source provenance, and failed attempts remain preserved.

  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

All on Linux x64 (Fedora, Node 22.23, Ghidra 12.1.4, Temurin 21.0.12.1, zig 0.17.0's clang + lld-link for the fixture), final head of this branch:

  • npm run typecheck — clean
  • npm run lint — 0 errors, 1429 warnings (baseline count on upstream main)
  • oxfmt --check — clean
  • npm run verify:test-discovery — passed
  • npm run docs:check — passed
  • focused suites (src/ghidra, src/config, tests/boundary/providers/ghidra) — 752 tests pass
  • npm run test:prepare && npm run test:run — Test files 3 failed / 1284 passed / 14 skipped; Tests 3 failed / 15023 passed / 175 skipped. The 3 failures (packageImportAliases, runnerCancellation, closedStdout) fail identically on a clean upstream/main worktree on the same host.
  • npm run verify:ghidra:pdb — exit 0 (details above)

Compatibility, safety, and release

  • Breaking changes or migration steps: none; without the setting behaviour is unchanged (one limitation sentence reworded).
  • Real Hopper/Ghidra, browser, or OS coverage: real Ghidra 12.1.4 on Linux x64. Windows P0 and macOS: not run.
  • Package or release metadata impact: files gains the new Java script; one new npm script.
  • Security, privacy, process, or containment review: the PDB is read, never executed; identity is enforced twice (REA's MSF reader against the PE's RSDS record, then Ghidra's reader in the pre-script) and fails closed; the file is opened no-follow.

Author readiness

Human owner: @whalemenace

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

(Opened as a draft; AI-assisted implementation, the human owner completes the declarations.)

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3818 **Author:** [@whalemenace](https://github.com/whalemenace) **Created:** 10/15/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/ghidra-pdb-companion` --- ### 📝 Commits (4) - [`262c3c6`](https://github.com/morluto/rea/commit/262c3c6842d0972710c8414c410911d2b8d5c9a8) feat(ghidra): apply a caller-selected PDB that matches the target - [`90c653b`](https://github.com/morluto/rea/commit/90c653be5ee964dd6977adef15044f48b22f8338) test(ghidra): verify caller-selected PDBs against real Ghidra - [`a6d69e0`](https://github.com/morluto/rea/commit/a6d69e0f1576494998887125e6b2f2fe2f990af2) docs(ghidra): document caller-selected PDBs - [`5197a44`](https://github.com/morluto/rea/commit/5197a44a46f3c170f28d0331187e034e5713cddf) fix(ghidra): keep caller PDBs out of existing projects ### 📊 Changes **24 files changed** (+1482 additions, -17 deletions) <details> <summary>View changed files</summary> ➕ `bridge/ghidra/ReaGhidraApplyPdb.java` (+107 -0) 📝 `bridge/ghidra/ReaGhidraBridge.java` (+5 -0) 📝 `docs/ghidra-provider.md` (+39 -3) 📝 `docs/installation.md` (+9 -0) 📝 `docs/testing.md` (+12 -11) 📝 `docs/user-visible-errors.csv` (+1 -0) 📝 `docs/windows-ghidra-p0.md` (+6 -0) 📝 `package.json` (+2 -0) ➕ `scripts/verify-real-ghidra-pdb.mjs` (+305 -0) 📝 `src/config/environment.ts` (+22 -0) 📝 `src/config/parseConfig.ts` (+3 -0) 📝 `src/config/types.ts` (+2 -0) 📝 `src/ghidra/GhidraLauncher.ts` (+31 -0) ➕ `src/ghidra/GhidraPdbCompanion.test.ts` (+451 -0) ➕ `src/ghidra/GhidraPdbCompanion.ts` (+343 -0) 📝 `src/ghidra/GhidraPeProfile.ts` (+1 -1) 📝 `src/ghidra/GhidraProjectHistory.ts` (+3 -0) 📝 `src/ghidra/GhidraProvider.behavior.test.ts` (+9 -0) 📝 `src/ghidra/GhidraProvider.ts` (+14 -1) 📝 `src/ghidra/GhidraProviderClient.ts` (+30 -0) _...and 4 more files_ </details> ### 📄 Description ## Summary Let a caller hand the Ghidra provider the PDB that belongs to the target (`REA_GHIDRA_PDB_FILE`). Today REA states "REA does not admit a caller-selected PDB companion", so a shipped Windows binary whose vendor left its PDB in the install (it happens) is analysed without its own symbols even though Ghidra can apply them. Related to the use case in #3751 (large game binaries where names are the expensive part). ## Problem and expected behavior - Today: the PDB Universal analyzer only finds a PDB through Ghidra's configured symbol paths; a PDB sitting beside the target, or one the caller moved into a reference folder, is never applied, and REA reports "Original names and types may be missing". - Expected: with `REA_GHIDRA_PDB_FILE=<absolute path>`, REA verifies before Ghidra starts that the PDB's GUID and age equal the target's RSDS CodeView record (fail closed, `unsupported_target` naming both identities), then a pre-script verifies again with Ghidra's own PDB reader and hands the file to the analyzer. The profile records `pdb_companion` and the session limitations say what was applied and that the symbols are caller-supplied debug information, not Ghidra discoveries. ## Change and scope - `src/ghidra/GhidraPdbCompanion.ts` (new): reads the MSF 7.00 superblock, stream directory, PDB info stream (1) and DBI header (3) to get GUID and age without loading the PDB (DBI age wins when non-zero, Ghidra's rule); hashes the file through the same no-follow handle; builds the profile commitment `{format: rea-ghidra-pdb-v1, path, configured_path, sha256, bytes, guid, age}`; compares against `target.codeView` (REA's existing PE parse of the RSDS record). Refusals: GUID/age mismatch, PE without an RSDS GUID, non-PE target (`unsupported_target`); unreadable or non-MSF file (`ProviderAdapterError`, `setting: REA_GHIDRA_PDB_FILE`). - `bridge/ghidra/ReaGhidraApplyPdb.java` (new pre-script, modelled on `ReaGhidraApplySeeds.java`): re-hashes the file, reads GUID/age with `PdbParser` / `getIdentifiers()`, compares with the committed values and the Program's `PdbProgramAttributes`, and only then calls `PdbUniversalAnalyzer.setPdbFileOption`. Any mismatch throws and analyzeHeadless aborts before analysis. Outcome stored in the Program Info option "REA PDB companion"; the bridge reports it as `pdb_companion` in the handshake. (Ghidra 12.1.4's `PdbAnalyzerCommon.findPdb` applies a forced PDB without any GUID check, so REA's check is the only gate.) - Session (`GhidraProviderClient.ts`, `GhidraProvider.ts`, `GhidraSessionValues.ts`): fail-closed outcome check — report must match the commitment's sha256/GUID/age, the Program's GUID/age must match, and Ghidra must report `PDB Loaded` true; otherwise `ProviderAdapterError` and the session closes. New limitation line "PDB applied: … caller-supplied debug information, not Ghidra discoveries; a matching GUID and age identify the build, not the completeness of the PDB". The old "does not admit a caller-selected PDB companion" sentence now reads "To apply a PDB from this exact build, set REA_GHIDRA_PDB_FILE; REA does not search for one." - Config (`environment.ts`, `parseConfig.ts`, `types.ts`): the setting; `REA_GHIDRA_DISABLED_ANALYZERS` may not contain `PDB Universal` while it is set (`ConfigurationError`). - `package.json`: `ReaGhidraApplyPdb.java` added to `files` (the package would otherwise ship without it); `verify:ghidra:pdb` script. - Not included: symbol-server lookup, PDB search paths, applying a PDB to a persistent project (if #3817 lands first, `REA_GHIDRA_PDB_FILE` should join its refused import-time settings). Scope decision: not applicable — static analysis input, no execution, injection or mutation. ## Contract and boundary impact - Semantic owner and earliest changed stage: Ghidra provider; analysis-profile resolution. - Contribution scope fit and linked maintainer decision: in scope; not applicable. - CLI and MCP/tool-catalog contract: no new tools; `analysis_profile.parameters.pdb_companion` added when the setting is present; error contract row 90.8 in `docs/user-visible-errors.csv` (no new code). - Provider, bridge, target-format, or platform compatibility: PE targets only; Ghidra 12.1.x; verified Linux x64 with an LLD-linked PE+PDB pair. MSVC-produced PDBs, very large PDBs and Windows P0 unverified (documented). - Evidence, artifact, provenance, or reconstruction contract: the PDB's identity (sha256, GUID, age) is part of the profile; symbols from it are labelled caller-supplied in the limitations. - Target selection, authority, upstream behavior, and cleanup details: the caller selects the file; it is read only (no-follow handle; Ghidra reads it through the analyzer). Nothing is written next to the target. - Generated metadata, package, or installation impact: `files` list + one npm script; no catalog change. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: `src/ghidra/GhidraPdbCompanion.test.ts` (MSF parsing incl. truncated/invalid containers, GUID normalisation, DBI-age rule, commitment and refusal paths, outcome check), `tests/boundary/providers/ghidra/ghidraLauncher.test.ts` (pre-script argument order after seeds and before analyzer options; the disabled-analyzer conflict), `tests/conformance/ghidra/pdb-companion.c` (fixture source). - Base reproduction or other evidence: `npm run verify:ghidra:pdb` (new, `scripts/verify-real-ghidra-pdb.mjs`, needs `REA_CLANG` / `REA_LLD_LINK` as the cross-format lane does): links two variants of the fixture into x86-64 PE + PDB pairs (`/debug /pdb:`), moves each PDB away from the path recorded in the image, then through `rea mcp`: control without the setting → `rea_pdb_scale` absent from `list_procedures`; matching PDB → `rea_pdb_scale` and `rea_pdb_entry` present, pseudocode `int __cdecl rea_pdb_entry(void) { … iVar1 = rea_pdb_scale(4,5); … }`, `pdb_companion` GUID/age equal the RSDS record and sha256 equals the file; the other link's PDB → `unsupported_target` "The PDB …/other-renamed.pdb has GUID 13445a73-… and age 1, but the target's CodeView record declares GUID 1a2550ec-… and age 1. REA refuses to apply debug information from another build."; direct analyzeHeadless with a wrong committed sha → pre-script refusal, "Processing aborted as a result of pre-script." - User-visible CLI/MCP output: as quoted above. - Remaining proof gaps: MSVC-linked PDBs (the lane uses LLD's PDB writer); PDBs whose stream-directory block map spans more than one block are refused with a format error rather than parsed (32 KiB blocks reach that above 2 TiB); Windows P0 and macOS not run; time and heap for multi-GB PDBs (hashed twice: TypeScript at profile time, Java in the pre-script) not measured. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed All on Linux x64 (Fedora, Node 22.23, Ghidra 12.1.4, Temurin 21.0.12.1, zig 0.17.0's clang + lld-link for the fixture), final head of this branch: - `npm run typecheck` — clean - `npm run lint` — 0 errors, 1429 warnings (baseline count on upstream main) - `oxfmt --check` — clean - `npm run verify:test-discovery` — passed - `npm run docs:check` — passed - focused suites (`src/ghidra`, `src/config`, `tests/boundary/providers/ghidra`) — 752 tests pass - `npm run test:prepare && npm run test:run` — Test files 3 failed / 1284 passed / 14 skipped; Tests 3 failed / 15023 passed / 175 skipped. The 3 failures (`packageImportAliases`, `runnerCancellation`, `closedStdout`) fail identically on a clean `upstream/main` worktree on the same host. - `npm run verify:ghidra:pdb` — exit 0 (details above) ## Compatibility, safety, and release - Breaking changes or migration steps: none; without the setting behaviour is unchanged (one limitation sentence reworded). - Real Hopper/Ghidra, browser, or OS coverage: real Ghidra 12.1.4 on Linux x64. Windows P0 and macOS: not run. - Package or release metadata impact: `files` gains the new Java script; one new npm script. - Security, privacy, process, or containment review: the PDB is read, never executed; identity is enforced twice (REA's MSF reader against the PE's RSDS record, then Ghidra's reader in the pre-script) and fails closed; the file is opened no-follow. ## Author readiness Human owner: @whalemenace - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. (Opened as a draft; AI-assisted implementation, the human owner completes the declarations.) ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#5923
No description provided.