[PR #1599] [MERGED] feat(ghidra): admit bounded PSP ELF through installed Allegrex #2995

New issue

Closed

opened 2026-10-17 21:12:42 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:12:42 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1599
Author: @jaelliot
Created: 10/10/2026
Status: ✅ Merged
Merged: 10/11/2026
Merged by: @morluto

Base: main ← Head: feat/psp-allegrex-ghidra

📝 Commits (10+)

c07024d feat(ghidra): admit bounded PSP ELF through installed Allegrex
e52576a fix(ghidra): validate PSPSDK ABI records in PSP admission
69b0607 test(ghidra): bind PSP ABI verification to snapshot profiles
7e81582 test(ghidra): declare independent PSP ABI reader contract
35fe387 test(ghidra): retain the PSP fixture discriminated type
8e1d69b test(ghidra): contextually type PSP metadata as the MIPS target variant
2619488 chore: synchronize PSP contribution with upstream 926c2d6
4bfcf81 test(ghidra): keep carved firmware checks scoped to generic MIPS fixtures
8607d4d chore: synchronize PSP contribution with upstream main
0662433 style(ghidra): format PSP language-override guard

📊 Changes

17 files changed (+1637 additions, -146 deletions)

View changed files

📝 docs/ghidra-mips.md (+1 -1)
➕ docs/ghidra-psp.md (+105 -0)
📝 scripts/verify-real-ghidra-mips.mjs (+311 -130)
📝 src/ghidra/GhidraAnalysisProfile.ts (+30 -5)
📝 src/ghidra/GhidraLauncher.ts (+58 -7)
📝 src/ghidra/GhidraMipsProfile.test.ts (+1 -1)
📝 src/ghidra/GhidraProvider.ts (+10 -2)
📝 src/ghidra/GhidraProviderClient.ts (+35 -0)
➕ src/ghidra/GhidraPspExtension.ts (+212 -0)
➕ src/ghidra/GhidraPspProfile.ts (+60 -0)
➕ tests/boundary/filesystem/ghidraPspExtension.test.ts (+506 -0)
➕ tests/conformance/ghidra/psp-checks.test.ts (+51 -0)
➕ tests/conformance/ghidra/psp-fixture.d.mts (+3 -0)
➕ tests/conformance/ghidra/psp-fixture.mjs (+212 -0)
➕ tests/conformance/psp/Makefile (+8 -0)
➕ tests/conformance/psp/main.c (+20 -0)
➕ tests/conformance/psp/probe.S (+14 -0)

📄 Description

Upstream integration update (2026-10-11): The PSP contribution was reconciled with upstream commit 0ecf3f2438ec96a6b543942869808018e8d84452 using merge commit 8607d4d64b4729cc0c5268e862adecabbda1dd8c, then focused language-override regression formatting commits. Current PR head: 7f011dc16c446f42e47a841bcf4146debc1588f1. The current upstream CI run 38111212306 passed all 31 jobs, including test shards, static checks, package and coverage checks. GitHub currently reports mergeable/clean; the maintainer merge decision is unchanged. The existing real-Ghidra PSP verification 38091449625 is an earlier exact-source checkpoint on 4bfcf81, not fresh real-provider validation of 7f011dc. The current head preserves the PSP profile and upstream language override/analysis-seed paths; another targeted real-provider run is appropriate before claiming that integration fully revalidated.

Scope

Follow-on to merged #1362, relating to #1330: analyze a bounded class of PSP ELF executables through REA's existing Ghidra CLI/MCP workflow using a caller-installed Allegrex extension. Ready for maintainer review; unmerged. Both the focused real-Ghidra verification and the ordinary upstream CI run have passed for the contribution head.

Contribution head: 4bfcf81d757473b6edea66779a7b732e041d5356.
Upstream revision incorporated and tested in the focused real-provider run: 926c2d6ba7975522d63a8dbb0d448f5b9d1514d4.
Integration preserves the previously verified PSP checkpoint 8e1d69b; no published history was rewritten.

Support boundary

Concern	Initial scope
Host	Linux x64
Target	Little-endian ELF32 `ET_EXEC`, `EM_MIPS`, `e_flags=0x10a23001`
ABI record	Version 0, ISA 2/revision 0; `gprSize=1`, `cpr1Size=1`, `cpr2Size=0`, `fpAbi=2`; no ISA extension/ASEs; `flags1=1`, `flags2=0`
Installation	Compatible caller-installed `ghidra-allegrex` in the selected Ghidra distribution; verified pairing Ghidra 12.1.3 / extension v21.4 / JDK 21
Analysis	Existing inventory, instruction/data, function, Evidence and snapshot operations

Missing or different declarations remain unsupported, not inferred. This is a separate PSP profile; it does not relax generic MIPS32r2/o32 admission. See the PSP guide at this checkpoint for prerequisites and limitations.

Implementation

Reuse target parsing, provider lifecycle and public contracts. Select PspElfLoader, Allegrex:LE:32:default, and compiler default; verify the observed language/compiler before serving results. Validate extension metadata and required files, bind installed content to the analysis profile, and recheck that identity before launch. Missing or incompatible prerequisites produce actionable errors without silent fallback.

Evidence retains target/profile identity and the extension's known VFPU limitations. No new provider, MCP catalog, processor implementation, runtime dependency, automatic installer, proprietary fixture or commercial engine is added. Allegrex semantics stay in the upstream Ghidra extension.

Verified integration checkpoint

Verification run 38091449625, job 114328551589, completed successfully. Its artifact records source and final source as 4bfcf81d757473b6edea66779a7b732e041d5356, with clean Git status. The orchestration commit 81890208c02f4e53394e897cdad4a4ca2616bc3a is not the tested application source.

Gate	Actual result
Locked install and build	PASS; Node 24.18.0 / npm 11.16.0
Existing `check:ci`	PASS: all eight tasks and architecture guards; lint warnings remain
Actual focused Vitest	233 passed, 0 failed, 3 skipped; 15 suites passed
Documentation / discovery	PASS; all 791 test files discovered exactly once
Missing extension through public CLI/MCP	PASS; CLI exit 1 and actionable `PSP_EXTENSION_UNAVAILABLE`
Real PSP public CLI/MCP	PASS: observed Allegrex language, known functions/calls, instruction/data facts and Evidence/snapshot binding
PSP A → B → A	PASS, including rejecting the wrong target/snapshot pairing
Generic LE and BE MIPS	PASS, including the upstream section-truncated carved-firmware cases; neither selects Allegrex
Source preservation and cleanup	PASS; no owned descendants remain in the three verifier reports

Tool archives were checksum-verified: Ghidra 12.1.3, Allegrex v21.4, PSPDEV v20261001. Targets were compiled and statically analyzed, not executed. The compact artifact is psp-upstream-integration-926c2d6 (SHA-256 85a4eda8c1bc1ae472fd9d55fe272f635b506367b138afc7db4004ada797a5a8).

The focused suites include PSP/MIPS regressions and the cleanup/setup areas implicated in the previous integration failures. These are existing tests, not 233 newly added cases. Discovery is not execution of the full suite. No required check is waived by the focused run.

Ordinary upstream CI

Upstream CI run 38092014434, attempt 1, completed successfully on October 10 at 22:51:53 UTC, associated with contribution head 4bfcf81d757473b6edea66779a7b732e041d5356. All 19 jobs completed successfully, including all four test shards, coverage/timing aggregation, static checks, documentation generation, source-built conformance fixtures, reconstruction readiness, real Apple artifact CLI/MCP E2E, the Windows capability/package lane, and Linux/macOS x64/arm64 package baselines.

This ordinary PR-integration run is separate from the exact-source real-Ghidra verification above. Passing platform package checks does not extend the PSP profile's verified host scope beyond Linux x64. The PR was marked ready for review after these results were checked; no additional source commits or workflow changes were made for that transition.

Integration correction

A clean merge exposed a verifier defect in run 38090704268: newly added generic carved-firmware checks also ran in PSP mode and expected a generic MIPS language and generic-fixture symbols. The two-file correction scopes those checks to generic mode. All generic carved-firmware assertions remain exercised; PSP retains its ordinary CLI/MCP, known-fact, snapshot and A → B → A checks and explicitly reports carved-PSP coverage as not run. The correction changes no production code or test threshold.

Not established / review focus

This is selected static conformance, not execution equivalence or complete VFPU semantics. PRX relocation behavior, PBP/ISO/CSO inputs, live overlays, emulation, game-specific reconstruction, other host platforms and independently published PSP binaries remain unverified/out of scope. The strict producer declaration does not imply general compatibility with historical commercial PSP executables.

Installed extension code is trusted: its digest supplies interpretation/cache identity, not a signature or sandbox. Do not change the installation during a session. Please review the narrow admission boundary and installed-extension identity approach. No issue is automatically closed by this description.

Historical verification and failures

Independent PSPSDK ABI inspection disproved the initial assumption that the fixture had no ABI record. Production admission and the verifier were corrected together; profile revision v2 binds the actual record.
Public PSP verification at 8e1d69b remains a historical passing checkpoint, not proof for later merges.
Previous upstream integration run failed. Its actual synthetic merge combined 8e1d69b with c1217e7. The pinned upstream update reuses subsequent changes instead of adding unrelated repairs to this contribution. Downloaded test reports showed six failures in cleanup/setup areas; their targeted current suites passed after integration. This is not a claim that the older base CI was green.
The first attempt to apply the verifier correction stopped at an overly strict patch-location assertion before changing source. Correcting that fork-only preparation step produced the successful run above.
Fork-only orchestration remains outside this PR's source diff. Earlier failed attempts remain available in Actions history.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1599 **Author:** [@jaelliot](https://github.com/jaelliot) **Created:** 10/10/2026 **Status:** ✅ Merged **Merged:** 10/11/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `feat/psp-allegrex-ghidra` --- ### 📝 Commits (10+) - [`c07024d`](https://github.com/morluto/rea/commit/c07024dec5cc59e23e2645293f8bd54252a6390a) feat(ghidra): admit bounded PSP ELF through installed Allegrex - [`e52576a`](https://github.com/morluto/rea/commit/e52576acc64a98f70cf8069e58b5a22a388fc968) fix(ghidra): validate PSPSDK ABI records in PSP admission - [`69b0607`](https://github.com/morluto/rea/commit/69b06073ef02bce188d78ffd0bce5e028122ea89) test(ghidra): bind PSP ABI verification to snapshot profiles - [`7e81582`](https://github.com/morluto/rea/commit/7e815824ae19230b604097f36c0a6e490fa243d8) test(ghidra): declare independent PSP ABI reader contract - [`35fe387`](https://github.com/morluto/rea/commit/35fe387cae12fa1f2837e65b78067a06f8dc63ff) test(ghidra): retain the PSP fixture discriminated type - [`8e1d69b`](https://github.com/morluto/rea/commit/8e1d69bd41ad118c4dd434e8a2bdb60a9e25fc0c) test(ghidra): contextually type PSP metadata as the MIPS target variant - [`2619488`](https://github.com/morluto/rea/commit/2619488261fdfd7d09b50195712ee8bd5ed84e77) chore: synchronize PSP contribution with upstream 926c2d6 - [`4bfcf81`](https://github.com/morluto/rea/commit/4bfcf81d757473b6edea66779a7b732e041d5356) test(ghidra): keep carved firmware checks scoped to generic MIPS fixtures - [`8607d4d`](https://github.com/morluto/rea/commit/8607d4d64b4729cc0c5268e862adecabbda1dd8c) chore: synchronize PSP contribution with upstream main - [`0662433`](https://github.com/morluto/rea/commit/066243380d2207ac0f9608c1c053d18ba492ba8b) style(ghidra): format PSP language-override guard ### 📊 Changes **17 files changed** (+1637 additions, -146 deletions) <details> <summary>View changed files</summary> 📝 `docs/ghidra-mips.md` (+1 -1) ➕ `docs/ghidra-psp.md` (+105 -0) 📝 `scripts/verify-real-ghidra-mips.mjs` (+311 -130) 📝 `src/ghidra/GhidraAnalysisProfile.ts` (+30 -5) 📝 `src/ghidra/GhidraLauncher.ts` (+58 -7) 📝 `src/ghidra/GhidraMipsProfile.test.ts` (+1 -1) 📝 `src/ghidra/GhidraProvider.ts` (+10 -2) 📝 `src/ghidra/GhidraProviderClient.ts` (+35 -0) ➕ `src/ghidra/GhidraPspExtension.ts` (+212 -0) ➕ `src/ghidra/GhidraPspProfile.ts` (+60 -0) ➕ `tests/boundary/filesystem/ghidraPspExtension.test.ts` (+506 -0) ➕ `tests/conformance/ghidra/psp-checks.test.ts` (+51 -0) ➕ `tests/conformance/ghidra/psp-fixture.d.mts` (+3 -0) ➕ `tests/conformance/ghidra/psp-fixture.mjs` (+212 -0) ➕ `tests/conformance/psp/Makefile` (+8 -0) ➕ `tests/conformance/psp/main.c` (+20 -0) ➕ `tests/conformance/psp/probe.S` (+14 -0) </details> ### 📄 Description > **Upstream integration update (2026-10-11):** The PSP contribution was reconciled with upstream commit `0ecf3f2438ec96a6b543942869808018e8d84452` using merge commit `8607d4d64b4729cc0c5268e862adecabbda1dd8c`, then focused language-override regression formatting commits. **Current PR head:** `7f011dc16c446f42e47a841bcf4146debc1588f1`. The [current upstream CI run 38111212306](https://github.com/morluto/rea/actions/runs/38111212306) passed all 31 jobs, including test shards, static checks, package and coverage checks. GitHub currently reports **mergeable/clean**; the maintainer merge decision is unchanged. The existing [real-Ghidra PSP verification 38091449625](https://github.com/jaelliot/rea/actions/runs/38091449625) is an earlier exact-source checkpoint on `4bfcf81`, **not fresh real-provider validation of `7f011dc`**. The current head preserves the PSP profile and upstream language override/analysis-seed paths; another targeted real-provider run is appropriate before claiming that integration fully revalidated. ## Scope Follow-on to merged #1362, relating to #1330: analyze a bounded class of PSP ELF executables through REA's existing Ghidra CLI/MCP workflow using a caller-installed Allegrex extension. **Ready for maintainer review; unmerged.** Both the focused real-Ghidra verification and the ordinary upstream CI run have passed for the contribution head. - Contribution head: `4bfcf81d757473b6edea66779a7b732e041d5356`. - Upstream revision incorporated and tested in the focused real-provider run: `926c2d6ba7975522d63a8dbb0d448f5b9d1514d4`. - Integration preserves the previously verified PSP checkpoint `8e1d69b`; no published history was rewritten. ## Support boundary | Concern | Initial scope | |---|---| | Host | Linux x64 | | Target | Little-endian ELF32 `ET_EXEC`, `EM_MIPS`, `e_flags=0x10a23001` | | ABI record | Version 0, ISA 2/revision 0; `gprSize=1`, `cpr1Size=1`, `cpr2Size=0`, `fpAbi=2`; no ISA extension/ASEs; `flags1=1`, `flags2=0` | | Installation | Compatible caller-installed `ghidra-allegrex` in the selected Ghidra distribution; verified pairing Ghidra 12.1.3 / extension v21.4 / JDK 21 | | Analysis | Existing inventory, instruction/data, function, Evidence and snapshot operations | Missing or different declarations remain unsupported, not inferred. This is a separate PSP profile; it does not relax generic MIPS32r2/o32 admission. See [the PSP guide at this checkpoint](https://github.com/jaelliot/rea/blob/4bfcf81d757473b6edea66779a7b732e041d5356/docs/ghidra-psp.md) for prerequisites and limitations. ## Implementation Reuse target parsing, provider lifecycle and public contracts. Select `PspElfLoader`, `Allegrex:LE:32:default`, and compiler `default`; verify the observed language/compiler before serving results. Validate extension metadata and required files, bind installed content to the analysis profile, and recheck that identity before launch. Missing or incompatible prerequisites produce actionable errors without silent fallback. Evidence retains target/profile identity and the extension's known VFPU limitations. No new provider, MCP catalog, processor implementation, runtime dependency, automatic installer, proprietary fixture or commercial engine is added. Allegrex semantics stay in the upstream Ghidra extension. ## Verified integration checkpoint **[Verification run 38091449625](https://github.com/jaelliot/rea/actions/runs/38091449625), [job 114328551589](https://github.com/jaelliot/rea/actions/runs/38091449625/job/114328551589), completed successfully.** Its artifact records source and final source as **`4bfcf81d757473b6edea66779a7b732e041d5356`**, with clean Git status. The orchestration commit `81890208c02f4e53394e897cdad4a4ca2616bc3a` is not the tested application source. | Gate | Actual result | |---|---| | Locked install and build | PASS; Node 24.18.0 / npm 11.16.0 | | Existing `check:ci` | PASS: all eight tasks and architecture guards; lint warnings remain | | Actual focused Vitest | **233 passed, 0 failed, 3 skipped; 15 suites passed** | | Documentation / discovery | PASS; all 791 test files discovered exactly once | | Missing extension through public CLI/MCP | PASS; CLI exit 1 and actionable `PSP_EXTENSION_UNAVAILABLE` | | Real PSP public CLI/MCP | PASS: observed Allegrex language, known functions/calls, instruction/data facts and Evidence/snapshot binding | | PSP A → B → A | PASS, including rejecting the wrong target/snapshot pairing | | Generic LE and BE MIPS | PASS, including the upstream section-truncated carved-firmware cases; neither selects Allegrex | | Source preservation and cleanup | PASS; no owned descendants remain in the three verifier reports | Tool archives were checksum-verified: Ghidra 12.1.3, Allegrex v21.4, PSPDEV v20261001. Targets were compiled and statically analyzed, not executed. The compact artifact is `psp-upstream-integration-926c2d6` (SHA-256 `85a4eda8c1bc1ae472fd9d55fe272f635b506367b138afc7db4004ada797a5a8`). The focused suites include PSP/MIPS regressions and the cleanup/setup areas implicated in the previous integration failures. These are existing tests, not 233 newly added cases. Discovery is not execution of the full suite. No required check is waived by the focused run. ## Ordinary upstream CI **[Upstream CI run 38092014434](https://github.com/morluto/rea/actions/runs/38092014434), attempt 1, completed successfully on October 10 at 22:51:53 UTC**, associated with contribution head `4bfcf81d757473b6edea66779a7b732e041d5356`. All **19 jobs** completed successfully, including all four test shards, coverage/timing aggregation, static checks, documentation generation, source-built conformance fixtures, reconstruction readiness, real Apple artifact CLI/MCP E2E, the Windows capability/package lane, and Linux/macOS x64/arm64 package baselines. This ordinary PR-integration run is separate from the exact-source real-Ghidra verification above. Passing platform package checks does not extend the PSP profile's verified host scope beyond Linux x64. The PR was marked ready for review after these results were checked; no additional source commits or workflow changes were made for that transition. ## Integration correction A clean merge exposed a verifier defect in [run 38090704268](https://github.com/jaelliot/rea/actions/runs/38090704268): newly added generic carved-firmware checks also ran in PSP mode and expected a generic MIPS language and generic-fixture symbols. The two-file correction scopes those checks to generic mode. All generic carved-firmware assertions remain exercised; PSP retains its ordinary CLI/MCP, known-fact, snapshot and A → B → A checks and explicitly reports carved-PSP coverage as **not run**. The correction changes no production code or test threshold. ## Not established / review focus This is selected static conformance, not execution equivalence or complete VFPU semantics. PRX relocation behavior, PBP/ISO/CSO inputs, live overlays, emulation, game-specific reconstruction, other host platforms and independently published PSP binaries remain unverified/out of scope. The strict producer declaration does not imply general compatibility with historical commercial PSP executables. Installed extension code is trusted: its digest supplies interpretation/cache identity, not a signature or sandbox. Do not change the installation during a session. Please review the narrow admission boundary and installed-extension identity approach. No issue is automatically closed by this description. <details> <summary>Historical verification and failures</summary> - [Independent PSPSDK ABI inspection](https://github.com/jaelliot/rea/actions/runs/38070552973) disproved the initial assumption that the fixture had no ABI record. Production admission and the verifier were corrected together; profile revision v2 binds the actual record. - [Public PSP verification at 8e1d69b](https://github.com/jaelliot/rea/actions/runs/38072285856) remains a historical passing checkpoint, not proof for later merges. - [Previous upstream integration run](https://github.com/morluto/rea/actions/runs/38072772089) failed. Its actual synthetic merge combined 8e1d69b with c1217e7. The pinned upstream update reuses subsequent changes instead of adding unrelated repairs to this contribution. Downloaded test reports showed six failures in cleanup/setup areas; their targeted current suites passed after integration. This is not a claim that the older base CI was green. - The first attempt to apply the verifier correction stopped at an overly strict patch-location assertion before changing source. Correcting that fork-only preparation step produced the successful run above. - Fork-only orchestration remains outside this PR's source diff. Earlier failed attempts remain available in Actions history. </details> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:12:42 +00:00