morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #4650] [MERGED] fix(ghidra): keep listing facts when one function fails to decompile #6343

New issue

Closed

opened 2026-10-31 01:13:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:13:43 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4650
Author: @RishTheFish21
Created: 10/17/2026
Status: ✅ Merged
Merged: 10/20/2026
Merged by: @gensokyo-dev

Base: main ← Head: fix/ghidra-decompiler-failure-dossier

📝 Commits (8)

78a3920 fix(ghidra): keep listing facts when one function fails to decompile
fda3516 Merge main into fix/ghidra-decompiler-failure-dossier
f1aa193 test(ghidra): inspect the decompiler failure lane's installation through the shared owner
03a0ef8 ci: rerun after unrelated process finalization timing failure
65548f3 Merge remote-tracking branch 'origin/pr-4650' into fix/4644-ghidra-partial-dossier
e605975 test(ghidra): qualify failure conformance and owned cleanup on Linux
d442c55 fix(ghidra): avoid replaying transient decompiler failures
364753c test(ghidra): format transient workflow fixture

📊 Changes

19 files changed (+971 additions, -31 deletions)

View changed files

📝 bridge/ghidra/ReaGhidraBridge.java (+44 -16)
📝 docs/mcp-contracts.md (+7 -1)
📝 knip.json (+1 -0)
➕ scripts/verify-real-ghidra-decompiler-failure.mjs (+282 -0)
📝 src/application/AnalysisProvider.ts (+6 -0)
📝 src/application/DirectAnalysis.ts (+18 -4)
📝 src/application/WorkflowEvidence.test.ts (+48 -0)
📝 src/application/WorkflowEvidence.ts (+12 -2)
📝 src/application/binary/BinarySession.ts (+8 -7)
📝 src/ghidra/GhidraFunctionValues.test.ts (+23 -0)
📝 src/ghidra/GhidraFunctionValues.ts (+8 -0)
📝 src/ghidra/GhidraProvider.behavior.test.ts (+46 -0)
📝 src/ghidra/GhidraProviderClient.ts (+8 -0)
📝 src/ghidra/GhidraSnapshotCache.test.ts (+63 -0)
📝 src/server/registerEnhancedTools.ts (+5 -1)
📝 tests/boundary/mcp/ghidraNativeApiMcp.test.ts (+54 -0)
➕ tests/boundary/process/ghidraDecompilerVerifierCleanup.test.ts (+112 -0)
📝 tests/conformance/ghidra/README.md (+13 -0)
➕ tests/conformance/ghidra/ReaDecompilerFailureProbe.java (+213 -0)

📄 Description

Summary

Fixes #4644. When one Ghidra function fails to decompile, analyze_function now returns its listing-backed dossier with pseudocode, native API and p-code value flow explicitly unavailable and the decompiler's reason preserved. Assembly, comments, callers/callees, references, strings/names, CFG and relationship coverage remain available. Cancellation still ends the request. procedure_pseudo_code still fails, with procedure-specific recovery advice instead of blaming environment readiness.

Keep @RishTheFish21's production fix. Qualification merges current main 052956af3, selects exact exported entries in the native probe, binds the Linux fixture's local calls, supervises compiler cleanup and requires positive lineage evidence before reporting cleanup complete. Reports now retain target/source/bridge digests and the provider/profile identity. The actual comparison contains eight focused files.

Contract and scope

The Ghidra bridge owns the unavailable-facet interpretation; the provider adapter projects its named error. CLI and MCP use the existing shared function workflow and dossier schemas. The bridge digest already versions analysis profiles. Existing read-only provider authority applies; additional maintainer scope decision is not applicable. Public schemas and read-only authority remain unchanged. A result-specific live policy prevents replay of transient failures in direct and composed snapshots; healthy caching remains enabled.

Executed validation

Linux x64; pinned Node 24.18.0 / npm 11.16.0; BYO Ghidra 12.1.4 / JDK 21:

Base provider projection: 1 regression failed. Candidate: 250 focused tests passed across provider projection, parsed contracts, snapshot recovery/import, shared workflow policy, verifier cleanup and MCP native API workflows, plus the earlier 8 relationship-coverage tests.
Real Ghidra with the main bridge and controlled decompiler failure: RequestFailure: Ghidra decompilation failed: Injected backend failure; the requested dossier is lost. With the candidate bridge, 34 checks passed. All listing facets and availability/relationship coverage match the healthy observation; decompiler facets retain the failure reason; pseudocode-only failure, cancellation and session recovery remain covered. The probe invokes the actual request response producer; both the native response and parsed observation mark transient failures live. The degraded dossier passes the real provider output contract; the healthy path also passes through the authenticated TCP provider.
Linux source-owned ELF fixture: 23 assembly lines, 1 caller, 1 callee, 7 blocks retained after failure. Target SHA-256 f8b25a00da8a04575cc5dc78fe737f5589cc99d2e6733f6dca7bf20379ada5f2; source/bridge digests and compiler arguments are emitted by the verifier.
Public MCP on the installed /usr/bin/true ELF, 35,664 bytes, SHA-256 c79bf44242829108e323378531f4ac839513ca1fba45efd6583643526e1e9fd2: select a provider-returned entry address, analyze twice in one session, verify identical normalized dossiers, export complete Evidence, close and verify immutable input/empty temporary roots. Protocol 2026-07-28.
Review regressions: a stopped failed compiler leaked its workspace before the fix; controlled incomplete/unknown cleanup retains it. The actual supervised failed-compiler control preserves exit-6 diagnostics and removes the workspace. Degraded direct/composed observations retain Evidence, skip replay entries, and observe recovery; healthy repeats still hit cache.
Final CI: source tests, package build and docs passed. Static/portable/required gates fail on main-existing duplication and EVM tests; cancelled shards are not claimed as passing. Latest Codex review: no findings; Greptile: 5/5.
Final formatting follow-up: the pre-commit lint fix changed a test after formatting; corrected it and passed format:check after the corrective commit. Runtime/native verification remains applicable because that follow-up only formats a test fixture.
Pre-push check:fast, full npm run check (typecheck, formatting, lint, knip, metadata/module/architecture guards), and npm run docs:check: passed; the documentation check also passed 5 consistency tests.

Initial Linux probe attempts stopped before the injected failure: PLT name ambiguity, then the fixture's ELF symbol interposition invalidating its direct-call expectations. Those are retained as harness diagnostics, not bug reproductions. Exact exported-address selection and explicit Linux local binding fix the producing fixture. A later stronger-handler attempt stopped at missing nested headless context before fault comparison; the final probe binds the actual HeadlessAnalyzer as Ghidra does for nested scripts, then passes all 34 checks. Native disassembly independently verifies the direct calls. The successful main/candidate fault comparison uses the corrected fixture.

Performance and limits

One CPU; 8-GiB hard / 7-GiB early family-memory stop; no swap. 34-check real conformance: 76.952 s elapsed, 74.492 s family CPU, 570.0 MiB peak. The probe uses the real request handler and positive lineage cleanup. Public real-ELF MCP task: 38.007 s elapsed, 36.655 s CPU, 1,100.7 MiB peak. Cold import/search 29.553 s; first function 181.1 ms, repeated warm dossier 29.6 ms. All completed with positive cleanup evidence and no OOM. OS cache is uncontrolled; these samples do not establish a general speedup. The failed baseline does not complete the same dossier task, so its failure time is not an equivalent latency comparison. The previous-head equivalent task measured 38.936 s / 37.550 CPU s / 1,116.6 MiB (174.5 ms first / 24.4 ms warm); single samples do not establish a performance change. Only transient failures opt out of existing caches.

Captured post-connect MCP traffic for six tool calls: 21,908 cl100k_base token estimate (519 outbound / 21,389 inbound; previous head 21,905). SDK initialization/startup catalog or schema traffic was not captured; actual agent/model usage and prompts are unmeasured. This is neither a whole-task token total nor a token-savings claim.

The decompiler failure is injected into a real Ghidra program; a naturally failing public binary is unavailable. The degraded public CLI with a real backend and Windows were not run. Controlled-wire MCP tests exercise degraded composed output and cache recovery; simulated failures do not establish natural decompiler failure conditions. The public real-ELF MCP control is healthy. Prior contributor macOS evidence predates this Linux qualification and was not rerun here. Target code is never executed.

Author readiness

Human owner: @RishTheFish21

The original readiness declaration is retained. The agent qualified the final diff for @gensokyo-dev under the task owner's explicit delegation to validate, update ready PRs and merge; no new separate human review by the original author is claimed.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

Focused existing-boundary fix; applicable scope/authority resolved; artifact identity, partial facts and exact cleanup owners preserved.
Final diff checked for unrelated changes/credentials; generated binaries, projects, local reports and dependencies remain outside the diff.
Final local checks and final-head automated review complete (Codex: no findings; Greptile: 5/5).
Merged as c374c2fe7290b914d73d9e27af38b498753ec593 with the task owner’s authorization to leave unrelated CI failures unchanged. Exact CI main already fails duplication (3.52498% > 3.5%; exact PR merge 3.52671%). The two failing EVM cancellation/cleanup cases also reproduce on that main; four other shards were cancelled, so their coverage is unavailable. No unrelated changes or threshold adjustments. See the controlled attribution.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4650 **Author:** [@RishTheFish21](https://github.com/RishTheFish21) **Created:** 10/17/2026 **Status:** ✅ Merged **Merged:** 10/20/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `fix/ghidra-decompiler-failure-dossier` --- ### 📝 Commits (8) - [`78a3920`](https://github.com/morluto/rea/commit/78a392043f829dae0af4c510eaf9449bb998ab11) fix(ghidra): keep listing facts when one function fails to decompile - [`fda3516`](https://github.com/morluto/rea/commit/fda35167e6a5cf95d72bc297d6015b556d1d09fd) Merge main into fix/ghidra-decompiler-failure-dossier - [`f1aa193`](https://github.com/morluto/rea/commit/f1aa193ad8dc2651ca0ac2fc1e0be24de1017232) test(ghidra): inspect the decompiler failure lane's installation through the shared owner - [`03a0ef8`](https://github.com/morluto/rea/commit/03a0ef8297eee438fee5d1a29d3f4b3ae4341e37) ci: rerun after unrelated process finalization timing failure - [`65548f3`](https://github.com/morluto/rea/commit/65548f314c8d35f3f2935b4974d82e61198fb220) Merge remote-tracking branch 'origin/pr-4650' into fix/4644-ghidra-partial-dossier - [`e605975`](https://github.com/morluto/rea/commit/e6059750ec9ad542e990bc7454d16279043e5c78) test(ghidra): qualify failure conformance and owned cleanup on Linux - [`d442c55`](https://github.com/morluto/rea/commit/d442c5569365d5a2c3c26a48b9b11c2d501ec53e) fix(ghidra): avoid replaying transient decompiler failures - [`364753c`](https://github.com/morluto/rea/commit/364753c57659dcc1526e3eb0262894be87f55fca) test(ghidra): format transient workflow fixture ### 📊 Changes **19 files changed** (+971 additions, -31 deletions) <details> <summary>View changed files</summary> 📝 `bridge/ghidra/ReaGhidraBridge.java` (+44 -16) 📝 `docs/mcp-contracts.md` (+7 -1) 📝 `knip.json` (+1 -0) ➕ `scripts/verify-real-ghidra-decompiler-failure.mjs` (+282 -0) 📝 `src/application/AnalysisProvider.ts` (+6 -0) 📝 `src/application/DirectAnalysis.ts` (+18 -4) 📝 `src/application/WorkflowEvidence.test.ts` (+48 -0) 📝 `src/application/WorkflowEvidence.ts` (+12 -2) 📝 `src/application/binary/BinarySession.ts` (+8 -7) 📝 `src/ghidra/GhidraFunctionValues.test.ts` (+23 -0) 📝 `src/ghidra/GhidraFunctionValues.ts` (+8 -0) 📝 `src/ghidra/GhidraProvider.behavior.test.ts` (+46 -0) 📝 `src/ghidra/GhidraProviderClient.ts` (+8 -0) 📝 `src/ghidra/GhidraSnapshotCache.test.ts` (+63 -0) 📝 `src/server/registerEnhancedTools.ts` (+5 -1) 📝 `tests/boundary/mcp/ghidraNativeApiMcp.test.ts` (+54 -0) ➕ `tests/boundary/process/ghidraDecompilerVerifierCleanup.test.ts` (+112 -0) 📝 `tests/conformance/ghidra/README.md` (+13 -0) ➕ `tests/conformance/ghidra/ReaDecompilerFailureProbe.java` (+213 -0) </details> ### 📄 Description ## Summary Fixes #4644. When one Ghidra function fails to decompile, `analyze_function` now returns its listing-backed dossier with pseudocode, native API and p-code value flow explicitly unavailable and the decompiler's reason preserved. Assembly, comments, callers/callees, references, strings/names, CFG and relationship coverage remain available. Cancellation still ends the request. `procedure_pseudo_code` still fails, with procedure-specific recovery advice instead of blaming environment readiness. Keep @RishTheFish21's production fix. Qualification merges current main `052956af3`, selects exact exported entries in the native probe, binds the Linux fixture's local calls, supervises compiler cleanup and requires positive lineage evidence before reporting cleanup complete. Reports now retain target/source/bridge digests and the provider/profile identity. The actual comparison contains eight focused files. ## Contract and scope The Ghidra bridge owns the unavailable-facet interpretation; the provider adapter projects its named error. CLI and MCP use the existing shared function workflow and dossier schemas. The bridge digest already versions analysis profiles. Existing read-only provider authority applies; additional maintainer scope decision is not applicable. Public schemas and read-only authority remain unchanged. A result-specific live policy prevents replay of transient failures in direct and composed snapshots; healthy caching remains enabled. ## Executed validation Linux x64; pinned Node 24.18.0 / npm 11.16.0; BYO Ghidra 12.1.4 / JDK 21: - Base provider projection: **1 regression failed**. Candidate: **250 focused tests passed** across provider projection, parsed contracts, snapshot recovery/import, shared workflow policy, verifier cleanup and MCP native API workflows, plus the earlier **8 relationship-coverage tests**. - Real Ghidra with the main bridge and controlled decompiler failure: `RequestFailure: Ghidra decompilation failed: Injected backend failure`; the requested dossier is lost. With the candidate bridge, **34 checks passed**. All listing facets and availability/relationship coverage match the healthy observation; decompiler facets retain the failure reason; pseudocode-only failure, cancellation and session recovery remain covered. The probe invokes the actual request response producer; both the native response and parsed observation mark transient failures live. The degraded dossier passes the real provider output contract; the healthy path also passes through the authenticated TCP provider. - Linux source-owned ELF fixture: **23 assembly lines, 1 caller, 1 callee, 7 blocks** retained after failure. Target SHA-256 `f8b25a00da8a04575cc5dc78fe737f5589cc99d2e6733f6dca7bf20379ada5f2`; source/bridge digests and compiler arguments are emitted by the verifier. - Public MCP on the installed `/usr/bin/true` ELF, **35,664 bytes**, SHA-256 `c79bf44242829108e323378531f4ac839513ca1fba45efd6583643526e1e9fd2`: select a provider-returned entry address, analyze twice in one session, verify identical normalized dossiers, export complete Evidence, close and verify immutable input/empty temporary roots. Protocol 2026-07-28. - Review regressions: a stopped failed compiler leaked its workspace before the fix; controlled incomplete/unknown cleanup retains it. The actual supervised failed-compiler control preserves exit-6 diagnostics and removes the workspace. Degraded direct/composed observations retain Evidence, skip replay entries, and observe recovery; healthy repeats still hit cache. - Final CI: source tests, package build and docs passed. Static/portable/required gates fail on main-existing duplication and EVM tests; cancelled shards are not claimed as passing. Latest Codex review: no findings; Greptile: 5/5. - Final formatting follow-up: the pre-commit lint fix changed a test after formatting; corrected it and passed `format:check` after the corrective commit. Runtime/native verification remains applicable because that follow-up only formats a test fixture. - Pre-push `check:fast`, full `npm run check` (typecheck, formatting, lint, knip, metadata/module/architecture guards), and `npm run docs:check`: passed; the documentation check also passed **5 consistency tests**. Initial Linux probe attempts stopped before the injected failure: PLT name ambiguity, then the fixture's ELF symbol interposition invalidating its direct-call expectations. Those are retained as harness diagnostics, not bug reproductions. Exact exported-address selection and explicit Linux local binding fix the producing fixture. A later stronger-handler attempt stopped at missing nested headless context before fault comparison; the final probe binds the actual HeadlessAnalyzer as Ghidra does for nested scripts, then passes all 34 checks. Native disassembly independently verifies the direct calls. The successful main/candidate fault comparison uses the corrected fixture. ## Performance and limits One CPU; 8-GiB hard / 7-GiB early family-memory stop; no swap. **34-check real conformance: 76.952 s elapsed, 74.492 s family CPU, 570.0 MiB peak**. The probe uses the real request handler and positive lineage cleanup. Public real-ELF MCP task: **38.007 s elapsed, 36.655 s CPU, 1,100.7 MiB peak**. Cold import/search **29.553 s**; first function **181.1 ms**, repeated warm dossier **29.6 ms**. All completed with positive cleanup evidence and no OOM. OS cache is uncontrolled; these samples do not establish a general speedup. The failed baseline does not complete the same dossier task, so its failure time is not an equivalent latency comparison. The previous-head equivalent task measured 38.936 s / 37.550 CPU s / 1,116.6 MiB (174.5 ms first / 24.4 ms warm); single samples do not establish a performance change. Only transient failures opt out of existing caches. Captured post-connect MCP traffic for six tool calls: **21,908 `cl100k_base` token estimate** (519 outbound / 21,389 inbound; previous head 21,905). SDK initialization/startup catalog or schema traffic was not captured; actual agent/model usage and prompts are unmeasured. This is neither a whole-task token total nor a token-savings claim. The decompiler failure is injected into a real Ghidra program; a naturally failing public binary is unavailable. The degraded public CLI with a real backend and Windows were not run. Controlled-wire MCP tests exercise degraded composed output and cache recovery; simulated failures do not establish natural decompiler failure conditions. The public real-ELF MCP control is healthy. Prior contributor macOS evidence predates this Linux qualification and was not rerun here. Target code is never executed. ## Author readiness Human owner: @RishTheFish21 The original readiness declaration is retained. The agent qualified the final diff for @gensokyo-dev under the task owner's explicit delegation to validate, update ready PRs and merge; no new separate human review by the original author is claimed. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] Focused existing-boundary fix; applicable scope/authority resolved; artifact identity, partial facts and exact cleanup owners preserved. - [x] Final diff checked for unrelated changes/credentials; generated binaries, projects, local reports and dependencies remain outside the diff. - [x] Final local checks and final-head automated review complete (Codex: no findings; Greptile: 5/5). - [x] Merged as `c374c2fe7290b914d73d9e27af38b498753ec593` with the task owner’s authorization to leave unrelated CI failures unchanged. Exact CI main already fails duplication (3.52498% > 3.5%; exact PR merge 3.52671%). The two failing EVM cancellation/cleanup cases also reproduce on that main; four other shards were cancelled, so their coverage is unavailable. No unrelated changes or threshold adjustments. See the [controlled attribution](https://github.com/morluto/rea/pull/4650#issuecomment-6290251572). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:13:43 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:46:49 +00:00

[PR #6345] [MERGED] fix(environment): preserve UUID renewal ownership #7293

bsakaguchi changed title from ~~[PR #4650] [MERGED] fix(ghidra): keep listing facts when one function fails to decompile~~ to [PR #4650] fix(ghidra): keep listing facts when one function fails to decompile

2026-11-03 13:32:52 +00:00

bsakaguchi changed title from ~~[PR #4650] fix(ghidra): keep listing facts when one function fails to decompile~~ to [PR #4650] [MERGED] fix(ghidra): keep listing facts when one function fails to decompile

2026-11-03 14:34:37 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6343

No description provided.

Rows
Columns