morluto/rea
1
0
Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00
Code Issues 1.3k Projects Packages Wiki Activity

[PR #4237] feat(ghidra): report blocks the decompiler removed as unreachable #6140

New issue
Open
opened 2026-10-31 01:12:10 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-31 01:12:10 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4237
Author: @RishTheFish21
Created: 10/15/2026
Status: 🔄 Open

Base: main ← Head: feat/2735-decompiler-completeness

📝 Commits (8)

  • 88f543d feat(ghidra): report blocks the decompiler removed as unreachable
  • 4334dd3 fix(ghidra): only count whole-line header removal warnings
  • 5ddaab7 Merge origin/main into feat/2735-decompiler-completeness
  • ce4e63c fix(ghidra): keep sub-word warning identities and flag unrecorded completeness
  • ffc09fc fix(ghidra): exclude caller plate comments from removed-block warnings
  • 60d75d5 Merge main into feat/2735-decompiler-completeness
  • 1874faf test(ghidra): align the removed-block checks with current main
  • f6334dd Merge commit 'd0067ba34a' into feat/2735-decompiler-completeness

📊 Changes

18 files changed (+939 additions, -37 deletions)

View changed files

📝 bridge/ghidra/ReaGhidraBridge.java (+144 -0)
📝 docs/mcp-contracts.md (+17 -0)
📝 docs/native-investigation.md (+3 -1)
📝 knip.json (+1 -0)
📝 scripts/verify-real-ghidra-function.mjs (+87 -0)
➕ scripts/verify-real-ghidra-warning-provenance.mjs (+150 -0)
📝 scripts/verify-real-ghidra.mjs (+68 -33)
📝 src/domain/analysisView/analysisView.test.ts (+61 -0)
📝 src/domain/analysisView/analysisView.ts (+1 -0)
📝 src/domain/analysisView/nativeFunctionView.ts (+23 -0)
📝 src/domain/hopperValues.ts (+32 -0)
📝 src/ghidra/GhidraAnalysisProfile.ts (+3 -2)
📝 src/ghidra/GhidraFunctionValues.test.ts (+78 -0)
📝 src/ghidra/GhidraFunctionValues.ts (+13 -0)
📝 src/ghidra/GhidraProvider.behavior.test.ts (+2 -1)
📝 tests/conformance/ghidra/README.md (+11 -0)
➕ tests/conformance/ghidra/ReaDecompilerWarningProvenanceProbe.java (+188 -0)
➕ tests/conformance/ghidra/decompiler-removed-block.S (+57 -0)

📄 Description

Summary

Ghidra's decompiler sometimes drops blocks it decides are unreachable. The only sign of this is a header comment in the pseudocode (/* WARNING: Removing unreachable block (ram,0x...) */). REA passes that comment through, but the dossier has no structured signal, and agents tend to skip comments and treat the pseudocode as complete. #2735 describes a case where an auth comparison sat entirely in a dropped block.

analyze_function dossiers from Ghidra now include decompiler_completeness. Its removed_blocks lists each dropped block's range and instructions, and its unmapped_warnings lists removal warnings whose address can't be placed in the function. Native pseudocode views now also say when no record exists at all (see below).

Fixes #2735

Problem and expected behavior

Before: when Ghidra removes a block, the only record is a comment inside the pseudocode. Nothing structured tells a caller that the pseudocode is missing code, or where the missing code is.

After: each removed block appears as:

  • start: parsed from Ghidra's per-block warning. I checked the format against Funcdata::removeUnreachableBlocks in the decompiler source, which emits one warning per removed block with its start address.
  • end: the exclusive end of the listing basic block containing start, or null if the listing has no block there.
  • reason: "unreachable".
  • assembly: the block's instructions from the listing. Empty when end is null.

A warning whose address can't be mapped into the analyzed function is kept in unmapped_warnings as "<space>,<offset>" (with Ghidra's +cut suffix for sub-word addresses) instead of being dropped.

What a caller sees:

  • Empty removed_blocks and unmapped_warnings: the function was decompiled and no removals were reported.
  • Absent field: the provider produced no pseudocode or doesn't report removals.
  • Blocks removed: the dossier gets a limitation for mapped removals and another for unmapped ones.

Native pseudocode views change for every provider, not only when blocks were removed. When the dossier has no decompiler_completeness record, the view adds this to unknowns:

decompiler_completeness was not recorded; blocks the decompiler may have removed from this pseudocode are unknown

That covers Hopper and IDA views, and Ghidra results cached under the old profile. When the record exists, the view adds a count entry for removed blocks and another for unmapped warnings, and adds nothing when both are empty. A new decompiler_removed_blocks native view pages the mapped entries.

Change and scope

  • Bridge: analyzeFunctionObservation parses the warnings before the pseudocode is replaced with "" for unavailable output.
    • Only whole-line WARNING: Removing unreachable block comments in the header (before the body's opening {) count. A string literal in the body can't match.
    • Each start is mapped into the program's address space, and end and the instructions come from Ghidra's BasicBlockModel.
  • Caller plate comments (ffc09fc, for the review on ce4e63c): Ghidra sends the entry-point plate comment to the decompiler as a header comment, and PrintC::emitCommentFuncHeader prints it with the same line renderer as the decompiler's own warnings. So a plate comment that spells a removal warning used to be counted as a real removal.
    • The bridge now renders the entry plate the way PrintLanguage::emitLineComment does at Ghidra_12.1.4_build: C comment delimiters, \r dropped, tabs as spaces, and \n starting a new line.
    • It then skips one matching header line for each matching line the caller wrote. A decompiler warning with the same text is printed separately and still counts.
    • The plate text itself stays in comments.
  • Schema: a new optional strict decompiler_completeness field on the shared dossier schema. The Ghidra refinement checks that addresses are canonical and starts are unique, and that assembly is present only when a range was mapped.
  • Profile: the Ghidra function_dossier_coverage key moves to observed-facets-call-relationships-and-decompiler-removals-v2, so older cached snapshots aren't replayed under the new contract.
  • Not included: the per-binary count in analyze that the issue mentions.
  • Related work: #2904 also adds a decompiler_completeness field to the dossier schema, with a different strict shape (status, warnings, dropped_blocks with end: null). The two can't both land unchanged. Whichever lands second has to adopt the other's shape. I'm happy to rework this one onto #2904's field if that's the maintainers' preference (see the comment below).

Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
Unresolved scope, authority, side effects, or required verification block
integration. Link the required maintainer scope decision for new providers or
capabilities involving target execution, injection, mutation, or protection
recovery; ordinary fixes within existing boundaries may state "not applicable".
Draft PRs may leave the scope decision pending and the readiness and merge checklists
unchecked. Maintainers can help resolve open questions.

Contract and boundary impact

  • Semantic owner and earliest changed stage: Ghidra bridge analyze_function (bridge/ghidra/ReaGhidraBridge.java), then the shared dossier schema (functionDossierSchema in src/domain/hopperValues.ts).
  • Contribution scope fit and linked maintainer decision: Within existing boundaries (adds read-only static-analysis output). Maintainer decision not applicable.
  • CLI and MCP/tool-catalog contract: Additive. A new optional decompiler_completeness field in analyze_function dossiers and a new native view facet decompiler_removed_blocks. Native pseudocode views gain unknowns entries: one whenever the record is absent (every Hopper and IDA view, and Ghidra results under the old profile), and counts when blocks were removed or warnings were unmapped.
  • Provider, bridge, target-format, or platform compatibility: Ghidra only. Hopper and IDA dossiers leave the field absent. The Ghidra analysis profile key changes.
  • Evidence, artifact, provenance, or reconstruction contract: function_dossier_coverage moves to observed-facets-call-relationships-and-decompiler-removals-v2, so earlier snapshots aren't reused as current results. start comes from Ghidra's decompiler warning, and end and assembly come from the listing. Caller-authored plate lines are excluded, not counted.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): Not applicable.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact (CI generates and validates ignored catalogs and skills/; edit authored instructions in .agents/skills/reverse-engineer-anything/ and never commit build outputs): No authored metadata changes. CI-generated catalogs may reflect the new optional field and view facet.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated:

    • src/ghidra/GhidraFunctionValues.test.ts: accepts removed-block records, including an unmapped one with end: null, and tells an empty record apart from an absent one. Rejects non-canonical addresses, unknown reasons, assembly without a mapped range, unexpected fields, duplicate starts, and a record missing unmapped_warnings.

    • src/domain/analysisView/analysisView.test.ts: the pseudocode view adds the "not recorded" unknown when the field is absent, adds nothing when both lists are empty, and adds counts for removed and unmapped blocks. The decompiler_removed_blocks view pages entries and reports "not recorded" when the field is absent.

    • New fixture tests/conformance/ghidra/decompiler-removed-block.S:

      • rea_removed_block branches on a register it just set to zero, so Ghidra provably drops the block at entry+0x10.
      • rea_reachable_block is a control that branches on its argument.
      • rea_warning_text is a decoy whose string literal spells a removal warning.
      • All three are checked in verify:ghidra:aarch64-jump-table on AArch64 ELF objects, and on Mach-O objects on an arm64 macOS host.

    • New real-Ghidra lane scripts/verify-real-ghidra-warning-provenance.mjs with tests/conformance/ghidra/ReaDecompilerWarningProvenanceProbe.java. It adds entry plate comments to the fixture in a real analysis database and drives the production bridge in-process, because REA's tools can't set plate comments. It checks that:

      • a plate spelling a removal warning for a valid in-body address, or for an unknown space, is rendered as a header line, isn't recorded, adds no omission limitation, and stays in comments;
      • a plate repeating the real warning leaves exactly the one real removal.

      It isn't wired into package.json or a workflow, and the README says to run it by hand, the same as verify:ghidra:aarch64-jump-table.

  • Base reproduction or other evidence: On the base revision the dossier only has the warning comment inside the pseudocode, so the lane's decompiler_completeness check fails. (This follows from the code. I didn't run the lane on base.) With the bridge from ce4e63c, the provenance lane fails because the plate decoy is recorded as a removed block. With this head, the lanes report:
    • ELF: {"start":"0x100010","end":"0x100018","reason":"unreachable","assembly":["0x100010: mov w0, #0x77","0x100014: ret"]}
    • Mach-O: {"start":"0x10","end":"0x18","reason":"unreachable","assembly":["0x10: mov w0, #0x77","0x14: ret"]}
    • Control and decoy functions: no removed blocks and no unmapped warnings, on both formats.
    • Provenance lane: all eight probe checks pass on both formats.
  • User-visible CLI/MCP output (if applicable): the new dossier field as shown above, plus this limitation: Ghidra's decompiler removed 1 block(s) it judged unreachable, so the pseudocode omits their code; decompiler_completeness.removed_blocks lists each range with its instructions. Native pseudocode views without a record show decompiler_completeness was not recorded; blocks the decompiler may have removed from this pseudocode are unknown in unknowns.
  • Remaining proof gaps:
    • Ghidra's warning names only a block's start. end is the end of the listing block containing that start, so if the listing splits a removed region differently from the decompiler, a range can be shorter than what was actually dropped.
    • The plate exclusion reproduces Ghidra 12.1.4's header rendering. A later Ghidra that renders header comments differently would need the renderer updated, and the provenance lane would catch that.
    • Only checked on AArch64 under macOS arm64. Linux and Windows hosts weren't run, and the MIPS busybox corpus from the issue wasn't reproduced.
    • No per-binary count yet.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

At head f6334dd, macOS arm64, Ghidra 12.1.4, Temurin 21:

  • npm run test:focused -- src/ghidra/GhidraFunctionValues.test.ts src/domain/analysisView/analysisView.test.ts src/ghidra/GhidraProvider.behavior.test.ts: passed
  • npm run check:fast: passed
  • npm run lint: passed
  • npm run knip: passed
  • npm run docs:check: passed
  • npm run verify:ghidra:aarch64-jump-table: passed. The removed-block results are listed above.
  • npm run build:cached && node scripts/verify-real-ghidra-warning-provenance.mjs: passed on ELF and Mach-O
  • The same provenance lane with bridge/ghidra/ReaGhidraBridge.java from ce4e63c: failed, as expected

Compatibility, safety, and release

  • Breaking changes or migration steps: None. The field and the view facet are additive and optional. The new pseudocode unknowns entry is additive. Cached Ghidra snapshots from the old profile won't be replayed.
  • Real Hopper/Ghidra, browser, or OS coverage: Real Ghidra 12.1.4 on macOS arm64, with AArch64 ELF and Mach-O objects. Hopper and IDA aren't changed, apart from the new pseudocode-view unknown.
  • Package or release metadata impact: None.
  • Security, privacy, process, or containment review: Read-only analysis output. No new process, network, or target interaction. The provenance lane writes plate comments only into its own temporary, read-only-imported project.

Author readiness

Human owner: @RishTheFish21

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4237 **Author:** [@RishTheFish21](https://github.com/RishTheFish21) **Created:** 10/15/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/2735-decompiler-completeness` --- ### 📝 Commits (8) - [`88f543d`](https://github.com/morluto/rea/commit/88f543df89c50430446567a4bf06b8c173c57bed) feat(ghidra): report blocks the decompiler removed as unreachable - [`4334dd3`](https://github.com/morluto/rea/commit/4334dd3c62df63716f82712e8d1b56eaef95749e) fix(ghidra): only count whole-line header removal warnings - [`5ddaab7`](https://github.com/morluto/rea/commit/5ddaab7946eced9fd2b473fdb6ef0231639f9e69) Merge origin/main into feat/2735-decompiler-completeness - [`ce4e63c`](https://github.com/morluto/rea/commit/ce4e63c92eb37c201452ca055542409a508bbcc8) fix(ghidra): keep sub-word warning identities and flag unrecorded completeness - [`ffc09fc`](https://github.com/morluto/rea/commit/ffc09fcea18b79bb0a8ce556764575d205f54d53) fix(ghidra): exclude caller plate comments from removed-block warnings - [`60d75d5`](https://github.com/morluto/rea/commit/60d75d5962d00049d32605536e89fb1a0cee304b) Merge main into feat/2735-decompiler-completeness - [`1874faf`](https://github.com/morluto/rea/commit/1874fafa09e9b91b0fbb800e26063eeb60c5eb64) test(ghidra): align the removed-block checks with current main - [`f6334dd`](https://github.com/morluto/rea/commit/f6334dd237f187d9097c6445b2295e5939f22f04) Merge commit 'd0067ba34a39ea47dd7e11bb2e6169bc0938cc3f' into feat/2735-decompiler-completeness ### 📊 Changes **18 files changed** (+939 additions, -37 deletions) <details> <summary>View changed files</summary> 📝 `bridge/ghidra/ReaGhidraBridge.java` (+144 -0) 📝 `docs/mcp-contracts.md` (+17 -0) 📝 `docs/native-investigation.md` (+3 -1) 📝 `knip.json` (+1 -0) 📝 `scripts/verify-real-ghidra-function.mjs` (+87 -0) ➕ `scripts/verify-real-ghidra-warning-provenance.mjs` (+150 -0) 📝 `scripts/verify-real-ghidra.mjs` (+68 -33) 📝 `src/domain/analysisView/analysisView.test.ts` (+61 -0) 📝 `src/domain/analysisView/analysisView.ts` (+1 -0) 📝 `src/domain/analysisView/nativeFunctionView.ts` (+23 -0) 📝 `src/domain/hopperValues.ts` (+32 -0) 📝 `src/ghidra/GhidraAnalysisProfile.ts` (+3 -2) 📝 `src/ghidra/GhidraFunctionValues.test.ts` (+78 -0) 📝 `src/ghidra/GhidraFunctionValues.ts` (+13 -0) 📝 `src/ghidra/GhidraProvider.behavior.test.ts` (+2 -1) 📝 `tests/conformance/ghidra/README.md` (+11 -0) ➕ `tests/conformance/ghidra/ReaDecompilerWarningProvenanceProbe.java` (+188 -0) ➕ `tests/conformance/ghidra/decompiler-removed-block.S` (+57 -0) </details> ### 📄 Description ## Summary Ghidra's decompiler sometimes drops blocks it decides are unreachable. The only sign of this is a header comment in the pseudocode (`/* WARNING: Removing unreachable block (ram,0x...) */`). REA passes that comment through, but the dossier has no structured signal, and agents tend to skip comments and treat the pseudocode as complete. #2735 describes a case where an auth comparison sat entirely in a dropped block. `analyze_function` dossiers from Ghidra now include `decompiler_completeness`. Its `removed_blocks` lists each dropped block's range and instructions, and its `unmapped_warnings` lists removal warnings whose address can't be placed in the function. Native `pseudocode` views now also say when no record exists at all (see below). Fixes #2735 ## Problem and expected behavior Before: when Ghidra removes a block, the only record is a comment inside the pseudocode. Nothing structured tells a caller that the pseudocode is missing code, or where the missing code is. After: each removed block appears as: - `start`: parsed from Ghidra's per-block warning. I checked the format against `Funcdata::removeUnreachableBlocks` in the decompiler source, which emits one warning per removed block with its start address. - `end`: the exclusive end of the listing basic block containing `start`, or `null` if the listing has no block there. - `reason`: `"unreachable"`. - `assembly`: the block's instructions from the listing. Empty when `end` is `null`. A warning whose address can't be mapped into the analyzed function is kept in `unmapped_warnings` as `"<space>,<offset>"` (with Ghidra's `+cut` suffix for sub-word addresses) instead of being dropped. What a caller sees: - **Empty `removed_blocks` and `unmapped_warnings`:** the function was decompiled and no removals were reported. - **Absent field:** the provider produced no pseudocode or doesn't report removals. - **Blocks removed:** the dossier gets a limitation for mapped removals and another for unmapped ones. **Native `pseudocode` views change for every provider, not only when blocks were removed.** When the dossier has no `decompiler_completeness` record, the view adds this to `unknowns`: > decompiler_completeness was not recorded; blocks the decompiler may have removed from this pseudocode are unknown That covers Hopper and IDA views, and Ghidra results cached under the old profile. When the record exists, the view adds a count entry for removed blocks and another for unmapped warnings, and adds nothing when both are empty. A new `decompiler_removed_blocks` native view pages the mapped entries. ## Change and scope - **Bridge:** `analyzeFunctionObservation` parses the warnings before the pseudocode is replaced with `""` for unavailable output. - Only whole-line `WARNING: Removing unreachable block` comments in the header (before the body's opening `{`) count. A string literal in the body can't match. - Each `start` is mapped into the program's address space, and `end` and the instructions come from Ghidra's `BasicBlockModel`. - **Caller plate comments (ffc09fc, for the review on ce4e63c):** Ghidra sends the entry-point plate comment to the decompiler as a header comment, and `PrintC::emitCommentFuncHeader` prints it with the same line renderer as the decompiler's own warnings. So a plate comment that spells a removal warning used to be counted as a real removal. - The bridge now renders the entry plate the way `PrintLanguage::emitLineComment` does at `Ghidra_12.1.4_build`: C comment delimiters, `\r` dropped, tabs as spaces, and `\n` starting a new line. - It then skips one matching header line for each matching line the caller wrote. A decompiler warning with the same text is printed separately and still counts. - The plate text itself stays in `comments`. - **Schema:** a new optional strict `decompiler_completeness` field on the shared dossier schema. The Ghidra refinement checks that addresses are canonical and starts are unique, and that `assembly` is present only when a range was mapped. - **Profile:** the Ghidra `function_dossier_coverage` key moves to `observed-facets-call-relationships-and-decompiler-removals-v2`, so older cached snapshots aren't replayed under the new contract. - **Not included:** the per-binary count in `analyze` that the issue mentions. - **Related work:** #2904 also adds a `decompiler_completeness` field to the dossier schema, with a different strict shape (`status`, `warnings`, `dropped_blocks` with `end: null`). The two can't both land unchanged. Whichever lands second has to adopt the other's shape. I'm happy to rework this one onto #2904's field if that's the maintainers' preference (see the comment below). Follow the [[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope)](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). Unresolved scope, authority, side effects, or required verification block integration. Link the required maintainer scope decision for new providers or capabilities involving target execution, injection, mutation, or protection recovery; ordinary fixes within existing boundaries may state "not applicable". Draft PRs may leave the scope decision pending and the readiness and merge checklists unchecked. Maintainers can help resolve open questions. ## Contract and boundary impact - Semantic owner and earliest changed stage: Ghidra bridge `analyze_function` (`bridge/ghidra/ReaGhidraBridge.java`), then the shared dossier schema (`functionDossierSchema` in `src/domain/hopperValues.ts`). - Contribution scope fit and linked maintainer decision: Within existing boundaries (adds read-only static-analysis output). Maintainer decision not applicable. - CLI and MCP/tool-catalog contract: Additive. A new optional `decompiler_completeness` field in `analyze_function` dossiers and a new native view facet `decompiler_removed_blocks`. Native `pseudocode` views gain `unknowns` entries: one whenever the record is absent (every Hopper and IDA view, and Ghidra results under the old profile), and counts when blocks were removed or warnings were unmapped. - Provider, bridge, target-format, or platform compatibility: Ghidra only. Hopper and IDA dossiers leave the field absent. The Ghidra analysis profile key changes. - Evidence, artifact, provenance, or reconstruction contract: `function_dossier_coverage` moves to `observed-facets-call-relationships-and-decompiler-removals-v2`, so earlier snapshots aren't reused as current results. `start` comes from Ghidra's decompiler warning, and `end` and `assembly` come from the listing. Caller-authored plate lines are excluded, not counted. - Target selection, authority, upstream behavior, and cleanup details (if applicable): Not applicable. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs): No authored metadata changes. CI-generated catalogs may reflect the new optional field and view facet. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - `src/ghidra/GhidraFunctionValues.test.ts`: accepts removed-block records, including an unmapped one with `end: null`, and tells an empty record apart from an absent one. Rejects non-canonical addresses, unknown reasons, assembly without a mapped range, unexpected fields, duplicate starts, and a record missing `unmapped_warnings`. - `src/domain/analysisView/analysisView.test.ts`: the pseudocode view adds the "not recorded" unknown when the field is absent, adds nothing when both lists are empty, and adds counts for removed and unmapped blocks. The `decompiler_removed_blocks` view pages entries and reports "not recorded" when the field is absent. - New fixture `tests/conformance/ghidra/decompiler-removed-block.S`: - `rea_removed_block` branches on a register it just set to zero, so Ghidra provably drops the block at entry+0x10. - `rea_reachable_block` is a control that branches on its argument. - `rea_warning_text` is a decoy whose string literal spells a removal warning. - All three are checked in `verify:ghidra:aarch64-jump-table` on AArch64 ELF objects, and on Mach-O objects on an arm64 macOS host. - New real-Ghidra lane `scripts/verify-real-ghidra-warning-provenance.mjs` with `tests/conformance/ghidra/ReaDecompilerWarningProvenanceProbe.java`. It adds entry plate comments to the fixture in a real analysis database and drives the production bridge in-process, because REA's tools can't set plate comments. It checks that: - a plate spelling a removal warning for a valid in-body address, or for an unknown space, is rendered as a header line, isn't recorded, adds no omission limitation, and stays in `comments`; - a plate repeating the real warning leaves exactly the one real removal. It isn't wired into `package.json` or a workflow, and the README says to run it by hand, the same as `verify:ghidra:aarch64-jump-table`. - Base reproduction or other evidence: On the base revision the dossier only has the warning comment inside the pseudocode, so the lane's `decompiler_completeness` check fails. (This follows from the code. I didn't run the lane on base.) With the bridge from ce4e63c, the provenance lane fails because the plate decoy is recorded as a removed block. With this head, the lanes report: - ELF: `{"start":"0x100010","end":"0x100018","reason":"unreachable","assembly":["0x100010: mov w0, #0x77","0x100014: ret"]}` - Mach-O: `{"start":"0x10","end":"0x18","reason":"unreachable","assembly":["0x10: mov w0, #0x77","0x14: ret"]}` - Control and decoy functions: no removed blocks and no unmapped warnings, on both formats. - Provenance lane: all eight probe checks pass on both formats. - User-visible CLI/MCP output (if applicable): the new dossier field as shown above, plus this limitation: `Ghidra's decompiler removed 1 block(s) it judged unreachable, so the pseudocode omits their code; decompiler_completeness.removed_blocks lists each range with its instructions.` Native pseudocode views without a record show `decompiler_completeness was not recorded; blocks the decompiler may have removed from this pseudocode are unknown` in `unknowns`. - Remaining proof gaps: - Ghidra's warning names only a block's start. `end` is the end of the listing block containing that start, so if the listing splits a removed region differently from the decompiler, a range can be shorter than what was actually dropped. - The plate exclusion reproduces Ghidra 12.1.4's header rendering. A later Ghidra that renders header comments differently would need the renderer updated, and the provenance lane would catch that. - Only checked on AArch64 under macOS arm64. Linux and Windows hosts weren't run, and the MIPS busybox corpus from the issue wasn't reproduced. - No per-binary count yet. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed At head `f6334dd`, macOS arm64, Ghidra 12.1.4, Temurin 21: - `npm run test:focused -- src/ghidra/GhidraFunctionValues.test.ts src/domain/analysisView/analysisView.test.ts src/ghidra/GhidraProvider.behavior.test.ts`: passed - `npm run check:fast`: passed - `npm run lint`: passed - `npm run knip`: passed - `npm run docs:check`: passed - `npm run verify:ghidra:aarch64-jump-table`: passed. The removed-block results are listed above. - `npm run build:cached && node scripts/verify-real-ghidra-warning-provenance.mjs`: passed on ELF and Mach-O - The same provenance lane with `bridge/ghidra/ReaGhidraBridge.java` from ce4e63c: failed, as expected ## Compatibility, safety, and release - Breaking changes or migration steps: None. The field and the view facet are additive and optional. The new pseudocode `unknowns` entry is additive. Cached Ghidra snapshots from the old profile won't be replayed. - Real Hopper/Ghidra, browser, or OS coverage: Real Ghidra 12.1.4 on macOS arm64, with AArch64 ELF and Mach-O objects. Hopper and IDA aren't changed, apart from the new pseudocode-view unknown. - Package or release metadata impact: None. - Security, privacy, process, or containment review: Read-only analysis output. No new process, network, or target interaction. The provenance lane writes plate comments only into its own temporary, read-only-imported project. ## Author readiness Human owner: @RishTheFish21 - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels   
area: bridge

   
area: bridge

   
area: build

   
area: build

   
area: build

   
area: cli

   
area: domain

   
area: evidence

   
area: mcp

   
area: native

   
area: providers

   
area: runtime

   
area: server

   
area: setup

   
area: tests

   
bug

   
design

   
documentation

   
enhancement

   
enhancement

   
enhancement

   
experiment

   
help wanted

   
i18n

   
invalid

   
P0: critical

   
P1: high

   
P2: medium

   
P2: medium

   
P2: medium

   
P3: low

   
performance

   
provider-request

   
pull-request
Mirrored from GitHub Pull Request

  
question

   
roadmap

   
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6140
No description provided.