[GH-ISSUE #8189] [Bug] DOS verifier compares fresh CLI symbol provenance with a renamed MCP session #6472

Closed
opened 2026-11-01 21:11:25 +00:00 by bsakaguchi · 2 comments
Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8189

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

  • I searched the open and closed issues for a related report.
  • I removed credentials, tokens, private paths, and sensitive target data.

Area

Tests or verification

Observed behavior

On macOS arm64, npm run verify:ghidra:dos reaches the real Ghidra CLI/MCP checks but fails the final read_function_instructions parity assertion: the CLI reports the entry symbol source as imported, while the already-edited MCP session reports user_defined. The instructions, function identity and body shown in the assertion match.

Reproduction

Run npm run verify:ghidra:dos with Ghidra 12.1.4, JDK 21, GHIDRA_INSTALL_DIR, and JAVA_HOME. In an independent real MCP session on the verifier's generated MZ fixture: open it with Ghidra, read function instructions at 0x10000, save address_name, rename the entry with set_address_name, restore the old name with set_address_name, and read instructions again. Both name operations return true; the symbol source changes from imported to user_defined while its name returns to entry.

Expected behavior

Compare CLI and MCP results at equivalent session states, or explicitly account for the changed provenance caused by the verifier's own edit. Retain coverage for alias collision rejection, restoration of the display name, and the actual symbol source; do not hide a provenance difference by declaring it identical.

Impact and scope

The optional DOS lane fails on this host even though it reached its final comparison. This does not establish a general CLI/MCP parity bug: the CLI uses a fresh session while MCP has undergone a rename and a rename back. The verifier reports no completed success or cleanup summary after the assertion; no broad host coverage is claimed.

REA version

d48e472fa plus unrelated Kuna baseline changes from #8187.

Runtime and platform

macOS arm64; Node 24.22.1; Ghidra 12.1.4; JDK 21.

Target and artifact context

The source-owned verifier MZ fixture, SHA-256 1ed3d7160764fd5b61325b8212046ac451c7639a9bcc1f713e4c3785bc7ccc15; not executed. The independent test checked that the file digest after closing still matched its initial digest.

Interface

Real CLI, stdio MCP, and verification script.

Earliest failure stage

Verifier parity assertion after a session annotation and restoration by name.

Relevant output or evidence

The assertion at scripts/verify-real-ghidra-dos.mjs:387 reports only source: imported versus source: user_defined in its printed diff. The boundary helper first uses set_addresses_names for a temporary alias and finally calls set_address_name to restore the original name. A separate MCP reproduction confirmed the source change without relying on the verifier's comparison.

Typed outcome or recovery expected

A full DOS verifier run should accurately compare equivalent observations, report cleanup, and continue testing rejected oversized coordinates and alias collisions.

Analysis or suspected cause

Restoring a symbol name through an explicit user edit does not restore its prior Ghidra source type. The verifier subsequently compares that edited live session against a fresh CLI import. Whether REA should expose a separate reversible annotation workflow is a broader design question, not established by this verifier failure.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8189 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Tests or verification ### Observed behavior On macOS arm64, `npm run verify:ghidra:dos` reaches the real Ghidra CLI/MCP checks but fails the final `read_function_instructions` parity assertion: the CLI reports the entry symbol source as `imported`, while the already-edited MCP session reports `user_defined`. The instructions, function identity and body shown in the assertion match. ### Reproduction Run `npm run verify:ghidra:dos` with Ghidra 12.1.4, JDK 21, `GHIDRA_INSTALL_DIR`, and `JAVA_HOME`. In an independent real MCP session on the verifier's generated MZ fixture: open it with Ghidra, read function instructions at `0x10000`, save `address_name`, rename the entry with `set_address_name`, restore the old name with `set_address_name`, and read instructions again. Both name operations return true; the symbol source changes from `imported` to `user_defined` while its name returns to `entry`. ### Expected behavior Compare CLI and MCP results at equivalent session states, or explicitly account for the changed provenance caused by the verifier's own edit. Retain coverage for alias collision rejection, restoration of the display name, and the actual symbol source; do not hide a provenance difference by declaring it identical. ### Impact and scope The optional DOS lane fails on this host even though it reached its final comparison. This does not establish a general CLI/MCP parity bug: the CLI uses a fresh session while MCP has undergone a rename and a rename back. The verifier reports no completed success or cleanup summary after the assertion; no broad host coverage is claimed. ### REA version `d48e472fa` plus unrelated Kuna baseline changes from #8187. ### Runtime and platform macOS arm64; Node 24.22.1; Ghidra 12.1.4; JDK 21. ### Target and artifact context The source-owned verifier MZ fixture, SHA-256 `1ed3d7160764fd5b61325b8212046ac451c7639a9bcc1f713e4c3785bc7ccc15`; not executed. The independent test checked that the file digest after closing still matched its initial digest. ### Interface Real CLI, stdio MCP, and verification script. ### Earliest failure stage Verifier parity assertion after a session annotation and restoration by name. ### Relevant output or evidence The assertion at `scripts/verify-real-ghidra-dos.mjs:387` reports only `source: imported` versus `source: user_defined` in its printed diff. The boundary helper first uses `set_addresses_names` for a temporary alias and finally calls `set_address_name` to restore the original name. A separate MCP reproduction confirmed the source change without relying on the verifier's comparison. ### Typed outcome or recovery expected A full DOS verifier run should accurately compare equivalent observations, report cleanup, and continue testing rejected oversized coordinates and alias collisions. ### Analysis or suspected cause Restoring a symbol name through an explicit user edit does not restore its prior Ghidra source type. The verifier subsequently compares that edited live session against a fresh CLI import. Whether REA should expose a separate reversible annotation workflow is a broader design question, not established by this verifier failure.
bsakaguchi 2026-11-01 21:11:25 +00:00
  • closed this issue
  • added the
    bug
    triage
    labels
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

<!-- gh-comment-id:6538790622 --> @morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.
Author
Owner

@morluto commented on GitHub (Oct 31, 2026):

Current-main paired real-provider check on macOS arm64, Node 24.22.1, Ghidra 12.1.4 and explicit JDK 21: unmodified REA main 60149282 still fails npm run verify:ghidra:dos at the same final instruction-window comparison. The fresh CLI observation's symbol source is imported; the MCP observation after rename and restoration is user_defined. This remains a verifier comparison between different session histories, not a reproduced CLI/MCP product discrepancy.

In a detached worktree on the same main, I applied open PR #8339 head 4b6ee4aca99a138badf0358a25a30fb64bb1c9ba without committing; its one changed file exactly matches the PR head. The same full real DOS verifier passed with ok: true and cleanup: complete. Its report covered the entry, near and far functions, independent instruction-byte/call-target oracles, invalid and truncated address rejection, unchanged original fixture, and a verified verifier lineage with no descendants. The PR compares the CLI against MCP before its own rename, then separately asserts the restored name's user_defined provenance and unchanged remaining window.

This is one host, Ghidra/JDK version and source-owned 192-byte DOS MZ fixture; the target was not executed. It does not establish Windows support, arbitrary DOS correctness or hosted-CI coverage.

<!-- gh-comment-id:6546578242 --> @morluto commented on GitHub (Oct 31, 2026): Current-main paired real-provider check on macOS arm64, Node 24.22.1, Ghidra 12.1.4 and explicit JDK 21: unmodified REA main `60149282` still fails `npm run verify:ghidra:dos` at the same final instruction-window comparison. The fresh CLI observation's symbol source is `imported`; the MCP observation after rename and restoration is `user_defined`. This remains a verifier comparison between different session histories, not a reproduced CLI/MCP product discrepancy. In a detached worktree on the same main, I applied open PR #8339 head `4b6ee4aca99a138badf0358a25a30fb64bb1c9ba` without committing; its one changed file exactly matches the PR head. The same full real DOS verifier passed with `ok: true` and `cleanup: complete`. Its report covered the entry, near and far functions, independent instruction-byte/call-target oracles, invalid and truncated address rejection, unchanged original fixture, and a verified verifier lineage with no descendants. The PR compares the CLI against MCP before its own rename, then separately asserts the restored name's `user_defined` provenance and unchanged remaining window. This is one host, Ghidra/JDK version and source-owned 192-byte DOS MZ fixture; the target was not executed. It does not establish Windows support, arbitrary DOS correctness or hosted-CI coverage.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6472
No description provided.