morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3767] [Bug] DOS MZ: table-dispatched handlers are invisible to procedure tools (352 of 540 functions) #1466

New issue

Open

opened 2026-10-16 21:05:43 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:05:43 +00:00

Owner

Originally created by @max-freddyfire on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3767

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

On a 16-bit DOS MZ executable, Ghidra-backed analysis exposes 352 procedures, but the program has at least 540 executed functions. The 188 missing functions (about 15.5 KB of 47.3 KB of code, 33 %) are handlers reached only through runtime-filled tables. For those addresses every procedure_* tool returns not_found ("No procedure exists at the requested address"), and resolve_containing_procedure returns found: false, reason: not_in_procedure. Nothing in REA can disassemble or decompile these handlers.

Reproduction

open_binary on an unpacked 16-bit DOS MZ program that dispatches through CALL/JMP word ptr [reg+table], where the tables are filled at runtime with MOV word ptr [..], imm16.
list_procedures returns 352 entries.
procedure_assembly 0x14b0c (a handler written into a dispatch table by an immediate store) returns not_found. The same happens with read_function_instructions and procedure_info.
resolve_containing_procedure 0x14b0c returns not_in_procedure.

Ran it on the MCP interface. rea analyze on the CLI reports the same procedure count.

Expected behavior

There is a supported way to analyze code that is reached only through indirect dispatch. For example, the caller declares function entry points, or REA follows immediate code pointers stored into known dispatch tables. Then procedure_* tools work on those handlers. Until then, the gap stays an explicit limitation rather than a plain not_found.

Impact and scope

High for this target. Most of the game logic (combat, monster AI, hit handlers, script opcodes) sits in table-dispatched handlers that REA cannot see. We used an independent function inventory from a separate Ghidra headless run plus a custom emulator trace instead. Likely affects other real-mode programs that use runtime-filled jump tables. Not tested on other targets.

REA version

6.5.0

Runtime and platform

Linux (WSL2, kernel 6.18), Node.js 24.13.0, OpenJDK 21.0.12.1, Ghidra 12.1.4

Target and artifact context

Commercial 1991 DOS game executable, unpacked (PKLITE and EXEPACK removed) before analysis. Profile: dos-mz, x86:LE:16:Real Mode, load segment 0x1000, linear addresses (1000:4B0C = 0x14b0c). The binary cannot be shared.

Interface

MCP

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

list_procedures: 352 procedures (ev_74914008556906d9e95d7b522bfb2650438e66af56e1955ff24831296e6726d8). rea analyze gives the same count (ev_146659192996d094822a1f0b969371040ad26b5087dd4bbb3b5aa5a5d9d2df5b).
resolve_containing_procedure 0x14b0c returns not_in_procedure (ev_1507e2fbd65089ce89cdf6cadb5f97e8a735c37bd59d35632c8fd0159ef052f7).
resolve_containing_procedure 0x12fad returns not_in_procedure (ev_4ad293cedbd7c1ba78ffbee8f3be2aec71b45391ce84b426dc4fe8e469e9d598).
procedure_assembly, read_function_instructions and procedure_info on 0x14b0c and 0x12deb return not_found with "No procedure exists at the requested address".

Typed outcome or recovery expected

A typed limitation saying that the address holds no analyzed code, possibly reachable through indirect dispatch, plus a supported recovery path (declared entry points or table-following). This would replace a plain not_found that hides that the bytes are unanalyzed code.

Analysis or suspected cause

Source-derived, not verified inside REA. Ghidra's default auto-analysis does not treat 16-bit immediates stored into memory as code pointers. The handler bytes are therefore never disassembled, and no function is created. The table-filling stores are plain MOV word ptr [..], imm16 instructions at a handful of sites.

Additional context

A related observation, not verified further: procedure_pseudo_code on one of these addresses falls back to treating the number as a file offset ("File offset 0x14b0c maps to analysis address 0x2360c, but no procedure ..."). That message is easy to misread as a different function.

Originally created by @max-freddyfire on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3767 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior On a 16-bit DOS MZ executable, Ghidra-backed analysis exposes 352 procedures, but the program has at least 540 executed functions. The 188 missing functions (about 15.5 KB of 47.3 KB of code, 33 %) are handlers reached only through runtime-filled tables. For those addresses every `procedure_*` tool returns `not_found` ("No procedure exists at the requested address"), and `resolve_containing_procedure` returns `found: false, reason: not_in_procedure`. Nothing in REA can disassemble or decompile these handlers. ### Reproduction 1. `open_binary` on an unpacked 16-bit DOS MZ program that dispatches through `CALL/JMP word ptr [reg+table]`, where the tables are filled at runtime with `MOV word ptr [..], imm16`. 2. `list_procedures` returns 352 entries. 3. `procedure_assembly 0x14b0c` (a handler written into a dispatch table by an immediate store) returns `not_found`. The same happens with `read_function_instructions` and `procedure_info`. 4. `resolve_containing_procedure 0x14b0c` returns `not_in_procedure`. Ran it on the MCP interface. `rea analyze` on the CLI reports the same procedure count. ### Expected behavior There is a supported way to analyze code that is reached only through indirect dispatch. For example, the caller declares function entry points, or REA follows immediate code pointers stored into known dispatch tables. Then `procedure_*` tools work on those handlers. Until then, the gap stays an explicit limitation rather than a plain `not_found`. ### Impact and scope High for this target. Most of the game logic (combat, monster AI, hit handlers, script opcodes) sits in table-dispatched handlers that REA cannot see. We used an independent function inventory from a separate Ghidra headless run plus a custom emulator trace instead. Likely affects other real-mode programs that use runtime-filled jump tables. Not tested on other targets. ### REA version 6.5.0 ### Runtime and platform Linux (WSL2, kernel 6.18), Node.js 24.13.0, OpenJDK 21.0.12.1, Ghidra 12.1.4 ### Target and artifact context Commercial 1991 DOS game executable, unpacked (PKLITE and EXEPACK removed) before analysis. Profile: `dos-mz`, `x86:LE:16:Real Mode`, load segment 0x1000, linear addresses (`1000:4B0C` = `0x14b0c`). The binary cannot be shared. ### Interface MCP ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence - `list_procedures`: 352 procedures (`ev_74914008556906d9e95d7b522bfb2650438e66af56e1955ff24831296e6726d8`). `rea analyze` gives the same count (`ev_146659192996d094822a1f0b969371040ad26b5087dd4bbb3b5aa5a5d9d2df5b`). - `resolve_containing_procedure 0x14b0c` returns `not_in_procedure` (`ev_1507e2fbd65089ce89cdf6cadb5f97e8a735c37bd59d35632c8fd0159ef052f7`). - `resolve_containing_procedure 0x12fad` returns `not_in_procedure` (`ev_4ad293cedbd7c1ba78ffbee8f3be2aec71b45391ce84b426dc4fe8e469e9d598`). - `procedure_assembly`, `read_function_instructions` and `procedure_info` on `0x14b0c` and `0x12deb` return `not_found` with "No procedure exists at the requested address". ### Typed outcome or recovery expected A typed limitation saying that the address holds no analyzed code, possibly reachable through indirect dispatch, plus a supported recovery path (declared entry points or table-following). This would replace a plain `not_found` that hides that the bytes are unanalyzed code. ### Analysis or suspected cause Source-derived, not verified inside REA. Ghidra's default auto-analysis does not treat 16-bit immediates stored into memory as code pointers. The handler bytes are therefore never disassembled, and no function is created. The table-filling stores are plain `MOV word ptr [..], imm16` instructions at a handful of sites. ### Additional context A related observation, not verified further: `procedure_pseudo_code` on one of these addresses falls back to treating the number as a file offset ("File offset 0x14b0c maps to analysis address 0x2360c, but no procedure ..."). That message is easy to misread as a different function.

bsakaguchi commented

2026-10-16 21:05:44 +00:00

Author

Owner

@max-freddyfire commented on GitHub (Oct 15, 2026):

Workaround that works on 6.5.0, measured on the same target. It is not a fix, and it has caveats that point at what a real fix needs.

Seeded dos-com image. Take the MZ code segment from 1000:0100 up to its end (1000:D7F0) and write it as a COM image. REA loads it at 1000:0100 (linear 0x10100), so every address matches the original MZ analysis. Then patch 3 bytes: 1000:0100 becomes a JMP to a stub appended past the segment end. The stub has one JNZ +3; CALL <entry> per known function entry and ends with RET. The JNZ makes Ghidra follow both edges, so a no-return callee cannot cut the seeding short. The entry list came from an independent inventory: an earlier Ghidra headless run plus handlers derived from the dispatch tables.

open_binary {path: <seeded image>, format: "dos-com"}

Result. 540 procedures (ev_fb895bc731a0d4d40c2402773021074a173231b91365eb347014172f9676cc16), up from 352. 536 of 537 seeded entries became functions. procedure_assembly 0x12fad (ev_7125625bf492d368c3e5ae3045cee35779af65e913baa77458cbc9fb9d2629ca), procedure_pseudo_code 0x14b85 (ev_43b46709551a1f416d76bd2259e6b4d43aee3fb23ca2127afcf0e101aa10f461) and procedure_info 0x14298 (ev_60b137c9afb40e21802ee022e64a6d505359939c380cb2b8df68941bcbe9ee97) now return the handlers.

Caveats. These are why a supported path is still needed:

A COM image forces DS = CS = 0x1000. Memory operands keep the correct offsets, but DAT_1000_xxxx really means the program's own data segment. The decompiler then constant-folds code bytes it believes are data and removes branches ("Removing unreachable block": 21 in one handler, 22 in another). Assembly is reliable, but pseudocode is not, for handlers that test data-segment flags.
Evidence from the seeded image has a different subject digest than the original executable, so Evidence does not link back to the shipped artifact.
The 3 patched bytes and the synthetic stub show up in callers and the call graph and have to be ignored by hand.

Tested, does not help: annotate_native_function 0x12fad returns not_found, because it requires an existing function. set_address_name creates only a label. raw-binary with entry_point maps file offset 0 to the entry, so addresses no longer match unless the image is sliced per handler.

A caller-declared list of entry points on open_binary for the original MZ, keeping the real data segment and the shipped-artifact subject, would remove all three caveats.

@max-freddyfire commented on GitHub (Oct 15, 2026): Workaround that works on 6.5.0, measured on the same target. It is not a fix, and it has caveats that point at what a real fix needs. **Seeded `dos-com` image.** Take the MZ code segment from `1000:0100` up to its end (`1000:D7F0`) and write it as a COM image. REA loads it at `1000:0100` (linear `0x10100`), so every address matches the original MZ analysis. Then patch 3 bytes: `1000:0100` becomes a `JMP` to a stub appended past the segment end. The stub has one `JNZ +3; CALL <entry>` per known function entry and ends with `RET`. The `JNZ` makes Ghidra follow both edges, so a no-return callee cannot cut the seeding short. The entry list came from an independent inventory: an earlier Ghidra headless run plus handlers derived from the dispatch tables. ``` open_binary {path: <seeded image>, format: "dos-com"} ``` **Result.** 540 procedures (`ev_fb895bc731a0d4d40c2402773021074a173231b91365eb347014172f9676cc16`), up from 352. 536 of 537 seeded entries became functions. `procedure_assembly 0x12fad` (`ev_7125625bf492d368c3e5ae3045cee35779af65e913baa77458cbc9fb9d2629ca`), `procedure_pseudo_code 0x14b85` (`ev_43b46709551a1f416d76bd2259e6b4d43aee3fb23ca2127afcf0e101aa10f461`) and `procedure_info 0x14298` (`ev_60b137c9afb40e21802ee022e64a6d505359939c380cb2b8df68941bcbe9ee97`) now return the handlers. **Caveats. These are why a supported path is still needed:** 1. A COM image forces `DS = CS = 0x1000`. Memory operands keep the correct offsets, but `DAT_1000_xxxx` really means the program's own data segment. The decompiler then constant-folds code bytes it believes are data and removes branches ("Removing unreachable block": 21 in one handler, 22 in another). Assembly is reliable, but pseudocode is not, for handlers that test data-segment flags. 2. Evidence from the seeded image has a different subject digest than the original executable, so Evidence does not link back to the shipped artifact. 3. The 3 patched bytes and the synthetic stub show up in callers and the call graph and have to be ignored by hand. **Tested, does not help:** `annotate_native_function 0x12fad` returns `not_found`, because it requires an existing function. `set_address_name` creates only a label. `raw-binary` with `entry_point` maps file offset 0 to the entry, so addresses no longer match unless the image is sliced per handler. A caller-declared list of entry points on `open_binary` for the original MZ, keeping the real data segment and the shipped-artifact subject, would remove all three caveats.

bsakaguchi commented

2026-10-16 21:05:44 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Solid repro with hard numbers: 188 of 540 executed handlers are invisible purely because they are reached through runtime-filled dispatch tables, which makes 33% of code silent dead-zone for every procedure_* tool. The root cause is clear — table destinations are data until something tracks the immediate stores that fill them. Agree the most practical fix is a declared-entry-point mechanism (caller marks handler addresses, or rea follows immediate pointers written into known tables); that is exactly how REA already treats other indirect control flow, so it fits the model. Worth also reporting the count of unresolved indirect-dispatch targets in the analysis summary, so analysts know the gap exists instead of discovering it per-address.

@XinwangTao commented on GitHub (Oct 15, 2026): Solid repro with hard numbers: 188 of 540 executed handlers are invisible purely because they are reached through runtime-filled dispatch tables, which makes 33% of code silent dead-zone for every procedure_* tool. The root cause is clear — table destinations are data until something tracks the immediate stores that fill them. Agree the most practical fix is a declared-entry-point mechanism (caller marks handler addresses, or rea follows immediate pointers written into known tables); that is exactly how REA already treats other indirect control flow, so it fits the model. Worth also reporting the count of unresolved indirect-dispatch targets in the analysis summary, so analysts know the gap exists instead of discovering it per-address.

bsakaguchi referenced this issue

2026-10-17 21:11:40 +00:00

[PR #1466] [CLOSED] docs(process): clarify normalized terminal row geometry #2905

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1466

No description provided.

Rows
Columns