morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #3056] [Bug] Windows + IDA headless: analysis hangs for minutes and ends in cleanup_incomplete; the same supervisor driven directly answers in 2.4s #1071

New issue

Open

opened 2026-10-15 18:30:49 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:30:49 +00:00

Owner

Originally created by @alucardigo on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3056

Summary

On Windows, analysis through the ida provider (headless / database-supervisor profile) usually hangs for several minutes and then fails with cleanup_incomplete, while driving the same upstream supervisor directly over MCP stdio completes the same work in ~2.4 s. It is intermittent: one run out of several succeeded in 13 s with an unchanged configuration.

I could not establish a root cause, so this reports observed behaviour only.

Environment

rea-agents 6.4.0 (npm), invoked as node .../rea-agents/scripts/rea.mjs
Windows 11 x64, Node.js 24.18.0
IDA Pro 9.2 (licensed, idalib activated), REA_ANALYSIS_PROVIDER=ida
Upstream: mrexodia/ida-pro-mcp at c133c3853faa111a9b00ee615c013b720d0c4acd (the commit named in docs/ida-provider.md), installed in its own venv
Target: a 38 KB x86-64 PE (any small native PE reproduces it here)

REA_IDA_MCP_CONFIG registration:

{
  "command": "D:\\tools\\ida-mcp\\Scripts\\python.exe",
  "args": ["-m", "ida_pro_mcp.idalib_supervisor", "--stdio", "--max-workers", "1"],
  "mode": "headless",
  "env": { "IDADIR": "D:\\tools\\IDA Professional 9.2" }
}

rea doctor --provider ida --json reports healthy: true and the provider as configured and available.

Steps to reproduce

rea decompile "C:\path\to\small.exe" <function_name> --provider ida --format json

Observed

Most runs do not return for 5+ minutes. When allowed to run to completion, the result is:

{"error":"Analysis failed","code":"cleanup_incomplete","category":"execution_failure",
 "details":{"provider_id":"ida","operation":"close","cleanup":"incomplete",
   "resources":["D:\\Temp\\rea-ida-<id>"],
   "diagnostics":{"failure_reason":"IDA worker remains registered after close.",
     "reason":"IDA cleanup could not be verified; the remaining connection or private workspace is retained for retry."}}}

Measured, cleaning leftover supervisor processes and rea-ida-* workspaces between every run:

run	wall time	outcome
1	8 m 21 s	`cleanup_incomplete`
2	7 m 17 s	`cleanup_incomplete`
3	13 s	success (correct pseudocode)
4, 5	>5 min (capped)	did not return

The private workspace under TEMP is retained on each failure, as documented.

Contrast: the same upstream supervisor driven directly

Spawning the exact same command and speaking MCP to it directly — initialize → idb_open → server_health → decompile → idb_close → idb_list — completes in about 2.4 s end to end, and idb_list returns {"sessions": [], "count": 0} immediately after idb_close, i.e. the worker does deregister:

[   2.4s] idb_open:      isError=False  session_id ok
[   0.0s] server_health: isError=False  auto_analysis_ready: true, hexrays_ready: true
[   0.0s] decompile:     isError=False  pseudocode returned
[   0.0s] idb_close:     isError=False  owned: true, backend: worker
[   0.0s] idb_list:      {"sessions": [], "count": 0}

REA passes extra options on open (mode: "force_headless", run_auto_analysis, build_caches, init_hexrays, preferred_session_id), which should cost more than my minimal call — but the 13 s success shows the work itself is not inherently slow here, so the multi-minute runs look like waiting rather than analysing.

Ruled out

Stale workers / license contention. Reproduced after killing every leftover idalib_supervisor process and deleting all rea-ida-* workspaces.
A specific target. Same behaviour on different small PEs.
Environment propagation. I first suspected createIdaMcpConnection blanking DEFAULT_INHERITED_ENV_VARS before spreading the caller environment, and tried restoring SYSTEMROOT/PATH/SYSTEMDRIVE/PROGRAMFILES through the registration's env. It made no difference, and reading snapshotEnvironment shows keys are already canonicalised to upper case on Windows, so the real values do win. Mentioning it only so nobody re-walks that path.

What would help

Is there a way to get the provider's own view of where it is waiting — which upstream request is outstanding when it hangs? REA_LOG_LEVEL=debug only produced the final "CLI command completed" line here. Happy to run instrumented builds or collect the stdio traffic if you tell me what is useful.

Possibly related: #3032 (cleanup_incomplete on macOS with Ghidra), though that one is attributed to the JVM mid-exit and this is Windows + IDA.

Originally created by @alucardigo on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3056 ### Summary On Windows, analysis through the `ida` provider (headless / database-supervisor profile) usually hangs for several minutes and then fails with `cleanup_incomplete`, while driving the same upstream supervisor directly over MCP stdio completes the same work in ~2.4 s. It is intermittent: one run out of several succeeded in 13 s with an unchanged configuration. I could not establish a root cause, so this reports observed behaviour only. ### Environment - `rea-agents` 6.4.0 (npm), invoked as `node .../rea-agents/scripts/rea.mjs` - Windows 11 x64, Node.js 24.18.0 - IDA Pro 9.2 (licensed, `idalib` activated), `REA_ANALYSIS_PROVIDER=ida` - Upstream: `mrexodia/ida-pro-mcp` at `c133c3853faa111a9b00ee615c013b720d0c4acd` (the commit named in `docs/ida-provider.md`), installed in its own venv - Target: a 38 KB x86-64 PE (any small native PE reproduces it here) `REA_IDA_MCP_CONFIG` registration: ```json { "command": "D:\\tools\\ida-mcp\\Scripts\\python.exe", "args": ["-m", "ida_pro_mcp.idalib_supervisor", "--stdio", "--max-workers", "1"], "mode": "headless", "env": { "IDADIR": "D:\\tools\\IDA Professional 9.2" } } ``` `rea doctor --provider ida --json` reports `healthy: true` and the provider as configured and available. ### Steps to reproduce ```powershell rea decompile "C:\path\to\small.exe" <function_name> --provider ida --format json ``` ### Observed Most runs do not return for 5+ minutes. When allowed to run to completion, the result is: ```json {"error":"Analysis failed","code":"cleanup_incomplete","category":"execution_failure", "details":{"provider_id":"ida","operation":"close","cleanup":"incomplete", "resources":["D:\\Temp\\rea-ida-<id>"], "diagnostics":{"failure_reason":"IDA worker remains registered after close.", "reason":"IDA cleanup could not be verified; the remaining connection or private workspace is retained for retry."}}} ``` Measured, cleaning leftover supervisor processes and `rea-ida-*` workspaces between every run: | run | wall time | outcome | |---|---|---| | 1 | 8 m 21 s | `cleanup_incomplete` | | 2 | 7 m 17 s | `cleanup_incomplete` | | 3 | 13 s | **success** (correct pseudocode) | | 4, 5 | >5 min (capped) | did not return | The private workspace under `TEMP` is retained on each failure, as documented. ### Contrast: the same upstream supervisor driven directly Spawning the exact same command and speaking MCP to it directly — `initialize` → `idb_open` → `server_health` → `decompile` → `idb_close` → `idb_list` — completes in about 2.4 s end to end, and `idb_list` returns `{"sessions": [], "count": 0}` immediately after `idb_close`, i.e. the worker does deregister: ``` [ 2.4s] idb_open: isError=False session_id ok [ 0.0s] server_health: isError=False auto_analysis_ready: true, hexrays_ready: true [ 0.0s] decompile: isError=False pseudocode returned [ 0.0s] idb_close: isError=False owned: true, backend: worker [ 0.0s] idb_list: {"sessions": [], "count": 0} ``` REA passes extra options on open (`mode: "force_headless"`, `run_auto_analysis`, `build_caches`, `init_hexrays`, `preferred_session_id`), which should cost more than my minimal call — but the 13 s success shows the work itself is not inherently slow here, so the multi-minute runs look like waiting rather than analysing. ### Ruled out - **Stale workers / license contention.** Reproduced after killing every leftover `idalib_supervisor` process and deleting all `rea-ida-*` workspaces. - **A specific target.** Same behaviour on different small PEs. - **Environment propagation.** I first suspected `createIdaMcpConnection` blanking `DEFAULT_INHERITED_ENV_VARS` before spreading the caller environment, and tried restoring `SYSTEMROOT`/`PATH`/`SYSTEMDRIVE`/`PROGRAMFILES` through the registration's `env`. It made no difference, and reading `snapshotEnvironment` shows keys are already canonicalised to upper case on Windows, so the real values do win. Mentioning it only so nobody re-walks that path. ### What would help Is there a way to get the provider's own view of where it is waiting — which upstream request is outstanding when it hangs? `REA_LOG_LEVEL=debug` only produced the final "CLI command completed" line here. Happy to run instrumented builds or collect the stdio traffic if you tell me what is useful. Possibly related: #3032 (`cleanup_incomplete` on macOS with Ghidra), though that one is attributed to the JVM mid-exit and this is Windows + IDA.

bsakaguchi added the

bug

label

2026-10-15 18:30:49 +00:00

bsakaguchi commented

2026-10-15 18:30:50 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Fixed one independently reproduced readiness-budget defect in 9a44fea34: successive server_health polls previously received a fresh full SDK timeout. An 800 ms readiness allowance took about 1,358 ms with a delayed busy reply followed by a stalled request; passing only the remaining request budget reduced that controlled probe to about 805 ms.

The permanent regression uses the actual pinned MCP SDK over a mocked HTTP transport, a virtual readiness/pause clock and a real SDK timer. It verifies the second request gets 250 ms, preserves the overall 800 ms diagnostic, last busy observation and captured output, and avoids a fragile wall-time assertion. Existing read-only drain-before-close and caller-cancelled outcomes remain intact. Complete qualification: https://github.com/morluto/rea/actions/runs/38652688611.

Keeping this report open: this does not reproduce the Windows/commercial-IDA hang or explain the worker still present after idb_close. Those symptoms still lack a failing-run RPC/stage timeline and subsequent worker inventory. The earlier SDK timeout-identity fix now makes those failures report their actual operation and budget.

@morluto commented on GitHub (Oct 14, 2026): Fixed one independently reproduced readiness-budget defect in `9a44fea34`: successive `server_health` polls previously received a fresh full SDK timeout. An 800 ms readiness allowance took about 1,358 ms with a delayed busy reply followed by a stalled request; passing only the remaining request budget reduced that controlled probe to about 805 ms. The permanent regression uses the actual pinned MCP SDK over a mocked HTTP transport, a virtual readiness/pause clock and a real SDK timer. It verifies the second request gets 250 ms, preserves the overall 800 ms diagnostic, last busy observation and captured output, and avoids a fragile wall-time assertion. Existing read-only drain-before-close and caller-cancelled outcomes remain intact. Complete qualification: https://github.com/morluto/rea/actions/runs/38652688611. Keeping this report open: this does not reproduce the Windows/commercial-IDA hang or explain the worker still present after `idb_close`. Those symptoms still lack a failing-run RPC/stage timeline and subsequent worker inventory. The earlier SDK timeout-identity fix now makes those failures report their actual operation and budget.

bsakaguchi commented

2026-10-16 21:02:41 +00:00

Author

Owner

@alucardigo commented on GitHub (Oct 15, 2026):

6.5.0 fixes the hang — same host, target and registration, runs went from 5–9 minutes to 13–27 seconds. Thanks.

cleanup_incomplete remains, and I have the root cause with the payload.

Root cause

Upstream idb_list can return the worker mid-teardown, as a row that identifies no live owned session but still carries the input path:

{"session_id": "", "input_path": "<private workspace>\\cat.exe", "filename": "cat.exe",
 "created_at": "2026-10-15T19:47:11.091884+00:00", "last_accessed": "",
 "is_analyzing": false, "is_active": false, "backend": "worker",
 "owned": false, "adopted": false, "pid": 45344, "worker_pid": 45344,
 "metadata": {"backend": "worker", "host": "127.0.0.1", "port": 54350}}

The post-close check in IdaSessionClient.#closeOnce matches on either arm:

const remaining = inventory.sessions.filter((entry) =>
  entry.session_id === this.#database ||
  sameIdaHostPath(entry.input_path, this.#workspace?.inputPath ?? ""));
if (this.#recordBlockingSessions(remaining))
  throw new AnalysisProtocolError("IDA worker remains registered after close.");

The row's session_id is "", so the first arm cannot match — but the path arm does. remaining.length === 1, and the run fails even though idb_close already acknowledged {"success": true, "owned": true, "backend": "worker"}, the pseudocode was produced, and the row describes a session that is neither owned nor active.

That is also why the diagnostic shows session_id: "" with owned: false — those fields are copied straight from this row by #recordBlockingSessions, so it is a real upstream row, not a synthesised placeholder.

Confirmation

Changing only the filter predicate — ignore a row that has no session_id and is neither owned nor active:

const remaining = inventory.sessions.filter((entry) =>
  (entry.session_id === this.#database ||
   sameIdaHostPath(entry.input_path, this.#workspace?.inputPath ?? "")) &&
  !(!entry.session_id && entry.owned !== true && entry.is_active !== true));

build	runs	result
6.5.0 stock	12	11 × `cleanup_incomplete`, 1 pass
6.5.0 + predicate above	6	6 × pass, 0 workspaces retained

Nothing else changed between the two.

Timing

The post-close idb_list is the only slow call in the session — everything after open is 9–60 ms, that one is ~1 s:

 10.741s  idb_close      answered in 0.009s
 11.765s  idb_list       answered in 1.021s   <- decides the verdict
outstanding requests at exit: []   server exit code: 0

Measured per run, the duration tracks the outcome:

post-close `idb_list`	payload	outcome
1016 ms	`{"sessions": [], "count": 0}`	pass
1021 ms	`{"sessions": [], "count": 0}`	pass
522 ms	the tear-down row above	fail

When that call returns early it catches the worker before deregistration completes.

Two notes on method

A transparent stdio proxy hides the bug: 8/8 pass with one in the path, 11/12 fail without. The payloads above were therefore captured by instrumenting in-process rather than on the wire.
The CLI loads dist/cli-runtime/binary-*.js, not dist/ida/IdaSessionClient.js — patching the latter has no effect on rea <command>. Worth knowing for anyone else instrumenting this.

Driving the supervisor directly with REA's own open arguments (force_headless, run_auto_analysis, build_caches, init_hexrays, preferred_session_id) never reproduced it: idb_close then idb_list returned {"sessions": [], "count": 0} immediately and again 3 s later. The race needs REA's exact call cadence.

Possible shape of a fix

A row with a blanked session_id that is neither owned nor active does not establish a live session, so treating it as blocking looks wrong on its own terms. It also seems worth distinguishing the two arms: the session_id arm proves identity, while the path arm is a heuristic that cannot tell "my worker, still exiting" from "a different session on the same input". A short re-poll before concluding would cover the same ground, since the row disappears within ~1 s.

Upstream arguably should not report a session whose session_id is empty, but REA can defend against it either way.

Glad to turn the predicate into a PR, or to run any other experiment — the failing path reproduces in about 20 s here.

Environment

rea-agents 6.5.0 (npm) · Node.js 24.18.0 · Windows 11 x64 · IDA Pro 9.2 (licensed, idalib) · upstream mrexodia/ida-pro-mcp at c133c3853faa111a9b00ee615c013b720d0c4acd · registration mode: "headless", --max-workers 1, timeoutMs: 540000 · target: 38 KB x86-64 PE

@alucardigo commented on GitHub (Oct 15, 2026): **6.5.0 fixes the hang** — same host, target and registration, runs went from 5–9 minutes to 13–27 seconds. Thanks. `cleanup_incomplete` remains, and I have the root cause with the payload. ### Root cause Upstream `idb_list` can return the worker **mid-teardown**, as a row that identifies no live owned session but still carries the input path: ```json {"session_id": "", "input_path": "<private workspace>\\cat.exe", "filename": "cat.exe", "created_at": "2026-10-15T19:47:11.091884+00:00", "last_accessed": "", "is_analyzing": false, "is_active": false, "backend": "worker", "owned": false, "adopted": false, "pid": 45344, "worker_pid": 45344, "metadata": {"backend": "worker", "host": "127.0.0.1", "port": 54350}} ``` The post-close check in `IdaSessionClient.#closeOnce` matches on either arm: ```js const remaining = inventory.sessions.filter((entry) => entry.session_id === this.#database || sameIdaHostPath(entry.input_path, this.#workspace?.inputPath ?? "")); if (this.#recordBlockingSessions(remaining)) throw new AnalysisProtocolError("IDA worker remains registered after close."); ``` The row's `session_id` is `""`, so the first arm cannot match — but the **path arm does**. `remaining.length === 1`, and the run fails even though `idb_close` already acknowledged `{"success": true, "owned": true, "backend": "worker"}`, the pseudocode was produced, and the row describes a session that is neither owned nor active. That is also why the diagnostic shows `session_id: ""` with `owned: false` — those fields are copied straight from this row by `#recordBlockingSessions`, so it is a real upstream row, not a synthesised placeholder. ### Confirmation Changing only the filter predicate — ignore a row that has no `session_id` and is neither owned nor active: ```js const remaining = inventory.sessions.filter((entry) => (entry.session_id === this.#database || sameIdaHostPath(entry.input_path, this.#workspace?.inputPath ?? "")) && !(!entry.session_id && entry.owned !== true && entry.is_active !== true)); ``` | build | runs | result | |---|---|---| | 6.5.0 stock | 12 | **11 × `cleanup_incomplete`**, 1 pass | | 6.5.0 + predicate above | 6 | **6 × pass**, 0 workspaces retained | Nothing else changed between the two. ### Timing The post-close `idb_list` is the only slow call in the session — everything after open is 9–60 ms, that one is ~1 s: ``` 10.741s idb_close answered in 0.009s 11.765s idb_list answered in 1.021s <- decides the verdict outstanding requests at exit: [] server exit code: 0 ``` Measured per run, the duration tracks the outcome: | post-close `idb_list` | payload | outcome | |---|---|---| | 1016 ms | `{"sessions": [], "count": 0}` | pass | | 1021 ms | `{"sessions": [], "count": 0}` | pass | | **522 ms** | the tear-down row above | **fail** | When that call returns early it catches the worker before deregistration completes. ### Two notes on method 1. **A transparent stdio proxy hides the bug**: 8/8 pass with one in the path, 11/12 fail without. The payloads above were therefore captured by instrumenting in-process rather than on the wire. 2. The CLI loads `dist/cli-runtime/binary-*.js`, not `dist/ida/IdaSessionClient.js` — patching the latter has no effect on `rea <command>`. Worth knowing for anyone else instrumenting this. Driving the supervisor directly with REA's own open arguments (`force_headless`, `run_auto_analysis`, `build_caches`, `init_hexrays`, `preferred_session_id`) never reproduced it: `idb_close` then `idb_list` returned `{"sessions": [], "count": 0}` immediately and again 3 s later. The race needs REA's exact call cadence. ### Possible shape of a fix A row with a blanked `session_id` that is neither owned nor active does not establish a live session, so treating it as blocking looks wrong on its own terms. It also seems worth distinguishing the two arms: the `session_id` arm proves identity, while the path arm is a heuristic that cannot tell "my worker, still exiting" from "a different session on the same input". A short re-poll before concluding would cover the same ground, since the row disappears within ~1 s. Upstream arguably should not report a session whose `session_id` is empty, but REA can defend against it either way. Glad to turn the predicate into a PR, or to run any other experiment — the failing path reproduces in about 20 s here. ### Environment `rea-agents` 6.5.0 (npm) · Node.js 24.18.0 · Windows 11 x64 · IDA Pro 9.2 (licensed, `idalib`) · upstream `mrexodia/ida-pro-mcp` at `c133c3853faa111a9b00ee615c013b720d0c4acd` · registration `mode: "headless"`, `--max-workers 1`, `timeoutMs: 540000` · target: 38 KB x86-64 PE

bsakaguchi referenced this issue

2026-10-17 21:08:18 +00:00

[PR #1071] [MERGED] fix(apple): report apps without asset catalogs as empty inventories #2620

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1071

No description provided.

Rows
Columns