morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3509] [Bug] Open target lost after MCP transport failure; Ghidra analyses longer than the client's idle timeout can never complete (stdio) #1306

New issue

Open

opened 2026-10-16 21:04:02 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-16 21:04:02 +00:00

Owner

Originally created by @Michael-Logies on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3509

Environment: rea-agents 6.4.0 (npm), MCP transport: stdio; client: Qwen Code 0.25.1 (300000 ms idle timeout); provider: Ghidra 12.1.4 headless (JDK 21), Linux x86_64, Node v24.21.0. Target: a publicly available 10 MB PE32 GUI Windows binary (SHA256 644b7bec89a2e4d57d119d15aa36af1df12a4c3547b692bc0462af35a93ddc96) — benign, standard PE with no protection mechanisms.

Repro sequence:

open_binary → OK.
binary_overview → aborted by the client after 300000 ms idle timeout. Full Ghidra auto-analysis of a 10 MB PE exceeds 5 min, and REA emits no progress notifications, so an idle-timeout-based client cannot distinguish "working" from "stalled".
binary_session → open:true. The session survives step 2 — consistent with the documented limitation: "Cancelling an active Ghidra request terminates its ephemeral database … the next provider query imports the original artifact again."
Two calls issued concurrently in one client turn — a client-side tool-registry lookup (does not reach the REA server) and search_strings. search_strings fails with: "MCP tool execution may have completed before the connection failed. Automatic replay was skipped because the call could not be verified as safe to replay. Do not retry automatically; verify the outcome before trying again."
binary_session → open:false. The server still answers, but the open target is silently gone.

Expected: long provider operations emit MCP progress notifications so idle-timeout clients don't cancel them; a transport hiccup either preserves the open target or reports its loss clearly.

Actual: with one aborted analysis plus one concurrent unrelated client operation, the target is silently lost, and the "may have completed" message is misleading. Because cancellation discards the ephemeral analysis database, any analysis exceeding the client's idle timeout can never complete — every retry re-imports from scratch.

Related: #2195, #2975 (same class: permanent session-state loss after transport failure).

Caveat: during step 4 the machine showed OOM-killer activity (an unrelated process was killed); the root cause of the transport failure itself could not be attributed with certainty — resource pressure cannot be fully excluded.

Suggested directions: (a) progress notifications during long provider analyses; (b) persist/re-attach the open target across server restarts; (c) robustness against concurrent client housekeeping traffic.

Authored by Qwen Code (GLM-5.3-Flash via OpenRouter), submitted by Michael Logies.

Originally created by @Michael-Logies on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3509 **Environment:** rea-agents 6.4.0 (npm), MCP transport: stdio; client: Qwen Code 0.25.1 (300000 ms idle timeout); provider: Ghidra 12.1.4 headless (JDK 21), Linux x86_64, Node v24.21.0. Target: a publicly available 10 MB PE32 GUI Windows binary (SHA256 `644b7bec89a2e4d57d119d15aa36af1df12a4c3547b692bc0462af35a93ddc96`) — benign, standard PE with no protection mechanisms. **Repro sequence:** 1. `open_binary` → OK. 2. `binary_overview` → aborted by the client after 300000 ms idle timeout. Full Ghidra auto-analysis of a 10 MB PE exceeds 5 min, and REA emits no progress notifications, so an idle-timeout-based client cannot distinguish "working" from "stalled". 3. `binary_session` → `open:true`. The session survives step 2 — consistent with the documented limitation: *"Cancelling an active Ghidra request terminates its ephemeral database … the next provider query imports the original artifact again."* 4. Two calls issued concurrently in one client turn — a client-side tool-registry lookup (does not reach the REA server) and `search_strings`. `search_strings` fails with: *"MCP tool execution may have completed before the connection failed. Automatic replay was skipped because the call could not be verified as safe to replay. Do not retry automatically; verify the outcome before trying again."* 5. `binary_session` → `open:false`. The server still answers, but the open target is silently gone. **Expected:** long provider operations emit MCP progress notifications so idle-timeout clients don't cancel them; a transport hiccup either preserves the open target or reports its loss clearly. **Actual:** with one aborted analysis plus one concurrent unrelated client operation, the target is silently lost, and the "may have completed" message is misleading. Because cancellation discards the ephemeral analysis database, any analysis exceeding the client's idle timeout can never complete — every retry re-imports from scratch. **Related:** #2195, #2975 (same class: permanent session-state loss after transport failure). **Caveat:** during step 4 the machine showed OOM-killer activity (an unrelated process was killed); the root cause of the transport failure itself could not be attributed with certainty — resource pressure cannot be fully excluded. **Suggested directions:** (a) progress notifications during long provider analyses; (b) persist/re-attach the open target across server restarts; (c) robustness against concurrent client housekeeping traffic. --- *Authored by Qwen Code (GLM-5.3-Flash via OpenRouter), submitted by Michael Logies.*

bsakaguchi commented

2026-10-16 21:04:03 +00:00

Author

Owner

@Michael-Logies commented on GitHub (Oct 15, 2026):

Retest under rea-agents 6.5.0 — the reported failure sequence no longer reproduces; timeouts now surface as structured errors and the target survives

Environment: rea-agents 6.5.0 (stdio), same client with its 300 s idle timeout, Ghidra 12.1.4 headless, same target binary as originally reported: Picasa3.exe, Google's discontinued Picasa 3 photo organizer (10 MB PE32 GUI, still freely downloadable; identical SHA256 644b7bec89a2e4d57d119d15aa36af1df12a4c3547b692bc0462af35a93ddc96). The host VM was rebooted beforehand with substantially more free memory (16 GiB free; zero OOM events in dmesg for this boot), which excludes the memory-pressure caveat noted in the original report.

Same sequence as originally reported:

open_binary — OK.
binary_overview — no client-side abort this time. After ~330 s the server returned a structured error: provider_timeout (retryable), timeout_ms: 330000, failure_message: "Ghidra startup deadline elapsed", with full diagnostics (run_id, Ghidra log paths). The call ran ~330 s server-side and returned cleanly instead of the client aborting at its 300 s idle timeout. How the client survived past its own timeout was not verified (traffic on the wire, e.g. progress notifications, is a hypothesis).
binary_session — open: true (target survives).
search_strings — again a structured provider_timeout after ~330 s, with a fresh Ghidra import (new runtime root/PID, same run_id), consistent with the documented ephemeral-database behavior. No "connection failed" wording, no transport loss.
binary_session — open: true, response identical to step 3. The silent target loss no longer reproduces.

Still present: the first Ghidra analysis of this target takes longer than the 330 s provider deadline, so a full first analysis still cannot complete — and every follow-up query restarts the import from scratch (ephemeral DB). The returned remediation ("try a smaller request") is not actionable for this class of target. Suggested direction (c) from the original report looks improved for this sequence (the transport survived both deadline aborts); (a) could not be verified directly; (b) was not exercised.

Authored by Qwen Code (GLM-5.3-Flash via Z.ai), submitted by Michael Logies.

@Michael-Logies commented on GitHub (Oct 15, 2026): **Retest under rea-agents 6.5.0 — the reported failure sequence no longer reproduces; timeouts now surface as structured errors and the target survives** Environment: rea-agents 6.5.0 (stdio), same client with its 300 s idle timeout, Ghidra 12.1.4 headless, same target binary as originally reported: **`Picasa3.exe`**, Google's discontinued Picasa 3 photo organizer (10 MB PE32 GUI, still freely downloadable; identical SHA256 `644b7bec89a2e4d57d119d15aa36af1df12a4c3547b692bc0462af35a93ddc96`). The host VM was rebooted beforehand with substantially more free memory (16 GiB free; zero OOM events in `dmesg` for this boot), which excludes the memory-pressure caveat noted in the original report. Same sequence as originally reported: 1. `open_binary` — OK. 2. `binary_overview` — no client-side abort this time. After ~330 s the server returned a structured error: `provider_timeout` (retryable), `timeout_ms: 330000`, `failure_message: "Ghidra startup deadline elapsed"`, with full diagnostics (run_id, Ghidra log paths). The call ran ~330 s server-side and returned cleanly instead of the client aborting at its 300 s idle timeout. How the client survived past its own timeout was not verified (traffic on the wire, e.g. progress notifications, is a hypothesis). 3. `binary_session` — `open: true` (target survives). 4. `search_strings` — again a structured `provider_timeout` after ~330 s, with a fresh Ghidra import (new runtime root/PID, same run_id), consistent with the documented ephemeral-database behavior. **No "connection failed" wording, no transport loss.** 5. `binary_session` — `open: true`, response identical to step 3. **The silent target loss no longer reproduces.** Still present: the first Ghidra analysis of this target takes longer than the 330 s provider deadline, so a full first analysis still cannot complete — and every follow-up query restarts the import from scratch (ephemeral DB). The returned remediation ("try a smaller request") is not actionable for this class of target. Suggested direction (c) from the original report looks improved for this sequence (the transport survived both deadline aborts); (a) could not be verified directly; (b) was not exercised. Authored by Qwen Code (GLM-5.3-Flash via Z.ai), submitted by Michael Logies.

bsakaguchi referenced this issue

2026-10-17 21:10:14 +00:00

[PR #1306] [CLOSED] fix(apple): recognize legacy simulator Mach-O platforms #2785

bsakaguchi commented

2026-10-31 00:30:31 +00:00

Author

Owner

@goncalompontes commented on GitHub (Oct 19, 2026):

Root-cause notes from reading the Ghidra provider path, in case they help whoever picks this up. This is source reading, not a reproduction.

A Ghidra request runs through a serial per-Program queue. If the request's signal aborts while it is active, the queue calls #stopAfterActiveCancellation, which runs #requestClose(true), a forced stop.
A forced stop reaches #cleanup(forceStop), which calls #publishManagedProject(uncleanSession=true). That returns without publishing. So an analysis cancelled by an idle timeout is not written to the project cache, and the runtime root is removed. This is why the next open starts from import again.
The startup deadline (REA_GHIDRA_STARTUP_TIMEOUT_MS, default 330000 ms) covers import, analysis, bridge start, and readiness together, so a 10 MB PE that exceeds it can never complete through one blocking call.

Proposed direction, tracked in a design draft: run analysis as a background job whose lifetime is not tied to the request, and keep each wait short. A cancelled wait would then leave the job running. Until that exists, raising the startup timeout above the client's idle timeout does not help, because the client cancels first.

@goncalompontes commented on GitHub (Oct 19, 2026): Root-cause notes from reading the Ghidra provider path, in case they help whoever picks this up. This is source reading, not a reproduction. - A Ghidra request runs through a serial per-Program queue. If the request's signal aborts while it is active, the queue calls `#stopAfterActiveCancellation`, which runs `#requestClose(true)`, a forced stop. - A forced stop reaches `#cleanup(forceStop)`, which calls `#publishManagedProject(uncleanSession=true)`. That returns without publishing. So an analysis cancelled by an idle timeout is not written to the project cache, and the runtime root is removed. This is why the next open starts from import again. - The startup deadline (`REA_GHIDRA_STARTUP_TIMEOUT_MS`, default 330000 ms) covers import, analysis, bridge start, and readiness together, so a 10 MB PE that exceeds it can never complete through one blocking call. Proposed direction, tracked in a design draft: run analysis as a background job whose lifetime is not tied to the request, and keep each wait short. A cancelled wait would then leave the job running. Until that exists, raising the startup timeout above the client's idle timeout does not help, because the client cancels first.

bsakaguchi commented

2026-10-31 00:30:31 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 20, 2026):

Current-main investigation at 5b676ba9f302228b44c28ab076525abed3e647c1 (Linux x64, Node 24.18.0, real Ghidra 12.1.4 / JDK 21.0.12.1+1; stdio MCP protocol 2026-07-28):

Read-only target: installed GNU /usr/bin/ls, 151,344 bytes, SHA-256 cb30d69b24245bf2ecdc9e7f53bbad19159999970b6d82c0c00c7d32d9e37aa4; bytes remain unchanged. No target execution.
Controlled SDK request with 12,000-ms timeout and no progress callback raises SdkError / REQUEST_TIMEOUT after 12.007 s. The next binary_session reports open:true with the same digest.
On that same server, requesting progress with resetTimeoutOnProgress:true completes binary_overview in 58.547 s under the same 12-second idle policy (separate 180-second total cap). There are seven monotonic progress observations: start, five ghidra_startup messages at roughly 10-second intervals, completion. It returns 652 procedures. These are SDK probe options, not asserted Qwen settings.
A warm overview returns the same result in 32.9 ms, target remains open, full Evidence export and explicit close succeed, and owned temporary runtime roots are empty. Existing reuse is uninstrumented; no new cache or speedup is claimed.

The complete task (discovery, open, timeout/control, state checks, recovered analysis, warm query, export, close) costs 101.367 s elapsed, 97.687 s family CPU, 1,671.1 MiB peak, under one CPU / 8-GiB hard / 7-GiB early bounds, Java heap 1,536 MiB, no swap. Payload-exit counters confirm cleanup and no OOM; these are not final whole-scope counters. OS caches/shared-host load are uncontrolled.

Estimated cl100k_base: compact input-catalog proxy 44,848, selected overview schema 31, overview content 2,477, complete captured protocol 790,430; local full Evidence 9,875 tokens. These are protocol/local-data estimates, not model usage; initialization outside hooks and downstream prompts are unmeasured.

Server-side progress already exists in the current Ghidra startup owner and only reaches MCP callers that negotiate a progress token. Client deadline/reset policy remains client-owned. The separate server-startup limit/remediation was addressed by #5356, whose retained different-target gsudo qualification completed in 590.755 s after increasing the configured limit; that is prior evidence, not a new Picasa test.

NOT_ENOUGH_INFO remains for the original Qwen 0.25.1/Picasa transport-hiccup sequence and any remaining current-client regression. This controlled SDK run does not reproduce that client or transport loss. The reporter's 6.5.0 retest also records preserved target state. No additional server defect is established here, so no speculative background-job/lifecycle change is integrated. Leaving this issue open for the unresolved client-specific evidence. Raw captures/receipts are retained locally under /tmp/rea-3509-proof.

@gensokyo-dev commented on GitHub (Oct 20, 2026): Current-main investigation at `5b676ba9f302228b44c28ab076525abed3e647c1` (Linux x64, Node 24.18.0, real Ghidra 12.1.4 / JDK 21.0.12.1+1; stdio MCP protocol 2026-07-28): - Read-only target: installed GNU `/usr/bin/ls`, 151,344 bytes, SHA-256 `cb30d69b24245bf2ecdc9e7f53bbad19159999970b6d82c0c00c7d32d9e37aa4`; bytes remain unchanged. No target execution. - Controlled SDK request with **12,000-ms timeout and no progress callback** raises `SdkError / REQUEST_TIMEOUT` after **12.007 s**. The next `binary_session` reports **open:true with the same digest**. - On that same server, requesting progress with `resetTimeoutOnProgress:true` completes `binary_overview` in **58.547 s** under the same 12-second idle policy (separate 180-second total cap). There are seven monotonic progress observations: start, five `ghidra_startup` messages at roughly 10-second intervals, completion. It returns **652 procedures**. These are SDK probe options, not asserted Qwen settings. - A warm overview returns the same result in **32.9 ms**, target remains open, full Evidence export and explicit close succeed, and owned temporary runtime roots are empty. Existing reuse is uninstrumented; no new cache or speedup is claimed. The complete task (discovery, open, timeout/control, state checks, recovered analysis, warm query, export, close) costs **101.367 s elapsed**, **97.687 s family CPU**, **1,671.1 MiB peak**, under one CPU / 8-GiB hard / 7-GiB early bounds, Java heap 1,536 MiB, no swap. Payload-exit counters confirm cleanup and no OOM; these are not final whole-scope counters. OS caches/shared-host load are uncontrolled. Estimated `cl100k_base`: compact input-catalog proxy **44,848**, selected overview schema **31**, overview content **2,477**, complete captured protocol **790,430**; local full Evidence **9,875** tokens. These are protocol/local-data estimates, not model usage; initialization outside hooks and downstream prompts are unmeasured. Server-side progress already exists in the current Ghidra startup owner and only reaches MCP callers that negotiate a progress token. Client deadline/reset policy remains client-owned. The separate server-startup limit/remediation was addressed by #5356, whose retained different-target gsudo qualification completed in 590.755 s after increasing the configured limit; that is prior evidence, not a new Picasa test. **NOT_ENOUGH_INFO remains for the original Qwen 0.25.1/Picasa transport-hiccup sequence and any remaining current-client regression.** This controlled SDK run does not reproduce that client or transport loss. The reporter's 6.5.0 retest also records preserved target state. No additional server defect is established here, so no speculative background-job/lifecycle change is integrated. Leaving this issue open for the unresolved client-specific evidence. Raw captures/receipts are retained locally under `/tmp/rea-3509-proof`.

bsakaguchi commented

2026-10-31 00:30:31 +00:00

Author

Owner

@Michael-Logies commented on GitHub (Oct 20, 2026):

Retest under 6.7.0 — current-client chain, two targets

Environment: rea-agents 6.7.0 (npm; server_identity verified 6.7.0, SDK 2.3.1), real Qwen Code client over stdio, Ghidra 12.1.4 / JDK 21, Linux VM, 23 GB RAM (12 GB free), no OOM entries in the kernel log.

Target 1 — ~1 MB PE32 DLL from a commercial application

open_binary OK. A single binary_overview call completed the entire first analysis — no client abort, no provider_timeout (2,384 procedures / 3,196 strings / 8 segments).
An immediately repeated binary_overview returned identical results with the limitation "Loaded from a local REA analysis snapshot; this call did not re-run the provider." — the first time we can demonstrate warm reuse in this setup (new vs. 6.5.0, where every follow-up call re-imported after a deadline failure).

Target 2 — Picasa3.exe (10 MB, same as our previous runs)

First attempt: the rea.mjs server process itself died ~50 s into the binary_overview call. The client saw "MCP tool execution may have completed before the connection failed". No kernel OOM, no Hotspot crash dump (hs_err_pid*), no leftover server logs; the client transparently re-spawned the server (new started_at visible in binary_session), which then trivially reported open:false. Not reproduced on retry — reporting it for completeness.
Retry: structured provider_timeout after the full 330 s (retryable:true, complete diagnostics — run_id, runtime_root, JVM PID, remediation pointing at REA_GHIDRA_STARTUP_TIMEOUT_MS). The real client survived the 330-s call. binary_session afterwards: open:true — the target stays bound.
A follow-up search_strings call produced another structured provider_timeout after 330 s with a fresh runtime_root and JVM PID but the same run_id — re-import per follow-up call after a failed first analysis is unchanged.

Assessment vs. the 19.10. current-client incident

The regression we hit live on 19.10. under 6.5.0 (~1 MB DLL, client idle-timeout → open:false, "connection failed" / "No app is open") did not reproduce under 6.7.0 in this chain — both long calls survived, and timeouts are structured and non-destructive. Remaining observations:

the one-off, unexplained rea.mjs process death above (client-visible as a tool/transport failure, not as a structured error);
the 330-s default deadline vs. a >330-s first analysis of the 10-MB PE — remediation is documented (env vars), but warm reuse only kicks in after a successful first analysis, so without raising the deadline the 10-MB target still never completes under a default config.

Authored by Qwen Code (glm-5.3-flash via Z.ai), submitted by Michael Logies.

@Michael-Logies commented on GitHub (Oct 20, 2026): **Retest under 6.7.0 — current-client chain, two targets** Environment: rea-agents **6.7.0** (npm; `server_identity` verified 6.7.0, SDK 2.3.1), **real Qwen Code client over stdio**, Ghidra 12.1.4 / JDK 21, Linux VM, 23 GB RAM (12 GB free), no OOM entries in the kernel log. **Target 1 — ~1 MB PE32 DLL from a commercial application** - `open_binary` OK. A single `binary_overview` call **completed the entire first analysis** — no client abort, no `provider_timeout` (2,384 procedures / 3,196 strings / 8 segments). - An immediately repeated `binary_overview` returned identical results with the limitation *"Loaded from a local REA analysis snapshot; this call did not re-run the provider."* — the first time we can **demonstrate warm reuse** in this setup (new vs. 6.5.0, where every follow-up call re-imported after a deadline failure). **Target 2 — Picasa3.exe (10 MB, same as our previous runs)** - First attempt: **the rea.mjs server process itself died ~50 s into the `binary_overview` call**. The client saw *"MCP tool execution may have completed before the connection failed"*. No kernel OOM, no Hotspot crash dump (`hs_err_pid*`), no leftover server logs; the client transparently re-spawned the server (new `started_at` visible in `binary_session`), which then trivially reported `open:false`. **Not reproduced on retry** — reporting it for completeness. - Retry: structured **`provider_timeout` after the full 330 s** (`retryable:true`, complete diagnostics — run_id, runtime_root, JVM PID, remediation pointing at `REA_GHIDRA_STARTUP_TIMEOUT_MS`). **The real client survived the 330-s call.** `binary_session` afterwards: **`open:true`** — the target stays bound. - A follow-up `search_strings` call produced another structured `provider_timeout` after 330 s with a **fresh runtime_root and JVM PID but the same run_id** — re-import per follow-up call after a *failed* first analysis is unchanged. **Assessment vs. the 19.10. current-client incident** The regression we hit live on 19.10. under 6.5.0 (~1 MB DLL, client idle-timeout → `open:false`, "connection failed" / "No app is open") **did not reproduce under 6.7.0** in this chain — both long calls survived, and timeouts are structured and non-destructive. Remaining observations: 1. the one-off, unexplained rea.mjs process death above (client-visible as a tool/transport failure, not as a structured error); 2. the 330-s default deadline vs. a >330-s first analysis of the 10-MB PE — remediation is documented (env vars), but warm reuse only kicks in *after* a successful first analysis, so without raising the deadline the 10-MB target still never completes under a default config. *Authored by Qwen Code (glm-5.3-flash via Z.ai), submitted by Michael Logies.*

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1306

No description provided.

Rows
Columns