morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5400] [Feature] Avoid repeated Ghidra startup logs in input-error replies #3270

New issue

Open

opened 2026-10-31 00:38:36 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-31 00:38:36 +00:00

Owner

Originally created by @cpruijsen on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5400

Area

MCP or agent integration.

Current limitation

Two consecutive invalid-input replies each include 32,429 bytes of identical Ghidra startup stdout. Each complete text reply is 35,460 UTF-8 bytes. The provider was already ready, and both failed operations produced no stdout or stderr.

Reproduction and observed evidence

Historical stdio MCP run on 19 October 2026:

Open Elgato Control Center with Ghidra and wait for a successful provider search.
In that session, make these two calls with the incorrect display-name document selector:

{"name":"procedure_pseudo_code","arguments":{"document":"Elgato Control Center","procedure":"ACCPMiredToKelvinValueTransformer::transformedValue:"}}
{"name":"procedure_pseudo_code","arguments":{"document":"Elgato Control Center","procedure":"ACCPMiredToKelvinValueTransformer::reverseTransformedValue:"}}

Both replies correctly return invalid_request:

Unknown Ghidra Program identity: Elgato Control Center; active program: target-865f822237eb.bin

Baseline 8-bit trace request/result pairs are 74/75 and 76/77. In both replies, error.details.diagnostics.failure_output has empty stdout/stderr and complete:true. Their cumulative diagnostics.stdout.text values are byte-identical. The repeated stdout has SHA-256 1568b8e9182ed02c5f231231774caaa72aee1c2e35f4a4e74195166b3d06feaf. Full startup logs remain private.

Desired outcome and success criteria

Proposed: deliver the typed argument error and operation-specific diagnostics without repeating the complete startup transcript in every ordinary input-error reply. Keep complete session diagnostics accessible. Preserve startup, crash, cleanup, and memory-failure evidence when relevant. This requires a compatible delivery choice, not silent truncation or removal of canonical diagnostic data.

Scope and attribution

The model supplied the wrong document identity. REA correctly rejected it. The separate harness clipped each text reply at 12,000 bytes. That client policy is not REA truncation. Generation-limit failures in the benchmark are also separate.

Verification available so far

Actual MCP results at source revision 3613fb11bd9d6b876822056a6d7ec2c18e3bfc9d, macOS arm64, Ghidra 12.1.4, Temurin 21.0.12.1+1-LTS. Node and exact macOS versions: NOT_ENOUGH_INFO, not recorded in the inspected traces.

Source review at d532b18123916a9e88f98c7880ba4f984e23d715 still shows cumulative stdout/stderr projection. No fresh runtime reproduction at that revision occurred.

Alternatives and references

Correcting or omitting document avoids the input error, but does not address diagnostic delivery. Related #723 preserves complete contracts while discussing response repetition. No before/after token or latency improvement is claimed.

Originally created by @cpruijsen on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5400 ### Area MCP or agent integration. ### Current limitation Two consecutive invalid-input replies each include 32,429 bytes of identical Ghidra startup stdout. Each complete text reply is 35,460 UTF-8 bytes. The provider was already ready, and both failed operations produced no stdout or stderr. ### Reproduction and observed evidence Historical stdio MCP run on 19 October 2026: 1. Open Elgato Control Center with Ghidra and wait for a successful provider search. 2. In that session, make these two calls with the incorrect display-name document selector: ```json {"name":"procedure_pseudo_code","arguments":{"document":"Elgato Control Center","procedure":"ACCPMiredToKelvinValueTransformer::transformedValue:"}} {"name":"procedure_pseudo_code","arguments":{"document":"Elgato Control Center","procedure":"ACCPMiredToKelvinValueTransformer::reverseTransformedValue:"}} ``` Both replies correctly return `invalid_request`: ```text Unknown Ghidra Program identity: Elgato Control Center; active program: target-865f822237eb.bin ``` Baseline 8-bit trace request/result pairs are 74/75 and 76/77. In both replies, `error.details.diagnostics.failure_output` has empty stdout/stderr and `complete:true`. Their cumulative `diagnostics.stdout.text` values are byte-identical. The repeated stdout has SHA-256 `1568b8e9182ed02c5f231231774caaa72aee1c2e35f4a4e74195166b3d06feaf`. Full startup logs remain private. ### Desired outcome and success criteria Proposed: deliver the typed argument error and operation-specific diagnostics without repeating the complete startup transcript in every ordinary input-error reply. Keep complete session diagnostics accessible. Preserve startup, crash, cleanup, and memory-failure evidence when relevant. This requires a compatible delivery choice, not silent truncation or removal of canonical diagnostic data. ### Scope and attribution The model supplied the wrong document identity. REA correctly rejected it. The separate harness clipped each text reply at 12,000 bytes. That client policy is not REA truncation. Generation-limit failures in the benchmark are also separate. ### Verification available so far Actual MCP results at source revision `3613fb11bd9d6b876822056a6d7ec2c18e3bfc9d`, macOS arm64, Ghidra 12.1.4, Temurin 21.0.12.1+1-LTS. Node and exact macOS versions: `NOT_ENOUGH_INFO`, not recorded in the inspected traces. Source review at `d532b18123916a9e88f98c7880ba4f984e23d715` still shows [cumulative stdout/stderr projection](https://github.com/morluto/rea/blob/d532b18123916a9e88f98c7880ba4f984e23d715/src/ghidra/GhidraDiagnostics.ts#L85-L101). No fresh runtime reproduction at that revision occurred. ### Alternatives and references Correcting or omitting `document` avoids the input error, but does not address diagnostic delivery. Related [#723](https://github.com/morluto/rea/issues/723) preserves complete contracts while discussing response repetition. No before/after token or latency improvement is claimed.

bsakaguchi commented

2026-10-31 00:38:37 +00:00

Author

Owner

@dark13b commented on GitHub (Oct 20, 2026):

I'd like to take this one as well.

I have Ghidra 12.1.4 on Windows 11, so I can check this against the real provider instead of from source only. I'll reproduce the repeated startup stdout in consecutive input-error replies first, then work on keeping the full session diagnostics accessible without repeating them in every ordinary input-error reply. Startup, crash, cleanup, and memory-failure evidence will stay where it is relevant. A draft PR will follow.

@dark13b commented on GitHub (Oct 20, 2026): I'd like to take this one as well. I have Ghidra 12.1.4 on Windows 11, so I can check this against the real provider instead of from source only. I'll reproduce the repeated startup stdout in consecutive input-error replies first, then work on keeping the full session diagnostics accessible without repeating them in every ordinary input-error reply. Startup, crash, cleanup, and memory-failure evidence will stay where it is relevant. A draft PR will follow.

bsakaguchi commented

2026-10-31 00:38:37 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #5340 ([Feature] Optional per-function Ghidra decompile deadline that fails one request instead of closing the session) and sibling #5342. Addresses Ghidra session resilience, log suppression on input errors, and per-operation boundary controls. Cataloged under Cluster BX in consolidation tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #5340 (`[Feature] Optional per-function Ghidra decompile deadline that fails one request instead of closing the session`) and sibling #5342. Addresses Ghidra session resilience, log suppression on input errors, and per-operation boundary controls. Cataloged under Cluster BX in consolidation tracker #4330.

bsakaguchi commented

2026-10-31 00:38:37 +00:00

Author

Owner

@morluto commented on GitHub (Oct 26, 2026):

Thank you for preserving the real MCP evidence and distinguishing client clipping from REA output. #7140 now scopes expected rejected-input diagnostics to the active request after a verified handshake, while retaining request output and keeping full diagnostics for startup, process, protocol and other failures. I am leaving this issue open for the requested caller-accessible complete session diagnostics: the merged change retains the bounded streams in the client and names ephemeral log paths, but that does not by itself establish a public retrieval path. No Ghidra run was performed in this triage.

@morluto commented on GitHub (Oct 26, 2026): Thank you for preserving the real MCP evidence and distinguishing client clipping from REA output. #7140 now scopes expected rejected-input diagnostics to the active request after a verified handshake, while retaining request output and keeping full diagnostics for startup, process, protocol and other failures. I am leaving this issue open for the requested caller-accessible complete session diagnostics: the merged change retains the bounded streams in the client and names ephemeral log paths, but that does not by itself establish a public retrieval path. No Ghidra run was performed in this triage.

bsakaguchi commented

2026-10-31 00:38:38 +00:00

Author

Owner

@cpruijsen commented on GitHub (Oct 30, 2026):

Re-ran the two calls from the report against real Ghidra on main (cd878ad57, includes #7140): macOS 27.0.1 arm64, Ghidra 12.1.4, Node 24.22.1, stdio MCP, the arm64 slice of Elgato Control Center (main still refuses the universal file without #6392).

Both replies are still invalid_request with remote_message "Unknown Ghidra Program identity: Elgato Control Center; active program: target-d3703d6687a0.bin". Each is now 2,355 bytes, down from 35,460, with diagnostic_scope: "request", empty failure_output (complete: true) and no cumulative stdout/stderr. So the rejected-input part is fixed.

On the part you kept open: the replies name ghidra_log_path and script_log_path, but both were gone once the session ended, so there is no way to get the full session output afterwards.

@cpruijsen commented on GitHub (Oct 30, 2026): Re-ran the two calls from the report against real Ghidra on main (cd878ad57, includes #7140): macOS 27.0.1 arm64, Ghidra 12.1.4, Node 24.22.1, stdio MCP, the arm64 slice of Elgato Control Center (main still refuses the universal file without #6392). Both replies are still `invalid_request` with `remote_message` "Unknown Ghidra Program identity: Elgato Control Center; active program: target-d3703d6687a0.bin". Each is now 2,355 bytes, down from 35,460, with `diagnostic_scope: "request"`, empty `failure_output` (`complete: true`) and no cumulative stdout/stderr. So the rejected-input part is fixed. On the part you kept open: the replies name `ghidra_log_path` and `script_log_path`, but both were gone once the session ended, so there is no way to get the full session output afterwards.

bsakaguchi referenced this issue

2026-10-31 01:08:39 +00:00

[PR #3270] [MERGED] docs(readme): add the Linux ps prerequisite to the Korean README #5688

bsakaguchi commented

2026-11-01 21:10:59 +00:00

Author

Owner

@dark13b commented on GitHub (Oct 30, 2026):

Thanks for re-running this on current main and confirming the split: the rejected-input replies are now request-scoped (2,355 bytes, empty failure_output, no cumulative stdout/stderr), and the remaining gap is that ghidra_log_path / script_log_path no longer exist once the session ends.

I opened a draft for that gap: https://github.com/morluto/rea/pull/8199 (refs this issue, does not close it). It is a draft proposal; it may change or not land in this form.

What it does:

Opt-in only. With REA_GHIDRA_DIAGNOSTICS_DIR set to an existing absolute directory, each Ghidra run writes its own subdirectory with stdout.log, stderr.log, ghidra.log, script.log and a manifest.json (run ID, target SHA-256, profile digest, per-file status). Error diagnostics then include diagnostics_export_path.
stdout/stderr are captured as they arrive, so they are not limited by the in-memory retention bound. ghidra.log and script.log are copied after the process stops.
The manifest marks each file as complete, incomplete, captured, unavailable or error. A copied Ghidra log is recorded as captured, not complete, since REA cannot tell whether Ghidra finished writing it.
The bridge token is redacted, including across chunk boundaries. Nothing else is rewritten, so the files can contain local paths and target data.
REA does not delete the export; removal is left to the caller. If the export itself fails, the runtime is still cleaned up and close_binary reports a diagnostic_export failure instead of cleanup_incomplete.

What is not verified:

I ran the real MCP workflow (open, two invalid-document calls, close) on Windows x64 with Ghidra 12.1.4 only. The exported files were readable after close and the runtime roots were removed.
The two new fixture-based tests fail on my Windows machine because the fake Ghidra process exits with EBUSY on an endpoint rename. The same fixture already fails 69 other tests on the PR's parent commit, so those two rely on Linux CI.
No macOS or arm64 run, and no reproduction with the Elgato binary.

No action is needed from you. If you ever try it on macOS arm64 and want to share what you see, that would be useful, but nothing is expected.

@dark13b commented on GitHub (Oct 30, 2026): Thanks for re-running this on current main and confirming the split: the rejected-input replies are now request-scoped (2,355 bytes, empty `failure_output`, no cumulative stdout/stderr), and the remaining gap is that `ghidra_log_path` / `script_log_path` no longer exist once the session ends. I opened a draft for that gap: https://github.com/morluto/rea/pull/8199 (refs this issue, does not close it). It is a draft proposal; it may change or not land in this form. What it does: - Opt-in only. With `REA_GHIDRA_DIAGNOSTICS_DIR` set to an existing absolute directory, each Ghidra run writes its own subdirectory with `stdout.log`, `stderr.log`, `ghidra.log`, `script.log` and a `manifest.json` (run ID, target SHA-256, profile digest, per-file status). Error diagnostics then include `diagnostics_export_path`. - stdout/stderr are captured as they arrive, so they are not limited by the in-memory retention bound. `ghidra.log` and `script.log` are copied after the process stops. - The manifest marks each file as complete, incomplete, captured, unavailable or error. A copied Ghidra log is recorded as captured, not complete, since REA cannot tell whether Ghidra finished writing it. - The bridge token is redacted, including across chunk boundaries. Nothing else is rewritten, so the files can contain local paths and target data. - REA does not delete the export; removal is left to the caller. If the export itself fails, the runtime is still cleaned up and `close_binary` reports a `diagnostic_export` failure instead of `cleanup_incomplete`. What is not verified: - I ran the real MCP workflow (open, two invalid-document calls, close) on Windows x64 with Ghidra 12.1.4 only. The exported files were readable after close and the runtime roots were removed. - The two new fixture-based tests fail on my Windows machine because the fake Ghidra process exits with `EBUSY` on an endpoint rename. The same fixture already fails 69 other tests on the PR's parent commit, so those two rely on Linux CI. - No macOS or arm64 run, and no reproduction with the Elgato binary. No action is needed from you. If you ever try it on macOS arm64 and want to share what you see, that would be useful, but nothing is expected.

bsakaguchi commented

2026-11-01 21:10:59 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Real-provider macOS control for exact PR #8199 head 43592896eb9469a19af0144da1007546ac8dd6ed, Node 24.22.1, Ghidra 12.1.4 and JDK 21. The target was a self-built Mach-O; it was not executed and its SHA-256 was unchanged.

With export disabled, a real stdio MCP session opened the target, listed procedures, returned request-scoped invalid_request for two wrong document selectors, listed procedures again, and closed. No diagnostic export directory was created and the private runtime was removed.
With an existing caller-selected diagnostics directory, the same sequence returned an export path in each request error. After close, the private runtime was gone and the exported manifest and four logs remained. The manifest preserved target/profile/run identity, marked both captured streams complete, both copied Ghidra files captured, and producer log completeness unknown.
For a controlled finalization failure, I created a caller-owned manifest.json.tmp in a new export directory before close. Close returned execution_failure for diagnostic_export; the private runtime was gone, binary_session reported closed with no cleanup pending, and a second close succeeded. The pre-existing marker was preserved and no final manifest was written.

These are macOS controls on one target and the exact PR head, complementing the PR's Windows evidence. The collision was injected deliberately; it is not a measured natural failure rate. This run did not verify credential redaction, startup failure, CLI, other platforms, or real cleanup failures alongside an export failure.

@morluto commented on GitHub (Oct 31, 2026): Real-provider macOS control for exact [PR #8199](https://github.com/morluto/rea/pull/8199) head `43592896eb9469a19af0144da1007546ac8dd6ed`, Node 24.22.1, Ghidra 12.1.4 and JDK 21. The target was a self-built Mach-O; it was not executed and its SHA-256 was unchanged. * With export disabled, a real stdio MCP session opened the target, listed procedures, returned request-scoped `invalid_request` for two wrong document selectors, listed procedures again, and closed. No diagnostic export directory was created and the private runtime was removed. * With an existing caller-selected diagnostics directory, the same sequence returned an export path in each request error. After close, the private runtime was gone and the exported manifest and four logs remained. The manifest preserved target/profile/run identity, marked both captured streams `complete`, both copied Ghidra files `captured`, and producer log completeness `unknown`. * For a controlled finalization failure, I created a caller-owned `manifest.json.tmp` in a new export directory before close. Close returned `execution_failure` for `diagnostic_export`; the private runtime was gone, `binary_session` reported closed with no cleanup pending, and a second close succeeded. The pre-existing marker was preserved and no final manifest was written. These are macOS controls on one target and the exact PR head, complementing the PR's Windows evidence. The collision was injected deliberately; it is not a measured natural failure rate. This run did not verify credential redaction, startup failure, CLI, other platforms, or real cleanup failures alongside an export failure.

bsakaguchi commented

2026-11-01 21:10:59 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Controlled real-Ghidra redaction follow-up to the earlier diagnostic-export check: on macOS arm64 with Ghidra 12.1.4 and JDK 21, I used a disposable checkout of PR #8199 head 43592896 with two deliberate test-only changes: the client used a known synthetic transport token, and the real bridge emitted that value with marked ordinary text to stdout, stderr, the Ghidra script logger, and an invalid_request response. No live generated credential was read or recovered.

Through public MCP, the injected response retained its marker and a caller-like /tmp/password-fixture?token=public-value string, replaced the synthetic transport token with [REDACTED], and the same session answered a normal query afterward. After close, all four exported logs (stdout, stderr, ghidra, and script) contained the injected marker, the redaction marker and the ordinary value, but none contained the synthetic credential. The manifest matched the target digest, the POSIX descriptor was absent after handshake, the runtime was removed, and the unexecuted target was unchanged. The owned test export was removed after inspection.

This corrects my earlier implication that a real-session redaction check necessarily required recovering the live descriptor: controlled injection can exercise the real bridge and export pipeline safely. The result is for the modified test build, one error path, macOS arm64, and a synthetic credential; it does not establish behavior for arbitrary log boundaries, startup failures, Windows or other releases. The existing focused tests separately cover synthetic split-chunk redaction. No production change or new security mechanism is proposed.

@morluto commented on GitHub (Oct 31, 2026): Controlled real-Ghidra redaction follow-up to the earlier diagnostic-export check: on macOS arm64 with Ghidra 12.1.4 and JDK 21, I used a disposable checkout of PR #8199 head `43592896` with **two deliberate test-only changes**: the client used a known synthetic transport token, and the real bridge emitted that value with marked ordinary text to stdout, stderr, the Ghidra script logger, and an `invalid_request` response. No live generated credential was read or recovered. Through public MCP, the injected response retained its marker and a caller-like `/tmp/password-fixture?token=public-value` string, replaced the synthetic transport token with `[REDACTED]`, and the same session answered a normal query afterward. After close, all four exported logs (`stdout`, `stderr`, `ghidra`, and `script`) contained the injected marker, the redaction marker and the ordinary value, but none contained the synthetic credential. The manifest matched the target digest, the POSIX descriptor was absent after handshake, the runtime was removed, and the unexecuted target was unchanged. The owned test export was removed after inspection. This corrects my earlier implication that a real-session redaction check necessarily required recovering the live descriptor: controlled injection can exercise the real bridge and export pipeline safely. The result is for the **modified test build**, one error path, macOS arm64, and a synthetic credential; it does not establish behavior for arbitrary log boundaries, startup failures, Windows or other releases. The existing focused tests separately cover synthetic split-chunk redaction. No production change or new security mechanism is proposed.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3270

No description provided.

Rows
Columns