morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #5509] [CLOSED] fix(ghidra): keep the session transcript out of bridge failure replies #6943

New issue

Closed

opened 2026-11-01 21:44:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:44:13 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5509
Author: @dark13b
Created: 10/20/2026
Status: ❌ Closed

Base: main ← Head: fix/ghidra-input-error-diagnostics

📝 Commits (2)

72dfc1f fix(ghidra): keep the session transcript out of bridge failure replies
fa85855 Merge upstream/main into fix/ghidra-input-error-diagnostics

📄 Description

Summary

Fixes #5400

When the Ghidra bridge answers a request on an established session with an ordinary failure (for example an unknown document), the error reply repeated the whole cumulative Ghidra session transcript in details.diagnostics.stdout.text / stderr.text. Two consecutive input errors carried the same startup, import and auto-analysis log. This keeps the operation's own output and replaces the repeated transcript text with an explicit identity: byte counts, a SHA-256 of the omitted text, and text_omitted: "session_transcript".

Problem and expected behavior

Smallest trigger, real Ghidra 12.1.4 over stdio MCP: open a target, then call procedure_pseudo_code {"document": "Wrong Program Name", "procedure": "<existing>"} twice.

Both replies correctly return invalid_request with Unknown Ghidra Program identity: Wrong Program Name; active program: target-….exe. In each one, diagnostics.failure_output is {stdout: "", stderr: "", complete: true}, so the operation produced no output, yet diagnostics.stdout.text holds 11,465 bytes of the session log (identical SHA-256 in both replies) and stderr.text another 1,024 bytes.

Expected (from the issue): deliver the typed error and operation-specific diagnostics without repeating the complete startup transcript in every ordinary input-error reply, keep startup, crash, cleanup and memory-failure evidence where it is relevant, and do not silently truncate or remove canonical diagnostic data.

Change and scope

GhidraDiagnostics.ts adds operationScopedGhidraDiagnostics. Only when failure_output.complete === true does it drop text from stdout and stderr, keeping bytes and retained_bytes and adding retained_sha256 (of the already-redacted text) and text_omitted: "session_transcript". If failure_output is missing, malformed or incomplete, the diagnostics are returned unchanged, because the operation scope is not proven.
GhidraClient.ts applies it in the request-queue path only, next to the existing process / protocol refresh, for remote failures. A remote failure there means the bridge received the request on an established session and returned a well-formed failure.

Not changed:

Start, process, protocol, timeout, analysis-timeout and cleanup failures keep the full transcript. A remote failure during startup still goes through #startupFailure, which rebuilds full diagnostics.
failure_output (including observed_oome), exit fields, ghidra_log_path / script_log_path, and all other diagnostic keys are untouched. During the session, Ghidra's own log stays on disk at ghidra_log_path (13,220 bytes in the run below).
GhidraClient.diagnostics() still returns the full streams.
Other providers are not touched.

Open question for maintainers: text_omitted and retained_sha256 are new keys inside details.diagnostics.stdout / stderr, which are free-form JSON in the error contract. If you prefer another delivery (for example one full transcript per run and references afterwards), I can switch to it.

Follow the contribution scope.
Unresolved scope, authority, side effects, or required verification block
integration. Link the required maintainer scope decision for new providers or
capabilities involving target execution, injection, mutation, or protection
recovery; ordinary fixes within existing boundaries may state "not applicable".
Draft PRs may leave the scope decision pending and the readiness and merge checklists
unchecked. Maintainers can help resolve open questions.

Contract and boundary impact

Semantic owner and earliest changed stage: GhidraClient request queue for established-session operations; projection in GhidraDiagnostics.ts.
Contribution scope fit and linked maintainer decision: ordinary fix within existing boundaries; not applicable.
CLI and MCP/tool-catalog contract: error codes, categories, messages, issues and failure_output are unchanged. For bridge-reported failures with complete operation output, details.diagnostics.stdout / stderr no longer carry text; they carry bytes, retained_bytes, retained_sha256, text_omitted. No tool schema changes.
Provider, bridge, target-format, or platform compatibility: Ghidra only; no bridge (Java) change.
Evidence, artifact, provenance, or reconstruction contract: none; this is error diagnostics only.
Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable; no new process, file or cleanup behavior.
Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated:
- src/ghidra/GhidraDiagnostics.test.ts: complete operation output drops the text and keeps bytes, retained_bytes, the matching SHA-256 and text_omitted; incomplete and missing failure_output return the diagnostics unchanged.
- tests/boundary/providers/ghidra/ghidraClient.test.ts: the existing remote_error fixture test now also asserts no stdout.text, text_omitted: "session_transcript", a 64-hex retained_sha256, and a complete failure_output.
Base reproduction or other evidence: real Ghidra measurement below, before and after, same machine, fixture and calls.

User-visible CLI/MCP output (if applicable): details.diagnostics.stdout after the change:

{"bytes": 11467, "retained_bytes": 11467, "retained_sha256": "d6690f980a20df65183b352cca7945c3c3d0da8faae01c592c04901a73418c27", "text_omitted": "session_transcript"}

Remaining proof gaps: see the Windows note under validation; the boundary test runs in CI.

Measurement: stdio MCP against node scripts/rea.mjs mcp from this checkout, Ghidra 12.1.4 PUBLIC, Temurin 21.0.12.1, Windows 11 x64, Node 24.21.0. Target: build/fixtures/rea-ghidra-windows.exe from npm run build:ghidra-windows-fixture (source-owned static PE). Sequence: open_binary {provider_id: "ghidra"}, list_procedures {}, then procedure_pseudo_code twice with the wrong document, then close_binary. Tokens are estimates from tokenx estimateTokenCount over the reply text, not a model tokenizer.

Per `procedure_pseudo_code` input-error reply	`main` (`21e0c580e`)	This PR
Reply text, UTF-8 bytes	16,705	3,660
Estimated tokens (`tokenx`)	4,933	1,096
`details.diagnostics`, bytes	16,268	3,223
`diagnostics.stdout.text`, bytes	11,464	omitted (`bytes: 11,467`, `retained_sha256` given)
`diagnostics.stderr.text`, bytes	1,024	omitted (identity given)
`failure_output`	empty, `complete: true`	unchanged
Elapsed, first / second call	8 ms / 3 ms	10 ms / 4 ms

Both consecutive replies were byte-identical in each run. Over the two calls in this sequence that is 33,410 vs 7,320 bytes, or about 9,866 vs 2,192 estimated tokens. list_procedures before them was unaffected (4,078 bytes in both runs). The transcript is a few bytes different between runs because it contains per-run temporary paths.

Elapsed time of the failing calls is a few milliseconds either way; I did not measure CPU time or peak memory, and the added SHA-256 runs over at most the retained stream text.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Windows 11 x64, Node 24.21.0, npm 11.19.0, Ghidra 12.1.4 PUBLIC, Temurin 21.0.12.1. The Windows native addon and manifest were restored from the published rea-agents@6.7.0 package, matching the checkout version, as described in native/windows/README.md.

npm run build:cached — passed
Real Ghidra stdio MCP sequence above, on main and on this branch — results in the table
npm run test:focused -- src/ghidra/GhidraDiagnostics.test.ts src/ghidra/GhidraSessionError.test.ts src/ghidra/GhidraProvider.behavior.test.ts — 144 passed
npm run test:focused -- tests/boundary/providers/ghidra/ghidraClient.test.ts — 65 failed, 18 passed, 5 skipped, with the same 65 failing tests on main without this change. They fail on this Windows host when tests/fixtures/fakeGhidra.mjs renames bridge-endpoint.json.pending (EBUSY). The updated remote_error test is one of them, so its new assertions have not run locally; I'm relying on CI for that file.
npm run check — passed (lint 0 errors; 1895 warnings, the same count as main, and no new warnings in the touched files)
npm run verify:ghidra:windows — passed (ok: true, Ghidra 12.1.4, authenticated-loopback-tcp, 27 operations, cleanup complete)

Compatibility, safety, and release

Breaking changes or migration steps: none to schemas or codes. A caller that read details.diagnostics.stdout.text from a bridge-reported input error now finds text_omitted and retained_sha256 instead; provider failures still include the text.
Real Hopper/Ghidra, browser, or OS coverage: real Ghidra 12.1.4 on Windows 11 x64 as above. Not run on Linux or macOS.
Package or release metadata impact: none.
Security, privacy, process, or containment review: the omitted text was already redacted; the digest is computed over that redacted text. Fewer local paths and environment lines reach the reply.

Author readiness

Human owner: @dark13b

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5509 **Author:** [@dark13b](https://github.com/dark13b) **Created:** 10/20/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/ghidra-input-error-diagnostics` --- ### 📝 Commits (2) - [`72dfc1f`](https://github.com/morluto/rea/commit/72dfc1f07e2c6ae8bca9a106d6560f4a1b4dd8f5) fix(ghidra): keep the session transcript out of bridge failure replies - [`fa85855`](https://github.com/morluto/rea/commit/fa85855461869aae877f09206f624d696c5743e6) Merge upstream/main into fix/ghidra-input-error-diagnostics ### 📄 Description ## Summary Fixes #5400 When the Ghidra bridge answers a request on an established session with an ordinary failure (for example an unknown `document`), the error reply repeated the whole cumulative Ghidra session transcript in `details.diagnostics.stdout.text` / `stderr.text`. Two consecutive input errors carried the same startup, import and auto-analysis log. This keeps the operation's own output and replaces the repeated transcript text with an explicit identity: byte counts, a SHA-256 of the omitted text, and `text_omitted: "session_transcript"`. ## Problem and expected behavior Smallest trigger, real Ghidra 12.1.4 over stdio MCP: open a target, then call `procedure_pseudo_code {"document": "Wrong Program Name", "procedure": "<existing>"}` twice. Both replies correctly return `invalid_request` with `Unknown Ghidra Program identity: Wrong Program Name; active program: target-….exe`. In each one, `diagnostics.failure_output` is `{stdout: "", stderr: "", complete: true}`, so the operation produced no output, yet `diagnostics.stdout.text` holds 11,465 bytes of the session log (identical SHA-256 in both replies) and `stderr.text` another 1,024 bytes. Expected (from the issue): deliver the typed error and operation-specific diagnostics without repeating the complete startup transcript in every ordinary input-error reply, keep startup, crash, cleanup and memory-failure evidence where it is relevant, and do not silently truncate or remove canonical diagnostic data. ## Change and scope - `GhidraDiagnostics.ts` adds `operationScopedGhidraDiagnostics`. Only when `failure_output.complete === true` does it drop `text` from `stdout` and `stderr`, keeping `bytes` and `retained_bytes` and adding `retained_sha256` (of the already-redacted text) and `text_omitted: "session_transcript"`. If `failure_output` is missing, malformed or incomplete, the diagnostics are returned unchanged, because the operation scope is not proven. - `GhidraClient.ts` applies it in the request-queue path only, next to the existing `process` / `protocol` refresh, for `remote` failures. A `remote` failure there means the bridge received the request on an established session and returned a well-formed failure. Not changed: - Start, process, protocol, timeout, analysis-timeout and cleanup failures keep the full transcript. A `remote` failure during startup still goes through `#startupFailure`, which rebuilds full diagnostics. - `failure_output` (including `observed_oome`), exit fields, `ghidra_log_path` / `script_log_path`, and all other diagnostic keys are untouched. During the session, Ghidra's own log stays on disk at `ghidra_log_path` (13,220 bytes in the run below). - `GhidraClient.diagnostics()` still returns the full streams. - Other providers are not touched. Open question for maintainers: `text_omitted` and `retained_sha256` are new keys inside `details.diagnostics.stdout` / `stderr`, which are free-form JSON in the error contract. If you prefer another delivery (for example one full transcript per run and references afterwards), I can switch to it. Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). Unresolved scope, authority, side effects, or required verification block integration. Link the required maintainer scope decision for new providers or capabilities involving target execution, injection, mutation, or protection recovery; ordinary fixes within existing boundaries may state "not applicable". Draft PRs may leave the scope decision pending and the readiness and merge checklists unchecked. Maintainers can help resolve open questions. ## Contract and boundary impact - Semantic owner and earliest changed stage: `GhidraClient` request queue for established-session operations; projection in `GhidraDiagnostics.ts`. - Contribution scope fit and linked maintainer decision: ordinary fix within existing boundaries; not applicable. - CLI and MCP/tool-catalog contract: error codes, categories, messages, issues and `failure_output` are unchanged. For bridge-reported failures with complete operation output, `details.diagnostics.stdout` / `stderr` no longer carry `text`; they carry `bytes`, `retained_bytes`, `retained_sha256`, `text_omitted`. No tool schema changes. - Provider, bridge, target-format, or platform compatibility: Ghidra only; no bridge (Java) change. - Evidence, artifact, provenance, or reconstruction contract: none; this is error diagnostics only. - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable; no new process, file or cleanup behavior. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [ ] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [x] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - `src/ghidra/GhidraDiagnostics.test.ts`: complete operation output drops the text and keeps `bytes`, `retained_bytes`, the matching SHA-256 and `text_omitted`; incomplete and missing `failure_output` return the diagnostics unchanged. - `tests/boundary/providers/ghidra/ghidraClient.test.ts`: the existing `remote_error` fixture test now also asserts no `stdout.text`, `text_omitted: "session_transcript"`, a 64-hex `retained_sha256`, and a complete `failure_output`. - Base reproduction or other evidence: real Ghidra measurement below, before and after, same machine, fixture and calls. - User-visible CLI/MCP output (if applicable): `details.diagnostics.stdout` after the change: ```json {"bytes": 11467, "retained_bytes": 11467, "retained_sha256": "d6690f980a20df65183b352cca7945c3c3d0da8faae01c592c04901a73418c27", "text_omitted": "session_transcript"} ``` - Remaining proof gaps: see the Windows note under validation; the boundary test runs in CI. Measurement: stdio MCP against `node scripts/rea.mjs mcp` from this checkout, Ghidra 12.1.4 PUBLIC, Temurin 21.0.12.1, Windows 11 x64, Node 24.21.0. Target: `build/fixtures/rea-ghidra-windows.exe` from `npm run build:ghidra-windows-fixture` (source-owned static PE). Sequence: `open_binary {provider_id: "ghidra"}`, `list_procedures {}`, then `procedure_pseudo_code` twice with the wrong `document`, then `close_binary`. Tokens are estimates from `tokenx` `estimateTokenCount` over the reply text, not a model tokenizer. | Per `procedure_pseudo_code` input-error reply | `main` (21e0c580e) | This PR | | --- | ---: | ---: | | Reply text, UTF-8 bytes | 16,705 | 3,660 | | Estimated tokens (`tokenx`) | 4,933 | 1,096 | | `details.diagnostics`, bytes | 16,268 | 3,223 | | `diagnostics.stdout.text`, bytes | 11,464 | omitted (`bytes: 11,467`, `retained_sha256` given) | | `diagnostics.stderr.text`, bytes | 1,024 | omitted (identity given) | | `failure_output` | empty, `complete: true` | unchanged | | Elapsed, first / second call | 8 ms / 3 ms | 10 ms / 4 ms | Both consecutive replies were byte-identical in each run. Over the two calls in this sequence that is 33,410 vs 7,320 bytes, or about 9,866 vs 2,192 estimated tokens. `list_procedures` before them was unaffected (4,078 bytes in both runs). The transcript is a few bytes different between runs because it contains per-run temporary paths. Elapsed time of the failing calls is a few milliseconds either way; I did not measure CPU time or peak memory, and the added SHA-256 runs over at most the retained stream text. For evidence-bearing changes: - [ ] Observed, derived, and inferred claims remain distinguishable. - [ ] Artifact identity, source provenance, and failed attempts remain preserved. - [ ] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Windows 11 x64, Node 24.21.0, npm 11.19.0, Ghidra 12.1.4 PUBLIC, Temurin 21.0.12.1. The Windows native addon and manifest were restored from the published `rea-agents@6.7.0` package, matching the checkout version, as described in `native/windows/README.md`. - `npm run build:cached` — passed - Real Ghidra stdio MCP sequence above, on `main` and on this branch — results in the table - `npm run test:focused -- src/ghidra/GhidraDiagnostics.test.ts src/ghidra/GhidraSessionError.test.ts src/ghidra/GhidraProvider.behavior.test.ts` — 144 passed - `npm run test:focused -- tests/boundary/providers/ghidra/ghidraClient.test.ts` — 65 failed, 18 passed, 5 skipped, with the same 65 failing tests on `main` without this change. They fail on this Windows host when `tests/fixtures/fakeGhidra.mjs` renames `bridge-endpoint.json.pending` (`EBUSY`). The updated `remote_error` test is one of them, so its new assertions have not run locally; I'm relying on CI for that file. - `npm run check` — passed (lint 0 errors; 1895 warnings, the same count as `main`, and no new warnings in the touched files) - `npm run verify:ghidra:windows` — passed (`ok: true`, Ghidra 12.1.4, `authenticated-loopback-tcp`, 27 operations, cleanup `complete`) ## Compatibility, safety, and release - Breaking changes or migration steps: none to schemas or codes. A caller that read `details.diagnostics.stdout.text` from a bridge-reported input error now finds `text_omitted` and `retained_sha256` instead; provider failures still include the text. - Real Hopper/Ghidra, browser, or OS coverage: real Ghidra 12.1.4 on Windows 11 x64 as above. Not run on Linux or macOS. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the omitted text was already redacted; the digest is computed over that redacted text. Fewer local paths and environment lines reach the reply. ## Author readiness Human owner: @dark13b - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:44:13 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:48:02 +00:00

[PR #6943] [MERGED] feat(erlang): query selected BEAM table facts #7464

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6943

No description provided.

Rows
Columns