[GH-ISSUE #5939] [Bug] NativeAOT snapshot digest rejection loses its reason and observed hash #3611

Closed
opened 2026-10-31 00:42:22 +00:00 by bsakaguchi · 1 comment
Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5939

Before submitting

  • I searched the open and closed issues for a related report.
  • I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

When the owned Ghidra target snapshot changes after admission but before NativeAOT metadata derivation, the parser rejects its digest correctly, but the provider reports execution_failure with rea doctor advice. The projected result omits the actual digest and the digest-mismatch explanation. It retains the admitted digest and captured byte count in partial_observation.

A focused production-boundary test reproduced this using actual snapshot creation and file reads, a simulated authenticated bridge, and an authored PE/RTR9.1 fixture. The unchanged control returns complete metadata with three MethodTables. No real Ghidra, public CLI/MCP transport, or target executable was run.

Reproduction

Save the following as src/ghidra/nativeAotSnapshotDigest.test.ts, then run node node_modules/vitest/vitest.mjs run --project adapters src/ghidra/nativeAotSnapshotDigest.test.ts --maxWorkers 1 with the locked dependencies.

The test uses the existing socket fixture for startup. It supplies PE image-base/compiler and load-image observations through the production provider factory, while the actual GhidraClient owns admission, private snapshot files, reads, and cleanup. In the changed case it writes one byte to that private copy immediately before its read; the original source stays unchanged. This is controlled fault injection, not an observed natural occurrence.

import { expect, it } from "vitest";
import { mkdtemp, readFile, rm, writeFile, access } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { GhidraClient } from "./GhidraClient.js";
import { GhidraProvider } from "./GhidraProvider.js";
import { ghidraProfileInstallationHost } from "./GhidraInstallation.fixture.js";
import { GhidraFixtureLauncher } from "../../tests/boundary/providers/ghidra/ghidraClient.fixture.js";
import { nativeAotPeFixture, nativeAotPeDigest } from "../../tests/fixtures/nativeaotPe.js";
import { parseConfig } from "../config/parseConfig.js";
import { silentLogger } from "../logger.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { ok } from "../domain/result.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";

it.each([false, true])("projects a physically read snapshot; changed=%s", async (changed) => {
  const root = await mkdtemp(join(tmpdir(), "nativeaot-digest-"));
  const bytes = nativeAotPeFixture();
  const originalDigest = nativeAotPeDigest(bytes);
  const altered = Buffer.from(bytes);
  altered[altered.length - 1] ^= 1;
  const observedDigest = nativeAotPeDigest(altered);
  const path = join(root, "fixture.exe");
  await writeFile(path, bytes);
  const fixture = new GhidraFixtureLauncher();
  let snapshotPath = "";
  let capturedDigest = "";
  const observations = {
    executable_format: "Portable Executable (PE)",
    language_id: "x86:LE:64:default", compiler_spec_id: "windows",
    image_base: "0x140000000", default_address_space: "ram",
    source_files: [{ name: "fixture.exe", size: bytes.length, original_sha256: originalDigest, modified_sha256: originalDigest }],
    mappings: [], relocations: [], entry_context: [], entry_points: [],
  };
  const config = parseConfig({ GHIDRA_INSTALL_DIR: join(root, "ghidra-install") });
  if (!config.ok) throw config.error;
  const provider = new GhidraProvider(config.value, silentLogger, {}, ghidraProfileInstallationHost, (options) => {
    const { expectedLanguageId: _language, expectedCompilerSpecId: _compiler, ...rest } = options;
    const physical = new GhidraClient({ ...rest, platform: process.platform, startupTimeoutMs: 5000,
      launcher: { launch: async (session) => {
        snapshotPath = session.targetPath;
        return fixture.launch(session);
      } },
    });
    return {
      start: async (signal, progress) => {
        const started = await physical.start(signal, progress);
        if (!started.ok) return started;
        return ok({ ...started.value, target: { ...started.value.target, image_base: "0x140000000", compiler_spec_id: "windows" } });
      },
      callTool: async () => ok(jsonValueSchema.parse(observations)),
      readTargetSnapshot: async (maximum, signal) => {
        if (changed) await writeFile(snapshotPath, altered);
        const captured = await physical.readTargetSnapshot(maximum, signal);
        if (captured.ok && captured.value.kind === "captured") capturedDigest = nativeAotPeDigest(captured.value.bytes);
        return captured;
      },
      close: () => physical.close(),
    };
  });
  const target: BinaryTarget = { path, sha256: originalDigest, kind: "executable", format: "pe", architecture: "x86_64", availableArchitectures: ["x86_64"] };
  const profile = await provider.resolveAnalysisProfile(target);
  if (!profile.ok || profile.value.profile === null) throw new Error("Expected admitted PE profile");
  const client = provider.createClient(target, profile.value.profile);
  try {
    const result = await client.execute("inspect_native_load_image", {});
    expect(capturedDigest).toBe(changed ? observedDigest : originalDigest);
    expect(await readFile(path)).toEqual(bytes);
    if (!changed) {
      expect(result).toMatchObject({ ok: true, value: { result: { observations: { metadata_recovery: [{ status: "complete", method_tables: 3 }] } } } });
      console.log(JSON.stringify({ changed, success: result.ok, capturedDigest }));
    } else {
      expect(result.ok).toBe(false);
      if (result.ok) throw new Error("Expected digest failure");
      const projected = projectAnalysisError(result.error);
      expect(result.error).toMatchObject({ _tag: "ProviderAdapterError", cause: { kind: "protocol", cause: { message: "NativeAOT snapshot digest does not match the admitted artifact identity" } } });
      expect(projected).toMatchObject({ code: "execution_failure", retryable: false });
      expect(JSON.stringify(projected)).not.toContain(observedDigest);
      expect(JSON.stringify(projected)).not.toContain("snapshot digest does not match");
      console.log(JSON.stringify({ changed, originalDigest, observedDigest, capturedDigest, projected }));
    }
  } finally {
    expect(await client.close()).toMatchObject({ ok: true });
    for (const runtime of fixture.runtimeRoots) await expect(access(runtime)).rejects.toMatchObject({ code: "ENOENT" });
    expect(fixture.processes.every((child) => child.exitCode !== null || child.signalCode !== null)).toBe(true);
    await rm(root, { recursive: true, force: true });
  }
}, 15000);

The final focused run passed both assertions of current behavior: unchanged snapshot succeeds; changed snapshot fails with the incorrect generic public classification. The tests also check private runtime removal and child-process exit.

Expected behavior

Preserve fail-closed digest rejection while exposing an artifact-identity failure and its expected/observed identity, with recovery advice to reacquire/reopen a stable target. The result should distinguish this condition from a Ghidra protocol malfunction.

The NativeAOT contract says parsing uses the exact admitted snapshot. Initial snapshot admission already uses AnalysisArtifactChangedError for a digest mismatch, and its recovery advice describes stable acquisition and reopening.

Impact and scope

P2: an identity rejection loses the observed identity and actionable reason, and advises provider diagnosis instead of reacquisition. Metadata derivation still fails closed; this test does not show acceptance of changed bytes or corruption of a successful result. Reopening a stable target is the existing source-supported recovery, but real-engine recovery was not tested.

Demonstrated scope is a 2,560-byte authored PE whose private copy changes before its first derivation. Natural frequency and real-provider occurrence are unknown. Changing only the caller's source after successful import is not this trigger: the parser reads the private copy.

REA version

Source revision 52415975942a4b9c53a817915b313ccc404f280e; package manifest version 6.7.0.

Runtime and platform

macOS 27.0.1 arm64, Node v24.22.0, locked Vitest 5.0.3. This run used Node 24.22.0 rather than the repository's exact pinned Node 24.18.0. Ghidra installation facts and bridge responses were simulated; no Java analysis engine ran.

Target and artifact context

Repository-authored x64 PE32+/RTR9.1 fixture from tests/fixtures/nativeaotPe.ts. The changed copy has the same length and differs in its final byte. No target executable was launched.

Interface

Other

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The focused run produced:

Unchanged: complete NativeAOT metadata, 3 MethodTables
Changed:   ProviderAdapterError -> execution_failure / execution_failure
retryable: false
message: Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.
partial_observation.kind: native-aot-snapshot
partial_observation.capture: {status: captured, bytes: 2560}

Admitted SHA-256: 3125368efaeda60afa0244e6b53bbaf573405a40b127783fd8b7c5d1ea87f17b.
Actual read SHA-256: 0090612c7f2742a0be510ff77e8aeb1bd765f68aa7610cd3e7410ccf55c0180b.

The projected partial_observation.analysis_artifact_sha256 contains the admitted hash. Neither the observed hash nor NativeAOT snapshot digest does not match the admitted artifact identity appears anywhere in the projected error. That message is present in the internal cause chain.

Source reachability: the supported load-image operation invokes the NativeAOT owner. The snapshot reader returns captured bytes; its file-state check compares state within the read, allowing a change completed before it begins to reach the parser's digest guard.

Typed outcome or recovery expected

Return an artifact-change/integrity outcome with the admitted and computed hashes and a reason specific to snapshot identity. Preserve existing partial observations. Direct callers to reopen/reacquire the target rather than diagnose a healthy Ghidra installation. This report does not require changing the fail-closed guard or broadening acquisition authority.

Analysis or suspected cause

Source-derived: the parser computes the digest then throws a plain Error, without attaching either digest to that failure. The owner catches it as a generic protocol failure. Provider projection recognizes typed physical artifact-change failures, but this cause is untyped and becomes ProviderAdapterError.

A typed failure at the digest-producing boundary could preserve both identity values and support correct recovery. Forwarding the current exception text alone cannot recover the computed hash. No implementation change was made.

Additional context

Fetched upstream main ce1f4b1bbaced57529d609f3007a76de583e11c4 retains the relevant parser, snapshot reader, owner, and provider logic. Its separate cleanup-history captured-output projection change does not alter this classification.

#5882 concerns hydration COPY bytes lost during buffer growth. #5890 concerns omitted signature-scan coverage. Both concern parsing content after successful digest admission; this case rejects before PE parsing and needs a distinct error-provenance fix. #1185 concerns unsupported-host admission, #4502 import descriptor finalization, #4752 snapshot-reader cleanup facts, #4753 shared derivation lifecycle, PR #4112 canonical type addresses, and PR #2497 Windows documentation. None fixes this digest-failure projection.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5939 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior When the owned Ghidra target snapshot changes after admission but before NativeAOT metadata derivation, the parser rejects its digest correctly, but the provider reports `execution_failure` with `rea doctor` advice. The projected result omits the actual digest and the digest-mismatch explanation. It retains the admitted digest and captured byte count in `partial_observation`. A focused production-boundary test reproduced this using actual snapshot creation and file reads, a simulated authenticated bridge, and an authored PE/RTR9.1 fixture. The unchanged control returns complete metadata with three MethodTables. No real Ghidra, public CLI/MCP transport, or target executable was run. ### Reproduction Save the following as `src/ghidra/nativeAotSnapshotDigest.test.ts`, then run `node node_modules/vitest/vitest.mjs run --project adapters src/ghidra/nativeAotSnapshotDigest.test.ts --maxWorkers 1` with the locked dependencies. The test uses the existing socket fixture for startup. It supplies PE image-base/compiler and load-image observations through the production provider factory, while the actual `GhidraClient` owns admission, private snapshot files, reads, and cleanup. In the changed case it writes one byte to that private copy immediately before its read; the original source stays unchanged. This is controlled fault injection, not an observed natural occurrence. ```ts import { expect, it } from "vitest"; import { mkdtemp, readFile, rm, writeFile, access } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { GhidraClient } from "./GhidraClient.js"; import { GhidraProvider } from "./GhidraProvider.js"; import { ghidraProfileInstallationHost } from "./GhidraInstallation.fixture.js"; import { GhidraFixtureLauncher } from "../../tests/boundary/providers/ghidra/ghidraClient.fixture.js"; import { nativeAotPeFixture, nativeAotPeDigest } from "../../tests/fixtures/nativeaotPe.js"; import { parseConfig } from "../config/parseConfig.js"; import { silentLogger } from "../logger.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import { jsonValueSchema } from "../domain/jsonValue.js"; import { ok } from "../domain/result.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; it.each([false, true])("projects a physically read snapshot; changed=%s", async (changed) => { const root = await mkdtemp(join(tmpdir(), "nativeaot-digest-")); const bytes = nativeAotPeFixture(); const originalDigest = nativeAotPeDigest(bytes); const altered = Buffer.from(bytes); altered[altered.length - 1] ^= 1; const observedDigest = nativeAotPeDigest(altered); const path = join(root, "fixture.exe"); await writeFile(path, bytes); const fixture = new GhidraFixtureLauncher(); let snapshotPath = ""; let capturedDigest = ""; const observations = { executable_format: "Portable Executable (PE)", language_id: "x86:LE:64:default", compiler_spec_id: "windows", image_base: "0x140000000", default_address_space: "ram", source_files: [{ name: "fixture.exe", size: bytes.length, original_sha256: originalDigest, modified_sha256: originalDigest }], mappings: [], relocations: [], entry_context: [], entry_points: [], }; const config = parseConfig({ GHIDRA_INSTALL_DIR: join(root, "ghidra-install") }); if (!config.ok) throw config.error; const provider = new GhidraProvider(config.value, silentLogger, {}, ghidraProfileInstallationHost, (options) => { const { expectedLanguageId: _language, expectedCompilerSpecId: _compiler, ...rest } = options; const physical = new GhidraClient({ ...rest, platform: process.platform, startupTimeoutMs: 5000, launcher: { launch: async (session) => { snapshotPath = session.targetPath; return fixture.launch(session); } }, }); return { start: async (signal, progress) => { const started = await physical.start(signal, progress); if (!started.ok) return started; return ok({ ...started.value, target: { ...started.value.target, image_base: "0x140000000", compiler_spec_id: "windows" } }); }, callTool: async () => ok(jsonValueSchema.parse(observations)), readTargetSnapshot: async (maximum, signal) => { if (changed) await writeFile(snapshotPath, altered); const captured = await physical.readTargetSnapshot(maximum, signal); if (captured.ok && captured.value.kind === "captured") capturedDigest = nativeAotPeDigest(captured.value.bytes); return captured; }, close: () => physical.close(), }; }); const target: BinaryTarget = { path, sha256: originalDigest, kind: "executable", format: "pe", architecture: "x86_64", availableArchitectures: ["x86_64"] }; const profile = await provider.resolveAnalysisProfile(target); if (!profile.ok || profile.value.profile === null) throw new Error("Expected admitted PE profile"); const client = provider.createClient(target, profile.value.profile); try { const result = await client.execute("inspect_native_load_image", {}); expect(capturedDigest).toBe(changed ? observedDigest : originalDigest); expect(await readFile(path)).toEqual(bytes); if (!changed) { expect(result).toMatchObject({ ok: true, value: { result: { observations: { metadata_recovery: [{ status: "complete", method_tables: 3 }] } } } }); console.log(JSON.stringify({ changed, success: result.ok, capturedDigest })); } else { expect(result.ok).toBe(false); if (result.ok) throw new Error("Expected digest failure"); const projected = projectAnalysisError(result.error); expect(result.error).toMatchObject({ _tag: "ProviderAdapterError", cause: { kind: "protocol", cause: { message: "NativeAOT snapshot digest does not match the admitted artifact identity" } } }); expect(projected).toMatchObject({ code: "execution_failure", retryable: false }); expect(JSON.stringify(projected)).not.toContain(observedDigest); expect(JSON.stringify(projected)).not.toContain("snapshot digest does not match"); console.log(JSON.stringify({ changed, originalDigest, observedDigest, capturedDigest, projected })); } } finally { expect(await client.close()).toMatchObject({ ok: true }); for (const runtime of fixture.runtimeRoots) await expect(access(runtime)).rejects.toMatchObject({ code: "ENOENT" }); expect(fixture.processes.every((child) => child.exitCode !== null || child.signalCode !== null)).toBe(true); await rm(root, { recursive: true, force: true }); } }, 15000); ``` The final focused run passed both assertions of current behavior: unchanged snapshot succeeds; changed snapshot fails with the incorrect generic public classification. The tests also check private runtime removal and child-process exit. ### Expected behavior Preserve fail-closed digest rejection while exposing an artifact-identity failure and its expected/observed identity, with recovery advice to reacquire/reopen a stable target. The result should distinguish this condition from a Ghidra protocol malfunction. The [NativeAOT contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/ghidra-nativeaot.md#L3-L5) says parsing uses the exact admitted snapshot. [Initial snapshot admission](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/ghidra/GhidraTargetSnapshot.ts#L113-L123) already uses `AnalysisArtifactChangedError` for a digest mismatch, and its [recovery advice](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L129-L130) describes stable acquisition and reopening. ### Impact and scope P2: an identity rejection loses the observed identity and actionable reason, and advises provider diagnosis instead of reacquisition. Metadata derivation still fails closed; this test does not show acceptance of changed bytes or corruption of a successful result. Reopening a stable target is the existing source-supported recovery, but real-engine recovery was not tested. Demonstrated scope is a 2,560-byte authored PE whose private copy changes before its first derivation. Natural frequency and real-provider occurrence are unknown. Changing only the caller's source after successful import is not this trigger: the parser reads the private copy. ### REA version Source revision `52415975942a4b9c53a817915b313ccc404f280e`; package manifest version 6.7.0. ### Runtime and platform macOS 27.0.1 arm64, Node v24.22.0, locked Vitest 5.0.3. This run used Node 24.22.0 rather than the repository's exact pinned Node 24.18.0. Ghidra installation facts and bridge responses were simulated; no Java analysis engine ran. ### Target and artifact context Repository-authored x64 PE32+/RTR9.1 fixture from `tests/fixtures/nativeaotPe.ts`. The changed copy has the same length and differs in its final byte. No target executable was launched. ### Interface Other ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence The focused run produced: ```text Unchanged: complete NativeAOT metadata, 3 MethodTables Changed: ProviderAdapterError -> execution_failure / execution_failure retryable: false message: Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed. partial_observation.kind: native-aot-snapshot partial_observation.capture: {status: captured, bytes: 2560} ``` Admitted SHA-256: `3125368efaeda60afa0244e6b53bbaf573405a40b127783fd8b7c5d1ea87f17b`. Actual read SHA-256: `0090612c7f2742a0be510ff77e8aeb1bd765f68aa7610cd3e7410ccf55c0180b`. The projected `partial_observation.analysis_artifact_sha256` contains the admitted hash. Neither the observed hash nor `NativeAOT snapshot digest does not match the admitted artifact identity` appears anywhere in the projected error. That message is present in the internal cause chain. Source reachability: [the supported load-image operation invokes the NativeAOT owner](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/ghidra/GhidraProviderClient.ts#L541-L563). The [snapshot reader](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/ghidra/GhidraClient.ts#L397-L430) returns captured bytes; its [file-state check](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/ghidra/GhidraClient.ts#L549-L568) compares state within the read, allowing a change completed before it begins to reach the parser's digest guard. ### Typed outcome or recovery expected Return an artifact-change/integrity outcome with the admitted and computed hashes and a reason specific to snapshot identity. Preserve existing partial observations. Direct callers to reopen/reacquire the target rather than diagnose a healthy Ghidra installation. This report does not require changing the fail-closed guard or broadening acquisition authority. ### Analysis or suspected cause Source-derived: [the parser computes the digest then throws a plain Error](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/native/nativeAotPe.ts#L1232-L1236), without attaching either digest to that failure. [The owner catches it as a generic protocol failure](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/ghidra/GhidraNativeAotSnapshotOwner.ts#L282-L301). [Provider projection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/ghidra/GhidraProviderClient.ts#L866-L893) recognizes typed physical artifact-change failures, but this cause is untyped and becomes `ProviderAdapterError`. A typed failure at the digest-producing boundary could preserve both identity values and support correct recovery. Forwarding the current exception text alone cannot recover the computed hash. No implementation change was made. ### Additional context Fetched upstream main `ce1f4b1bbaced57529d609f3007a76de583e11c4` retains the relevant parser, snapshot reader, owner, and provider logic. Its separate cleanup-history captured-output projection change does not alter this classification. #5882 concerns hydration COPY bytes lost during buffer growth. #5890 concerns omitted signature-scan coverage. Both concern parsing content after successful digest admission; this case rejects before PE parsing and needs a distinct error-provenance fix. #1185 concerns unsupported-host admission, #4502 import descriptor finalization, #4752 snapshot-reader cleanup facts, #4753 shared derivation lifecycle, PR #4112 canonical type addresses, and PR #2497 Windows documentation. None fixes this digest-failure projection.
Author
Owner

@coygeek commented on GitHub (Oct 24, 2026):

Source at d87e73f2db50be72d33a4b3c7199af35518b441c now retains both hashes in a typed snapshot identity error and maps it to artifact-change recovery with partial observations. The parser regression and owner regression assert the observed hash survives and rejection remains fail-closed.

I ran the cited regression files on this revision; all passed as part of the focused 28-file, 603-test run. Provider-facing checks here use controlled fixtures; this does not claim a run with the real analysis engine. Closing as completed on main.

<!-- gh-comment-id:6380647484 --> @coygeek commented on GitHub (Oct 24, 2026): Source at `d87e73f2db50be72d33a4b3c7199af35518b441c` now [retains both hashes in a typed snapshot identity error](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/domain/native/nativeAotPe.ts#L76-L86) and [maps it to artifact-change recovery with partial observations](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/ghidra/GhidraNativeAotSnapshotOwner.ts#L206-L227). The [parser regression](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/domain/native/nativeAotPe.test.ts#L123-L135) and [owner regression](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/ghidra/GhidraProvider.behavior.test.ts#L2333-L2369) assert the observed hash survives and rejection remains fail-closed. I ran the cited regression files on this revision; all passed as part of the focused 28-file, 603-test run. Provider-facing checks here use controlled fixtures; this does not claim a run with the real analysis engine. Closing as completed on main.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#3611
No description provided.