morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6048] [Bug] Native value tracing returns success and records Evidence after late cancellation #3696

New issue

Closed

opened 2026-10-31 00:43:08 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:43:08 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6048

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The public trace-native-values CLI returned exit 0 and completed trace_native_values Evidence after its request signal was aborted immediately before the final successful provider reply reached the workflow. A delegating spy also observed one call to the real session Evidence recorder for that trace. The uninterrupted control produced the same graph. This was a controlled provider-reply interleaving through production CLI/session/workflow code.

Reproduction

The CLI request exercised was rea trace-native-values sample fixture --provider hopper --json. Save the following as tests/process-global/nativeValueTraceCancellation.test.ts in a built checkout and run:

node node_modules/vitest/vitest.mjs run tests/process-global/nativeValueTraceCancellation.test.ts --maxWorkers 1

The socket fixture handles provider launch/handshake/shutdown. Only the analyze_function reply is substituted with the repository’s valid dossier, and interruption is delivered through the CLI’s registered SIGINT handler before returning that reply. The system executable supplies target admission and is never executed. Both cases below were executed as part of a focused combined run; all 12 controls passed their observed-behavior assertions.

import { copyFile, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, expect, it, vi } from "vitest";
import type { BridgeLauncher, BridgeSession } from "../../src/hopper/BridgeLauncher.js";
import { HopperFixtureLauncher } from "../boundary/providers/hopper/hopperClient.fixture.js";
import { serveCli } from "../boundary/cli/cliServeHarness.js";
import { waitForExit } from "../support/process/processFixture.js";
import { HopperClient } from "../../src/hopper/HopperClient.js";
import { BinarySession } from "../../src/application/binary/BinarySession.js";
import { ghidraFunctionDossier } from "../../src/domain/ghidraValues.fixture.js";
import { ok } from "../../src/domain/result.js";

const fixture = vi.hoisted(() => ({ launcher: undefined as BridgeLauncher | undefined }));
vi.mock("../../src/hopper/BridgeLauncher.js", async (original) => ({
  ...(await original<typeof import("../../src/hopper/BridgeLauncher.js")>()),
  HopperApplicationLauncher: class implements BridgeLauncher {
    launch(session: BridgeSession) {
      if (!fixture.launcher) throw new Error("No fixture launcher");
      return fixture.launcher.launch(session);
    }
  },
}));
const exitCode = process.exitCode;
afterEach(() => { vi.restoreAllMocks(); process.exitCode = exitCode; });

it.each([false, true])("native value trace public CLI caller abort=%s at final successful provider reply", async (cancel) => {
  const root = await mkdtemp(join(tmpdir(), "rea-trace-cancel-"));
  const target = join(root, "sample");
  const launcher = new HopperFixtureLauncher();
  fixture.launcher = launcher;
  const record = BinarySession.prototype.recordEvidence;
  let tracedEvidence = 0;
  vi.spyOn(BinarySession.prototype, "recordEvidence").mockImplementation(function(evidence) {
    if (evidence.operation === "trace_native_values") tracedEvidence++;
    return record.call(this, evidence);
  });
  const original = HopperClient.prototype.callTool;
  let signal: AbortSignal | undefined;
  vi.spyOn(HopperClient.prototype, "callTool").mockImplementation(async function(name, params, options) {
    if (name === "analyze_function") {
      signal = options?.signal;
      if (cancel) process.emit("SIGINT");
      return ok(ghidraFunctionDossier());
    }
    return original.call(this, name, params, options);
  });
  try {
    await copyFile("/usr/bin/true", target);
    const run = await serveCli(["trace-native-values", target, "fixture", "--provider", "hopper", "--json"],
      { HOME: root, HOPPER_LAUNCHER_PATH: process.execPath, REA_LOG_LEVEL: "silent" });
    const output = JSON.parse(run.stdout);
    console.log(JSON.stringify({ case: "native-trace-cli", cancel, aborted: signal?.aborted, exit: process.exitCode ?? run.exitCode,
      code: output.code ?? null, operation: output.operation ?? null, nodes: output.normalized_result?.total_nodes ?? null, tracedEvidence }));
    expect(output.operation).toBe("trace_native_values");
    expect(output.normalized_result.total_nodes).toBe(1);
    expect(signal?.aborted).toBe(cancel);
    expect(tracedEvidence).toBe(1);
  } finally {
    fixture.launcher = undefined;
    for (const child of launcher.processes) {
      if (child.exitCode === null && child.signalCode === null) child.kill("SIGTERM");
      await waitForExit(child, 5000);
    }
    await rm(root, { recursive: true, force: true });
  }
});

Expected behavior

Honor cancellation before returning a successful composed observation, even if the final provider result itself is valid. Retain the completed graph under typed cancellation. The lifecycle contract explicitly requires a final signal check and includes cancellation observed after provider completion. The provider-report regression contract also requires checking the application-consumption boundary. Preserve a real provider/output failure if one already exists; these controls supply a valid successful result.

Impact and scope

P2: a bounded cancellation/completion classification error allows a caller that canceled the trace to receive and record it as a completed observation. A caller that retains its own cancellation state can discard the late successful response. No data loss, target mutation or ongoing provider activity after shutdown was observed. One deterministic canceled public CLI invocation and its successful control were executed. Natural frequency, real-engine behavior, disk snapshot persistence and MCP transport behavior were not measured.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e. Fetched upstream 6d356b84344659085f2b912857b5e27fe0160c9b leaves the relevant producing code unchanged; that newer revision was source-compared, not executed.

Runtime and platform

macOS 27.0.1 (26A434), arm64; Node.js v26.12.0; locked Vitest 5.0.3. The contributor Node 24.18.0 pin was not exercised. The Hopper socket fixture was used; no real Hopper engine was launched.

Target and artifact context

A system native executable supplies target admission; authored provider facts come from repository fixtures. The target bytes are never executed.

Interface

CLI

Earliest failure stage

Cleanup, cancellation, or lifecycle

Relevant output or evidence

Observed public CLI subset:

{"cancel":false,"aborted":false,"exit":0,"code":null,"operation":"trace_native_values","nodes":1,"tracedEvidence":1}
{"cancel":true,"aborted":true,"exit":0,"code":null,"operation":"trace_native_values","nodes":1,"tracedEvidence":1}

tracedEvidence counts calls that delegate to the actual session recorder. Direct EnhancedTools.execute controls also returned ok: true with the interrupted signal already aborted. All simulated provider children were awaited and all fixture directories were removed.

Typed outcome or recovery expected

Return AnalysisCancelledError for trace_native_values, retain the caller reason and the completed observation as partial data, and project the existing cancelled outcome. A canceled observation should not be recorded as ordinary completed workflow Evidence.

Analysis or suspected cause

Source-derived: the traversal samples cancellation before dispatch, then returns the graph without a post-await check. EnhancedTools calls this function with the raw analysis port. DirectAnalysis builds successful workflow Evidence, and records it when that result succeeds. Restoring only cancellation causes on existing early exits will not cover this success path.

Additional context

Related #5433 concerns an unsuccessful seed losing its provider diagnostic. This case uses a valid successful seed. #5996 concerns an IDA failure incorrectly replaced by a late cancellation; no prior failure is present here. #5944 concerns cause preservation in a separate derived-operation helper. #4227 requests focused lifecycle findings. These require different fixes. No unknown-registry mutation is claimed: trace_native_values is excluded by recordWorkflowUnknowns.

Related but separate: #6049 concerns Hopper regex search returning success after cancellation during matching-worker termination. This report concerns final provider-result consumption in the native value-trace workflow; the two producing boundaries require separate completion checks.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6048 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior The public `trace-native-values` CLI returned exit 0 and completed `trace_native_values` Evidence after its request signal was aborted immediately before the final successful provider reply reached the workflow. A delegating spy also observed one call to the real session Evidence recorder for that trace. The uninterrupted control produced the same graph. This was a controlled provider-reply interleaving through production CLI/session/workflow code. ### Reproduction The CLI request exercised was `rea trace-native-values sample fixture --provider hopper --json`. Save the following as `tests/process-global/nativeValueTraceCancellation.test.ts` in a built checkout and run: ```sh node node_modules/vitest/vitest.mjs run tests/process-global/nativeValueTraceCancellation.test.ts --maxWorkers 1 ``` The socket fixture handles provider launch/handshake/shutdown. Only the `analyze_function` reply is substituted with the repository’s valid dossier, and interruption is delivered through the CLI’s registered SIGINT handler before returning that reply. The system executable supplies target admission and is never executed. Both cases below were executed as part of a focused combined run; all 12 controls passed their observed-behavior assertions. ```ts import { copyFile, mkdtemp, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { afterEach, expect, it, vi } from "vitest"; import type { BridgeLauncher, BridgeSession } from "../../src/hopper/BridgeLauncher.js"; import { HopperFixtureLauncher } from "../boundary/providers/hopper/hopperClient.fixture.js"; import { serveCli } from "../boundary/cli/cliServeHarness.js"; import { waitForExit } from "../support/process/processFixture.js"; import { HopperClient } from "../../src/hopper/HopperClient.js"; import { BinarySession } from "../../src/application/binary/BinarySession.js"; import { ghidraFunctionDossier } from "../../src/domain/ghidraValues.fixture.js"; import { ok } from "../../src/domain/result.js"; const fixture = vi.hoisted(() => ({ launcher: undefined as BridgeLauncher | undefined })); vi.mock("../../src/hopper/BridgeLauncher.js", async (original) => ({ ...(await original<typeof import("../../src/hopper/BridgeLauncher.js")>()), HopperApplicationLauncher: class implements BridgeLauncher { launch(session: BridgeSession) { if (!fixture.launcher) throw new Error("No fixture launcher"); return fixture.launcher.launch(session); } }, })); const exitCode = process.exitCode; afterEach(() => { vi.restoreAllMocks(); process.exitCode = exitCode; }); it.each([false, true])("native value trace public CLI caller abort=%s at final successful provider reply", async (cancel) => { const root = await mkdtemp(join(tmpdir(), "rea-trace-cancel-")); const target = join(root, "sample"); const launcher = new HopperFixtureLauncher(); fixture.launcher = launcher; const record = BinarySession.prototype.recordEvidence; let tracedEvidence = 0; vi.spyOn(BinarySession.prototype, "recordEvidence").mockImplementation(function(evidence) { if (evidence.operation === "trace_native_values") tracedEvidence++; return record.call(this, evidence); }); const original = HopperClient.prototype.callTool; let signal: AbortSignal | undefined; vi.spyOn(HopperClient.prototype, "callTool").mockImplementation(async function(name, params, options) { if (name === "analyze_function") { signal = options?.signal; if (cancel) process.emit("SIGINT"); return ok(ghidraFunctionDossier()); } return original.call(this, name, params, options); }); try { await copyFile("/usr/bin/true", target); const run = await serveCli(["trace-native-values", target, "fixture", "--provider", "hopper", "--json"], { HOME: root, HOPPER_LAUNCHER_PATH: process.execPath, REA_LOG_LEVEL: "silent" }); const output = JSON.parse(run.stdout); console.log(JSON.stringify({ case: "native-trace-cli", cancel, aborted: signal?.aborted, exit: process.exitCode ?? run.exitCode, code: output.code ?? null, operation: output.operation ?? null, nodes: output.normalized_result?.total_nodes ?? null, tracedEvidence })); expect(output.operation).toBe("trace_native_values"); expect(output.normalized_result.total_nodes).toBe(1); expect(signal?.aborted).toBe(cancel); expect(tracedEvidence).toBe(1); } finally { fixture.launcher = undefined; for (const child of launcher.processes) { if (child.exitCode === null && child.signalCode === null) child.kill("SIGTERM"); await waitForExit(child, 5000); } await rm(root, { recursive: true, force: true }); } }); ``` ### Expected behavior Honor cancellation before returning a successful composed observation, even if the final provider result itself is valid. Retain the completed graph under typed cancellation. The [lifecycle contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/tool-design.md#L101-L107) explicitly requires a final signal check and includes cancellation observed after provider completion. The [provider-report regression contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/testing.md#L321-L326) also requires checking the application-consumption boundary. Preserve a real provider/output failure if one already exists; these controls supply a valid successful result. ### Impact and scope P2: a bounded cancellation/completion classification error allows a caller that canceled the trace to receive and record it as a completed observation. A caller that retains its own cancellation state can discard the late successful response. No data loss, target mutation or ongoing provider activity after shutdown was observed. One deterministic canceled public CLI invocation and its successful control were executed. Natural frequency, real-engine behavior, disk snapshot persistence and MCP transport behavior were not measured. ### REA version rea-agents 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. Fetched upstream `6d356b84344659085f2b912857b5e27fe0160c9b` leaves the relevant producing code unchanged; that newer revision was source-compared, not executed. ### Runtime and platform macOS 27.0.1 (26A434), arm64; Node.js v26.12.0; locked Vitest 5.0.3. The contributor Node 24.18.0 pin was not exercised. The Hopper socket fixture was used; no real Hopper engine was launched. ### Target and artifact context A system native executable supplies target admission; authored provider facts come from repository fixtures. The target bytes are never executed. ### Interface CLI ### Earliest failure stage Cleanup, cancellation, or lifecycle ### Relevant output or evidence Observed public CLI subset: ```json {"cancel":false,"aborted":false,"exit":0,"code":null,"operation":"trace_native_values","nodes":1,"tracedEvidence":1} {"cancel":true,"aborted":true,"exit":0,"code":null,"operation":"trace_native_values","nodes":1,"tracedEvidence":1} ``` `tracedEvidence` counts calls that delegate to the actual session recorder. Direct `EnhancedTools.execute` controls also returned `ok: true` with the interrupted signal already aborted. All simulated provider children were awaited and all fixture directories were removed. ### Typed outcome or recovery expected Return `AnalysisCancelledError` for `trace_native_values`, retain the caller reason and the completed observation as partial data, and project the existing `cancelled` outcome. A canceled observation should not be recorded as ordinary completed workflow Evidence. ### Analysis or suspected cause Source-derived: [the traversal](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/native/NativeValueTrace.ts#L74-L114) samples cancellation before dispatch, then [returns the graph](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/native/NativeValueTrace.ts#L418-L449) without a post-await check. [EnhancedTools](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/EnhancedTools.ts#L239-L240) calls this function with the raw analysis port. [DirectAnalysis](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/DirectAnalysis.ts#L469-L486) builds successful workflow Evidence, and [records it](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/DirectAnalysis.ts#L369-L385) when that result succeeds. Restoring only cancellation causes on existing early exits will not cover this success path. ### Additional context Related #5433 concerns an unsuccessful seed losing its provider diagnostic. This case uses a valid successful seed. [#5996](https://github.com/morluto/rea/issues/5996) concerns an IDA failure incorrectly replaced by a late cancellation; no prior failure is present here. #5944 concerns cause preservation in a separate derived-operation helper. #4227 requests focused lifecycle findings. These require different fixes. No unknown-registry mutation is claimed: trace_native_values is excluded by recordWorkflowUnknowns. Related but separate: [#6049](https://github.com/morluto/rea/issues/6049) concerns Hopper regex search returning success after cancellation during matching-worker termination. This report concerns final provider-result consumption in the native value-trace workflow; the two producing boundaries require separate completion checks.

bsakaguchi closed this issue

2026-10-31 00:43:09 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3696

No description provided.

Rows
Columns