morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #4009] [Bug] Native call observation returns successful Evidence after cancellation during finalization #1594

New issue

Closed

opened 2026-10-16 21:07:40 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-16 21:07:40 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4009

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

observe_native_calls returns successful complete Evidence when its cancellation signal becomes aborted after the LLDB launch result is classified as exited but before final result consumption. The production tracer still reads its result and captured output and removes its runtime directory after that point. Neither it nor the native provider or CLI workflow checks cancellation before publishing the final success. A cancelled operation can therefore be presented as a completed observation.

The recorded reproduction used the production tracer, provider, binary session, and CLI application workflow with injected LLDB launch and tool-resolution seams. No real debuggee, LLDB process, or OS signal was exercised.

Reproduction

Use a focused source test alongside the native adapter, with the imports below. This is the executed production-boundary reproduction. It follows the existing simulated LLDB result format in tests/boundary/mcp/nativeCallObservationMcp.test.ts and the launch/config seam in tests/boundary/providers/native/lldbCallTracerFailures.test.ts.
Materialize the 32-byte arm64 Mach-O header shown in the test. Supply the observe_native_calls input shown below to runProviderAnalysis, the shared workflow called by the registered observe-native-calls CLI command.
The fake LLDB launch writes a schema-valid completed run and both capture files, aborts the supplied controller, then returns an exited launch result. This fixes the timing at the post-launch finalization boundary while retaining the production tracer's subsequent reads and cleanup. Run the focused test with npm run test:local and its selected file path.

import { expect, it } from "vitest";
import { writeFile, readFile } from "node:fs/promises";
import { join } from "node:path";
import { NativeMacOSProvider } from "./NativeMacOSProvider.js";
import { LldbCallTracer, type NativeCallTracer } from "./LldbCallTracer.js";
import { ok } from "../domain/result.js";
import { createTestBinarySession } from "../../tests/fixtures/binarySession.js";
import { NativeFixtureRunner } from "../../tests/fixtures/nativeCommands.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { runProviderAnalysis } from "../application/DirectAnalysis.js";
import { parseConfig } from "../config/parseConfig.js";
import { silentLogger } from "../logger.js";
const tracer: NativeCallTracer & { calls: number } = {
  calls: 0,
  trace(request) {
    this.calls++;
    return Promise.resolve(
      ok({
        debugger: {
          path: process.execPath,
          sha256: "b".repeat(64),
          version: "lldb-fixture",
        },
        run: {
          status: "traced",
          version: "lldb-fixture",
          pid: 99,
          outcome: "exited",
          exit_status: 0,
          exit_description: null,
          killed: false,
          terminated: true,
          target_identity: {
            loaded_image_sha256: null,
            selected_file_sha256: request.expectedSha256,
            file_device: "1",
            file_inode: "2",
            module_path: request.executable,
            module_uuid: "fixture-uuid",
            stable: true,
          },
          elapsed_ms: 1,
          breakpoints: request.input.breakpoints.map((_, index) => ({
            index,
            location_count: 1,
            locations: [],
          })),
          events: [],
          resource_limit_reached: false,
          target_output: {
            stdout_bytes: 0,
            stderr_bytes: 0,
            stdout_truncated: false,
            stderr_truncated: false,
            stdout_complete: true,
            stderr_complete: true,
          },
          other_stops: [],
        },
        stdout: { text: "", bytes: 0, truncated: false, complete: true },
        stderr: { text: "", bytes: 0, truncated: false, complete: true },
        terminated: true,
      }),
    );
  },
  close: () => Promise.resolve(ok(null)),
};


it.each([true, false])("records native call finalization (abort=%s)", async(abort)=>{
  const controller=new AbortController();
  const path=join(await createTestTempDirectory("rea-call-cancellation-"),"Tool");
  const bytes=Buffer.alloc(32); bytes.writeUInt32LE(0xfeedfacf,0);bytes.writeUInt32LE(0x0100000c,4);bytes.writeUInt32LE(2,12);
  await writeFile(path,bytes);
  const lateTracer=new LldbCallTracer({},async(_executable,args)=>{
    const command=args.at(-1)!;
    const config=JSON.parse(await readFile(command.slice("rea_trace ".length),"utf8"));
    const completed=await tracer.trace({executable:path,architecture:"arm64",expectedSha256:config.expected_sha256,input:config});
    if(!completed.ok)throw completed.error;
    await writeFile(config.result_path,JSON.stringify(completed.value.run));
    await writeFile(config.stdout_capture_path,"");await writeFile(config.stderr_capture_path,"");
    if (abort) controller.abort();
    return {kind:"exited",targetLaunched:false,exitCode:0,output:"",targetIdentity:undefined,cleanupFailure:undefined};
  },async()=>ok({path:process.execPath,sha256:"b".repeat(64)}));
  const makeSession=()=>createTestBinarySession(new NativeMacOSProvider({},new NativeFixtureRunner(),"darwin",()=>lateTracer));
  const result=await runProviderAnalysis({readConfiguration:()=>parseConfig({}),createBinarySession:makeSession,createArtifactBinarySession:makeSession,createManagedBinarySession:makeSession},path,"observe_native_calls",{breakpoints:[{kind:"function",name:"open"}],duration_ms:1000},silentLogger,controller.signal);
  console.log("CANCEL",JSON.stringify({aborted:controller.signal.aborted,operation:(result as any).operation,process:(result as any).normalized_result?.process,coverage:(result as any).normalized_result?.coverage}));
  expect(controller.signal.aborted).toBe(abort);expect((result as any).normalized_result.process.outcome).toBe("exited");
});

The test runs with and without the finalization abort. Both runs return successful exited-process Evidence; the aborted case is the observed defect and the other case is its control. These successful probe assertions are not a restoration claim.

Expected behavior

A cancellation observed before final publication must return the typed cancelled failure. Available native observations must remain separate from successful completion and cleanup status. docs/tool-design.md, Define the contract, requires reporting cancellation. docs/mcp-contracts.md, Progress and cancellation, distinguishes cancellation from timeout and requires native call failures to retain available observations in details.partial_observation.

Impact and scope

P2. The recorded finalization interleaving turns an aborted native observation into successful complete Evidence. Real LLDB frequency and SIGINT timing are unknown. No supported setting was verified to prevent this race or retain the observations under typed cancellation.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Synthetic 32-byte arm64 Mach-O header; function breakpoint on open; duration_ms: 1000. The production tracer consumes a schema-valid exited run and empty stdout/stderr captures supplied by the injected launch callback.

Interface

CLI

Earliest failure stage

Cleanup, cancellation, or lifecycle

Relevant output or evidence

The production workflow returned this successful projection after the caller signal was aborted:

{"aborted":true,"operation":"observe_native_calls","process":{"pid":99,"outcome":"exited","exit_status":0,"terminated":true},"coverage":{"status":"complete","event_limit_reached":false,"resource_limit_reached":false,"unresolved_breakpoints":[]}}

It returned a canonical Evidence record rather than an error. The source probe reproduced this with both a directly injected completed tracer and the production tracer with an injected LLDB launch. Real-debuggee timing and SIGINT behavior remain untested; the observed defect is at the production finalization boundary.

Typed outcome or recovery expected

Run the same post-launch abort interleaving through the CLI application workflow. It must return a typed cancellation instead of complete successful Evidence, with available observations retained according to the native-call failure contract and owned cleanup completed or explicitly reported. Repeat without aborting the controller as a control; that run must continue to return successful exited-process Evidence. This must not collapse an actual timeout or cleanup failure into an unrelated error classification.

Analysis or suspected cause

Source-derived reasoning: src/native/LldbCallTracer.ts:420 reads the final result after launch completion. Lines 549 through 580 read captures and return success without checking the request signal. src/native/NativeCallObservation.ts:90 accepts the completed trace without a post-await cancellation check. The native provider and CLI application workflow preserve that success. These cited source paths are unchanged between the tested baseline and upstream main at 6fcfc5b5e0.

Additional context

The reproduction uses the existing simulated result shape in tests/boundary/mcp/nativeCallObservationMcp.test.ts and launch/config seam in tests/boundary/providers/native/lldbCallTracerFailures.test.ts. Its successful assertions record the defect and control, not a restored cancellation contract. Freshness was checked by source comparison; the recorded runtime probe was not rerun on upstream main.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4009 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `observe_native_calls` returns successful complete Evidence when its cancellation signal becomes aborted after the LLDB launch result is classified as exited but before final result consumption. The production tracer still reads its result and captured output and removes its runtime directory after that point. Neither it nor the native provider or CLI workflow checks cancellation before publishing the final success. A cancelled operation can therefore be presented as a completed observation. The recorded reproduction used the production tracer, provider, binary session, and CLI application workflow with injected LLDB launch and tool-resolution seams. No real debuggee, LLDB process, or OS signal was exercised. ### Reproduction 1. Use a focused source test alongside the native adapter, with the imports below. This is the executed production-boundary reproduction. It follows the existing simulated LLDB result format in `tests/boundary/mcp/nativeCallObservationMcp.test.ts` and the launch/config seam in `tests/boundary/providers/native/lldbCallTracerFailures.test.ts`. 2. Materialize the 32-byte arm64 Mach-O header shown in the test. Supply the `observe_native_calls` input shown below to `runProviderAnalysis`, the shared workflow called by the registered `observe-native-calls` CLI command. 3. The fake LLDB launch writes a schema-valid completed run and both capture files, aborts the supplied controller, then returns an exited launch result. This fixes the timing at the post-launch finalization boundary while retaining the production tracer's subsequent reads and cleanup. Run the focused test with `npm run test:local` and its selected file path. ```ts import { expect, it } from "vitest"; import { writeFile, readFile } from "node:fs/promises"; import { join } from "node:path"; import { NativeMacOSProvider } from "./NativeMacOSProvider.js"; import { LldbCallTracer, type NativeCallTracer } from "./LldbCallTracer.js"; import { ok } from "../domain/result.js"; import { createTestBinarySession } from "../../tests/fixtures/binarySession.js"; import { NativeFixtureRunner } from "../../tests/fixtures/nativeCommands.js"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { runProviderAnalysis } from "../application/DirectAnalysis.js"; import { parseConfig } from "../config/parseConfig.js"; import { silentLogger } from "../logger.js"; const tracer: NativeCallTracer & { calls: number } = { calls: 0, trace(request) { this.calls++; return Promise.resolve( ok({ debugger: { path: process.execPath, sha256: "b".repeat(64), version: "lldb-fixture", }, run: { status: "traced", version: "lldb-fixture", pid: 99, outcome: "exited", exit_status: 0, exit_description: null, killed: false, terminated: true, target_identity: { loaded_image_sha256: null, selected_file_sha256: request.expectedSha256, file_device: "1", file_inode: "2", module_path: request.executable, module_uuid: "fixture-uuid", stable: true, }, elapsed_ms: 1, breakpoints: request.input.breakpoints.map((_, index) => ({ index, location_count: 1, locations: [], })), events: [], resource_limit_reached: false, target_output: { stdout_bytes: 0, stderr_bytes: 0, stdout_truncated: false, stderr_truncated: false, stdout_complete: true, stderr_complete: true, }, other_stops: [], }, stdout: { text: "", bytes: 0, truncated: false, complete: true }, stderr: { text: "", bytes: 0, truncated: false, complete: true }, terminated: true, }), ); }, close: () => Promise.resolve(ok(null)), }; it.each([true, false])("records native call finalization (abort=%s)", async(abort)=>{ const controller=new AbortController(); const path=join(await createTestTempDirectory("rea-call-cancellation-"),"Tool"); const bytes=Buffer.alloc(32); bytes.writeUInt32LE(0xfeedfacf,0);bytes.writeUInt32LE(0x0100000c,4);bytes.writeUInt32LE(2,12); await writeFile(path,bytes); const lateTracer=new LldbCallTracer({},async(_executable,args)=>{ const command=args.at(-1)!; const config=JSON.parse(await readFile(command.slice("rea_trace ".length),"utf8")); const completed=await tracer.trace({executable:path,architecture:"arm64",expectedSha256:config.expected_sha256,input:config}); if(!completed.ok)throw completed.error; await writeFile(config.result_path,JSON.stringify(completed.value.run)); await writeFile(config.stdout_capture_path,"");await writeFile(config.stderr_capture_path,""); if (abort) controller.abort(); return {kind:"exited",targetLaunched:false,exitCode:0,output:"",targetIdentity:undefined,cleanupFailure:undefined}; },async()=>ok({path:process.execPath,sha256:"b".repeat(64)})); const makeSession=()=>createTestBinarySession(new NativeMacOSProvider({},new NativeFixtureRunner(),"darwin",()=>lateTracer)); const result=await runProviderAnalysis({readConfiguration:()=>parseConfig({}),createBinarySession:makeSession,createArtifactBinarySession:makeSession,createManagedBinarySession:makeSession},path,"observe_native_calls",{breakpoints:[{kind:"function",name:"open"}],duration_ms:1000},silentLogger,controller.signal); console.log("CANCEL",JSON.stringify({aborted:controller.signal.aborted,operation:(result as any).operation,process:(result as any).normalized_result?.process,coverage:(result as any).normalized_result?.coverage})); expect(controller.signal.aborted).toBe(abort);expect((result as any).normalized_result.process.outcome).toBe("exited"); }); ``` The test runs with and without the finalization abort. Both runs return successful exited-process Evidence; the aborted case is the observed defect and the other case is its control. These successful probe assertions are not a restoration claim. ### Expected behavior A cancellation observed before final publication must return the typed `cancelled` failure. Available native observations must remain separate from successful completion and cleanup status. `docs/tool-design.md`, Define the contract, requires reporting cancellation. `docs/mcp-contracts.md`, Progress and cancellation, distinguishes cancellation from timeout and requires native call failures to retain available observations in `details.partial_observation`. ### Impact and scope P2. The recorded finalization interleaving turns an aborted native observation into successful complete Evidence. Real LLDB frequency and SIGINT timing are unknown. No supported setting was verified to prevent this race or retain the observations under typed cancellation. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Synthetic 32-byte arm64 Mach-O header; function breakpoint on open; duration_ms: 1000. The production tracer consumes a schema-valid exited run and empty stdout/stderr captures supplied by the injected launch callback. ### Interface CLI ### Earliest failure stage Cleanup, cancellation, or lifecycle ### Relevant output or evidence The production workflow returned this successful projection after the caller signal was aborted: ```json {"aborted":true,"operation":"observe_native_calls","process":{"pid":99,"outcome":"exited","exit_status":0,"terminated":true},"coverage":{"status":"complete","event_limit_reached":false,"resource_limit_reached":false,"unresolved_breakpoints":[]}} ``` It returned a canonical Evidence record rather than an error. The source probe reproduced this with both a directly injected completed tracer and the production tracer with an injected LLDB launch. Real-debuggee timing and SIGINT behavior remain untested; the observed defect is at the production finalization boundary. ### Typed outcome or recovery expected Run the same post-launch abort interleaving through the CLI application workflow. It must return a typed cancellation instead of complete successful Evidence, with available observations retained according to the native-call failure contract and owned cleanup completed or explicitly reported. Repeat without aborting the controller as a control; that run must continue to return successful exited-process Evidence. This must not collapse an actual timeout or cleanup failure into an unrelated error classification. ### Analysis or suspected cause Source-derived reasoning: `src/native/LldbCallTracer.ts:420` reads the final result after launch completion. Lines 549 through 580 read captures and return success without checking the request signal. `src/native/NativeCallObservation.ts:90` accepts the completed trace without a post-await cancellation check. The native provider and CLI application workflow preserve that success. These cited source paths are unchanged between the tested baseline and upstream main at 6fcfc5b5e0c063300cd5278dd8bad5b1ab1374f6. ### Additional context The reproduction uses the existing simulated result shape in `tests/boundary/mcp/nativeCallObservationMcp.test.ts` and launch/config seam in `tests/boundary/providers/native/lldbCallTracerFailures.test.ts`. Its successful assertions record the defect and control, not a restored cancellation contract. Freshness was checked by source comparison; the recorded runtime probe was not rerun on upstream main.

bsakaguchi closed this issue

2026-10-16 21:07:40 +00:00

bsakaguchi commented

2026-10-16 21:07:41 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Confirmed — a cancelled observe_native_calls run can still publish successful Evidence because the cancellation signal isn't re-checked after the LLDB launch is classified as exited but before the final result is consumed. The fix should check the controller's cancellation at that finalization boundary (and after cleanup) before publishing success, mirroring the pattern used in other post-cancellation evidence paths.

@XinwangTao commented on GitHub (Oct 15, 2026): Confirmed — a cancelled observe_native_calls run can still publish successful Evidence because the cancellation signal isn't re-checked after the LLDB launch is classified as exited but before the final result is consumed. The fix should check the controller's cancellation at that finalization boundary (and after cleanup) before publishing success, mirroring the pattern used in other post-cancellation evidence paths.

bsakaguchi commented

2026-10-16 21:07:41 +00:00

Author

Owner

@yankikucuk commented on GitHub (Oct 15, 2026):

Working on this. Reproduced at the production tracer boundary (fake LLDB launch that writes a traced result, aborts the request and returns exited) and at the workflow boundary (completing tracer that aborts the request): both published success. The tracer now treats an aborted request as cancellation once the launch has exited, after owned cleanup and with the completed run retained as partial_observation (reason cancelled; a fired deadline stays a timeout), and observeNativeCalls applies the same check after the tracer returns. Non-aborting controls still succeed. Draft PR follows shortly.

@yankikucuk commented on GitHub (Oct 15, 2026): Working on this. Reproduced at the production tracer boundary (fake LLDB launch that writes a traced result, aborts the request and returns exited) and at the workflow boundary (completing tracer that aborts the request): both published success. The tracer now treats an aborted request as cancellation once the launch has exited, after owned cleanup and with the completed run retained as `partial_observation` (reason `cancelled`; a fired deadline stays a timeout), and `observeNativeCalls` applies the same check after the tracer returns. Non-aborting controls still succeed. Draft PR follows shortly.

bsakaguchi commented

2026-10-16 21:07:41 +00:00

Author

Owner

@morluto commented on GitHub (Oct 15, 2026):

Local repair now checks cancellation at the producer's final completion boundary after runtime-root cleanup, so both result consumption and awaited temporary-root removal are covered. Successful collected trace data becomes typed cancellation with a partial observation; existing failures keep their own classification and cleanup precedence.

Production LldbCallTracer boundary regressions cover abort after the launch result and abort during actual runtime-root close, with retained coverage.reason: cancelled. The cancellation repair is separate from denied-launch ownership #4232. The real CLI/stdio MCP verify:native-calls lane also passes on Apple LLDB lldb-2100.0.17.108, including normal exit, event/duration limits and hardened-runtime denial/entitlement controls. The deterministic finalization-abort regressions use the launch seam, not a claimed real-debuggee SIGINT timing reproduction.

Patch is local, uncommitted and unreleased; no global server replacement is claimed.

@morluto commented on GitHub (Oct 15, 2026): Local repair now checks cancellation at the producer's final completion boundary **after runtime-root cleanup**, so both result consumption and awaited temporary-root removal are covered. Successful collected trace data becomes typed cancellation with a partial observation; existing failures keep their own classification and cleanup precedence. Production `LldbCallTracer` boundary regressions cover abort after the launch result and abort during actual runtime-root close, with retained `coverage.reason: cancelled`. The cancellation repair is separate from denied-launch ownership #4232. The real CLI/stdio MCP `verify:native-calls` lane also passes on Apple LLDB `lldb-2100.0.17.108`, including normal exit, event/duration limits and hardened-runtime denial/entitlement controls. The deterministic finalization-abort regressions use the launch seam, not a claimed real-debuggee SIGINT timing reproduction. Patch is local, uncommitted and unreleased; no global server replacement is claimed.

bsakaguchi commented

2026-10-16 21:07:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 16, 2026):

Fixed on main in 0e1f0bf2d. Native call tracing checks cancellation after final runtime cleanup and retains completed trace observations under typed cancellation.

Controlled launch/final-close timing regressions cover the reported window; this does not claim a real-debuggee SIGINT timing reproduction.

Hosted main CI is still pending; this closure is based on the implemented fix and scoped regression evidence.

@morluto commented on GitHub (Oct 16, 2026): Fixed on `main` in [0e1f0bf2d](https://github.com/morluto/rea/commit/0e1f0bf2d06943e59253b8a66923a1ae6005e81e). Native call tracing checks cancellation after final runtime cleanup and retains completed trace observations under typed cancellation. Controlled launch/final-close timing regressions cover the reported window; this does not claim a real-debuggee SIGINT timing reproduction. Hosted main CI is still pending; this closure is based on the implemented fix and scoped regression evidence.

bsakaguchi commented

2026-10-16 21:07:42 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 16, 2026):

Thanks for folding this into the same lifecycle fix — the post-launch finalization window is exactly where observe_native_calls could slip through as complete despite an aborted controller. Rechecking the signal before publishing success closes that gap for the native adapter too. Appreciate keeping the captured output reads and runtime-directory removal intact while fixing the result projection.

@XinwangTao commented on GitHub (Oct 16, 2026): Thanks for folding this into the same lifecycle fix — the post-launch finalization window is exactly where observe_native_calls could slip through as complete despite an aborted controller. Rechecking the signal before publishing success closes that gap for the native adapter too. Appreciate keeping the captured output reads and runtime-directory removal intact while fixing the result projection.

bsakaguchi commented

2026-10-16 21:07:42 +00:00

Author

Owner

@musabbiralamalam52-beep commented on GitHub (Oct 16, 2026):

Learning

@musabbiralamalam52-beep commented on GitHub (Oct 16, 2026): Learning

bsakaguchi referenced this issue

2026-10-17 21:12:39 +00:00

[PR #1593] [CLOSED] fix(plist): report binary plist __proto__ keys that results omit #2993

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1594

No description provided.

Rows
Columns