morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6056] [Bug] LLDB prelaunch failures discard unresolved cleanup state and ownership #3701

New issue

Closed

opened 2026-10-31 00:43:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:43:11 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6056

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

LLDB prelaunch failure handling discards unresolved cleanup state from two producer outcomes:

If ownership inspection fails before LLDB is spawned, the production launch handler returns an “exited” outcome. The resulting ProviderAdapterError has cleanupIncomplete: false even when the preparation error carries an unresolved helper-root cleanup marker. It reports LLDB exited without a usable tracer result with exit_code: null and loses the structured compiler outcome/output.
If LLDB discovery returns a NativeCommandFailure with an owned resolver-process supervisor and incomplete cleanup, the tracer replaces it with a new capability error. The result again has cleanupIncomplete: false; tracer close returns success without retrying the supervisor, and another trace re-enters discovery without settling that owner.

Both were executed at the production tracer boundary using controlled failures. No real LLDB or xcrun process-cleanup refusal was induced. These are two cases of one prelaunch cleanup-state transfer defect, before the tracer's current postlaunch retained-resource state can cover the producer.

Reproduction

In the prepared source checkout, save this as tests/boundary/providers/native/lldbPrelaunchCleanup.repro.test.ts and run:

node node_modules/vitest/vitest.mjs run tests/boundary/providers/native/lldbPrelaunchCleanup.repro.test.ts --maxWorkers 1

The assertions describe the observed incorrect behavior. The launch case uses the default production LLDB runner and replaces only the shared spawn boundary's rejected result. The discovery case uses the tracer's existing resolver seam with a real supervisor around an inert simulated process handle. No target is launched.

import { afterEach, expect, it, vi } from 'vitest';
import { EventEmitter } from 'node:events';
import { PassThrough } from 'node:stream';
import { LldbCallTracer } from '../../../../src/native/LldbCallTracer.js';
import { NativeCommandFailure } from '../../../../src/native/CommandRunner.js';
import { DarwinProcessOwnershipInspectionError } from '../../../../src/process/DarwinProcessRunTokenReader.js';
import * as processBoundary from '../../../../src/process/ProviderProcess.js';
import { ProviderCleanupError } from '../../../../src/domain/providerCleanupError.js';
import { nativeCallObservationInputSchema } from '../../../../src/domain/native/nativeCallObservation.js';
import { err, ok } from '../../../../src/domain/result.js';

const tool = { path: '/usr/bin/true', sha256: 'b'.repeat(64) };
const request = {
  executable: '/fixture', architecture: 'arm64' as const,
  expectedSha256: 'a'.repeat(64),
  input: nativeCallObservationInputSchema.parse({ breakpoints: [{kind:'function', name:'main'}] }),
};
function preparation() {
  const marker = new ProviderCleanupError('process-ownership', ['owned-helper-root'], {reason:'controlled root deletion failure'});
  const close = vi.fn(async () => { marker.resolveCleanup(); });
  const failure = new DarwinProcessOwnershipInspectionError('controlled compiler failure', {
    cleanupFailure: marker, cleanupOwner: {close},
    compilerOutcome: {outcome:'exited', code:1, signal:null, stderr:'compiler failed', stderrTruncated:false, outputTruncated:false, killed:false},
    capturedOutput: {stderr:'compiler failed', stderr_bytes:15, truncated:false},
  });
  return {marker, close, failure};
}
afterEach(() => vi.restoreAllMocks());

it('launch preparation failure: default LLDB runner loses helper owner', async () => {
  const p = preparation();
  const launch = vi.spyOn(processBoundary, 'spawnOwnedProviderProcess').mockRejectedValue(p.failure);
  const tracer = new LldbCallTracer({}, undefined, async () => ok(tool));
  try {
    const result = await tracer.trace(request);
    expect(launch).toHaveBeenCalledTimes(1);
    expect(result.ok).toBe(false);
    if (result.ok) throw new Error('expected failure');
    console.log('LAUNCH_PREPARATION', JSON.stringify({tag:result.error._tag, cleanupIncomplete:result.error.cleanupIncomplete, reason:result.error.diagnostics?.reason}));
    expect(result.error.diagnostics).toMatchObject({reason:'LLDB exited without a usable tracer result', exit_code:null, lldb_output:'controlled compiler failure'});
    expect(result.error.cleanupIncomplete).toBe(false);
    expect((await tracer.close()).ok).toBe(true);
    expect(p.close).not.toHaveBeenCalled();
    expect(p.marker.cleanupIncomplete).toBe(true);
  } finally {
    await tracer.close();
    await p.close();
  }
});

it('tool resolution failure: tracer drops process supervisor and marker', async () => {
  const child = Object.assign(new EventEmitter(), {stdout:new PassThrough(), stderr:new PassThrough(), exitCode:null, signalCode:null, kill:()=>true});
  const supervisor = new processBoundary.ProviderProcessSupervisor({
    process:child, ownsProcessLifetime:true,
    ownership:{runId:'controlled-resolver',leaderPid:46000,processGroupId:46000,expectedParentPid:process.pid},
    cleanup:async () => ({cleaned:false,reason:'controlled incomplete stop'}),
  });
  const stop = vi.spyOn(supervisor,'stop');
  const marker = new ProviderCleanupError('native-macos',['owned-resolver-process'],{reason:'controlled incomplete stop'});
  const resolver = vi.fn(async () => err(new NativeCommandFailure('lldb','io',null,{cleanupOwner:supervisor,cleanupMarker:marker},null,'controlled incomplete stop',['owned-resolver-process'])));
  const tracer = new LldbCallTracer({}, undefined, resolver);
  try {
    const result = await tracer.trace(request);
    expect(result.ok).toBe(false);
    if(result.ok) throw new Error('expected failure');
    console.log('RESOLVER_CLEANUP',JSON.stringify({tag:result.error._tag,cleanupIncomplete:result.error.cleanupIncomplete}));
    expect(result.error.cleanupIncomplete).toBe(false);
    expect((await tracer.close()).ok).toBe(true);
    expect(stop).not.toHaveBeenCalled();
    await tracer.trace(request);
    expect(resolver).toHaveBeenCalledTimes(2);
    expect(stop).not.toHaveBeenCalled();
  } finally {
    supervisor.dispose();
    child.stdout.destroy(); child.stderr.destroy();
    await tracer.close();
  }
});

Both LLDB cases passed these current-behavior assertions in the focused run. All supplied fake owners/process streams were explicitly cleaned up by the test.

Expected behavior

Keep the producer's unresolved cleanup observation visible in the returned error. A failure before spawning must retain preparation provenance and compiler facts rather than invent an LLDB exit. Retain tracer-owned resolver supervisors for cleanup retry through close and before another trace is admitted.

Ownership must remain precise: the Darwin helper root belongs to the shared reader. Preserving its live marker does not require closing the whole reader from an LLDB client. The existing native command path correctly keeps that foreign marker unresolved through client close and follows its reader's later confirmed release.

The lifecycle contract distinguishes preparation from process exit, and the cleanup ownership contract requires live, exact-owner cleanup state and retry before replacement work.

Impact and scope

P2: failed native-call observations can falsely report complete cleanup, hide the helper preparation stage, and abandon the tracer's retry path for an unresolved resolver-process owner. A second controlled trace was admitted without attempting that owner's stop. No surviving real process, persistent root leak, target corruption, or security boundary crossing was demonstrated; natural incidence is unknown.

The shared Darwin reader still retains its failed root and retries root cleanup before another compilation; it also has best-effort exit cleanup. Therefore this report does not claim a permanently orphaned helper root or that a native command client should take over the shared reader. Actual LLDB/Xcode behavior and CLI/MCP transport timing remain untested.

REA version

rea-agents 6.7.0, source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64, Node.js v24.18.0, repository-locked Vitest 5.0.3. Controlled producer failures and a simulated resolver process; no external debugger.

Target and artifact context

An inert synthetic native-call request and fake resolved tool identity. Failure occurs before LLDB or a target is launched; no binary is modified.

Interface

Other

Earliest failure stage

Cleanup, cancellation, or lifecycle

Relevant output or evidence

Reduced executed observations:

launch preparation source marker: cleanupIncomplete=true
returned launch error: ProviderAdapterError, cleanupIncomplete=false
launch diagnostics: LLDB exited without a usable tracer result; exit_code=null
tracer.close(): success; supplied helper close calls=0

discovery source: NativeCommandFailure with supervisor and cleanup marker
returned discovery error: AnalysisCapabilityUnavailableError, cleanupIncomplete=false
tracer.close(): success; supervisor.stop calls=0
second trace: resolver calls=2; supervisor.stop calls=0

The default process runner catch converts preparation rejection to exit, while finish derives cleanup only from a supervisor created after successful spawn. The discovery branch reconstructs an unrelated error. The resolver producer already forwards its cleanup supervisor and facts.

Public reachability is source-derived: the CLI dispatches observe_native_calls; the provider invokes the shared observation workflow, which calls this tracer. The focused execution entered at the tracer boundary, without CLI or MCP transport.

Typed outcome or recovery expected

A preparation-stage error preserving its compiler facts and unresolved shared-helper marker; an error bound to the tracer-owned resolver process while that process remains unresolved. Close and preflight retry must retry owned resolver cleanup, retain failed retries, and resolve only the exact owner that confirms release. Foreign helper state should follow its reader-owned marker.

Analysis or suspected cause

The tracer only retains postlaunch LLDB resources and retries that pending resource set. Discovery returns before creating it, while a preparation rejection occurs before a process supervisor exists. Both paths drop the typed producer's live cleanup state. A shared prelaunch failure/cleanup bridge at the tracer boundary could cover both outcomes while keeping shared-reader ownership distinct from tracer-owned command supervisors.

Additional context

#6037 covers discovery cancellation-cause loss and #6040 covers discovery timeout/I/O classification; neither reproduces cleanup-owner retention, and those classification claims are not new findings here. #5913 and its cleanup comment concern completed trace facts after launch. #5287/#5297 concern native command cancellation and stale state after an existing owner retry; #4522/#4526 concern compiler producer/output handling. These do not cover the LLDB prelaunch cleanup consumer. The fixed #3185 concerns false unresolved target cleanup after LLDB positively reports a pre-target integrity rejection; this report concerns real unresolved producer cleanup state lost before LLDB is spawned or discovery completes.

The existing foreign-owner regression was also executed: both ordinary and late-cancellation cases passed. It verifies that native client close leaves foreign helper cleanup unresolved and reader close subsequently resolves the same error. This is why adopting the shared reader into every native command client is outside the expected repair.

Fetched upstream main 1f92c773989585a7c0f3255b7a9264789394908c leaves the relevant tracer, command runner, native provider, owned-command and Darwin reader files unchanged relative to the executed revision. The newer revision was source-compared, not separately executed.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6056 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior LLDB prelaunch failure handling discards unresolved cleanup state from two producer outcomes: 1. If ownership inspection fails before LLDB is spawned, the production launch handler returns an “exited” outcome. The resulting `ProviderAdapterError` has `cleanupIncomplete: false` even when the preparation error carries an unresolved helper-root cleanup marker. It reports `LLDB exited without a usable tracer result` with `exit_code: null` and loses the structured compiler outcome/output. 2. If LLDB discovery returns a `NativeCommandFailure` with an owned resolver-process supervisor and incomplete cleanup, the tracer replaces it with a new capability error. The result again has `cleanupIncomplete: false`; tracer close returns success without retrying the supervisor, and another trace re-enters discovery without settling that owner. Both were executed at the production tracer boundary using controlled failures. No real LLDB or xcrun process-cleanup refusal was induced. These are two cases of one prelaunch cleanup-state transfer defect, before the tracer's current postlaunch retained-resource state can cover the producer. ### Reproduction In the prepared source checkout, save this as `tests/boundary/providers/native/lldbPrelaunchCleanup.repro.test.ts` and run: ```sh node node_modules/vitest/vitest.mjs run tests/boundary/providers/native/lldbPrelaunchCleanup.repro.test.ts --maxWorkers 1 ``` The assertions describe the observed incorrect behavior. The launch case uses the default production LLDB runner and replaces only the shared spawn boundary's rejected result. The discovery case uses the tracer's existing resolver seam with a real supervisor around an inert simulated process handle. No target is launched. ```ts import { afterEach, expect, it, vi } from 'vitest'; import { EventEmitter } from 'node:events'; import { PassThrough } from 'node:stream'; import { LldbCallTracer } from '../../../../src/native/LldbCallTracer.js'; import { NativeCommandFailure } from '../../../../src/native/CommandRunner.js'; import { DarwinProcessOwnershipInspectionError } from '../../../../src/process/DarwinProcessRunTokenReader.js'; import * as processBoundary from '../../../../src/process/ProviderProcess.js'; import { ProviderCleanupError } from '../../../../src/domain/providerCleanupError.js'; import { nativeCallObservationInputSchema } from '../../../../src/domain/native/nativeCallObservation.js'; import { err, ok } from '../../../../src/domain/result.js'; const tool = { path: '/usr/bin/true', sha256: 'b'.repeat(64) }; const request = { executable: '/fixture', architecture: 'arm64' as const, expectedSha256: 'a'.repeat(64), input: nativeCallObservationInputSchema.parse({ breakpoints: [{kind:'function', name:'main'}] }), }; function preparation() { const marker = new ProviderCleanupError('process-ownership', ['owned-helper-root'], {reason:'controlled root deletion failure'}); const close = vi.fn(async () => { marker.resolveCleanup(); }); const failure = new DarwinProcessOwnershipInspectionError('controlled compiler failure', { cleanupFailure: marker, cleanupOwner: {close}, compilerOutcome: {outcome:'exited', code:1, signal:null, stderr:'compiler failed', stderrTruncated:false, outputTruncated:false, killed:false}, capturedOutput: {stderr:'compiler failed', stderr_bytes:15, truncated:false}, }); return {marker, close, failure}; } afterEach(() => vi.restoreAllMocks()); it('launch preparation failure: default LLDB runner loses helper owner', async () => { const p = preparation(); const launch = vi.spyOn(processBoundary, 'spawnOwnedProviderProcess').mockRejectedValue(p.failure); const tracer = new LldbCallTracer({}, undefined, async () => ok(tool)); try { const result = await tracer.trace(request); expect(launch).toHaveBeenCalledTimes(1); expect(result.ok).toBe(false); if (result.ok) throw new Error('expected failure'); console.log('LAUNCH_PREPARATION', JSON.stringify({tag:result.error._tag, cleanupIncomplete:result.error.cleanupIncomplete, reason:result.error.diagnostics?.reason})); expect(result.error.diagnostics).toMatchObject({reason:'LLDB exited without a usable tracer result', exit_code:null, lldb_output:'controlled compiler failure'}); expect(result.error.cleanupIncomplete).toBe(false); expect((await tracer.close()).ok).toBe(true); expect(p.close).not.toHaveBeenCalled(); expect(p.marker.cleanupIncomplete).toBe(true); } finally { await tracer.close(); await p.close(); } }); it('tool resolution failure: tracer drops process supervisor and marker', async () => { const child = Object.assign(new EventEmitter(), {stdout:new PassThrough(), stderr:new PassThrough(), exitCode:null, signalCode:null, kill:()=>true}); const supervisor = new processBoundary.ProviderProcessSupervisor({ process:child, ownsProcessLifetime:true, ownership:{runId:'controlled-resolver',leaderPid:46000,processGroupId:46000,expectedParentPid:process.pid}, cleanup:async () => ({cleaned:false,reason:'controlled incomplete stop'}), }); const stop = vi.spyOn(supervisor,'stop'); const marker = new ProviderCleanupError('native-macos',['owned-resolver-process'],{reason:'controlled incomplete stop'}); const resolver = vi.fn(async () => err(new NativeCommandFailure('lldb','io',null,{cleanupOwner:supervisor,cleanupMarker:marker},null,'controlled incomplete stop',['owned-resolver-process']))); const tracer = new LldbCallTracer({}, undefined, resolver); try { const result = await tracer.trace(request); expect(result.ok).toBe(false); if(result.ok) throw new Error('expected failure'); console.log('RESOLVER_CLEANUP',JSON.stringify({tag:result.error._tag,cleanupIncomplete:result.error.cleanupIncomplete})); expect(result.error.cleanupIncomplete).toBe(false); expect((await tracer.close()).ok).toBe(true); expect(stop).not.toHaveBeenCalled(); await tracer.trace(request); expect(resolver).toHaveBeenCalledTimes(2); expect(stop).not.toHaveBeenCalled(); } finally { supervisor.dispose(); child.stdout.destroy(); child.stderr.destroy(); await tracer.close(); } }); ``` Both LLDB cases passed these current-behavior assertions in the focused run. All supplied fake owners/process streams were explicitly cleaned up by the test. ### Expected behavior Keep the producer's unresolved cleanup observation visible in the returned error. A failure before spawning must retain preparation provenance and compiler facts rather than invent an LLDB exit. Retain tracer-owned resolver supervisors for cleanup retry through close and before another trace is admitted. Ownership must remain precise: the Darwin helper root belongs to the shared reader. Preserving its live marker does not require closing the whole reader from an LLDB client. The existing native command path correctly keeps that foreign marker unresolved through client close and follows its reader's later confirmed release. The [lifecycle contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/tool-design.md#L78-L81) distinguishes preparation from process exit, and the [cleanup ownership contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/tool-design.md#L119-L147) requires live, exact-owner cleanup state and retry before replacement work. ### Impact and scope P2: failed native-call observations can falsely report complete cleanup, hide the helper preparation stage, and abandon the tracer's retry path for an unresolved resolver-process owner. A second controlled trace was admitted without attempting that owner's stop. No surviving real process, persistent root leak, target corruption, or security boundary crossing was demonstrated; natural incidence is unknown. The shared Darwin reader still retains its failed root and retries root cleanup before another compilation; it also has best-effort exit cleanup. Therefore this report does not claim a permanently orphaned helper root or that a native command client should take over the shared reader. Actual LLDB/Xcode behavior and CLI/MCP transport timing remain untested. ### REA version rea-agents 6.7.0, source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64, Node.js v24.18.0, repository-locked Vitest 5.0.3. Controlled producer failures and a simulated resolver process; no external debugger. ### Target and artifact context An inert synthetic native-call request and fake resolved tool identity. Failure occurs before LLDB or a target is launched; no binary is modified. ### Interface Other ### Earliest failure stage Cleanup, cancellation, or lifecycle ### Relevant output or evidence Reduced executed observations: ```text launch preparation source marker: cleanupIncomplete=true returned launch error: ProviderAdapterError, cleanupIncomplete=false launch diagnostics: LLDB exited without a usable tracer result; exit_code=null tracer.close(): success; supplied helper close calls=0 discovery source: NativeCommandFailure with supervisor and cleanup marker returned discovery error: AnalysisCapabilityUnavailableError, cleanupIncomplete=false tracer.close(): success; supervisor.stop calls=0 second trace: resolver calls=2; supervisor.stop calls=0 ``` The [default process runner catch](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/LldbCallTracer.ts#L1219-L1234) converts preparation rejection to exit, while [finish](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/LldbCallTracer.ts#L1121-L1146) derives cleanup only from a supervisor created after successful spawn. The [discovery branch](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/LldbCallTracer.ts#L213-L224) reconstructs an unrelated error. The [resolver producer](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/CommandRunner.ts#L361-L407) already forwards its cleanup supervisor and facts. Public reachability is source-derived: [the CLI](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/cli/nativeCallCommands.ts#L16-L70) dispatches `observe_native_calls`; [the provider](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/NativeMacOSProvider.ts#L316-L333) invokes [the shared observation workflow](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/NativeCallObservation.ts#L54-L88), which calls this tracer. The focused execution entered at the tracer boundary, without CLI or MCP transport. ### Typed outcome or recovery expected A preparation-stage error preserving its compiler facts and unresolved shared-helper marker; an error bound to the tracer-owned resolver process while that process remains unresolved. Close and preflight retry must retry owned resolver cleanup, retain failed retries, and resolve only the exact owner that confirms release. Foreign helper state should follow its reader-owned marker. ### Analysis or suspected cause The tracer only retains [postlaunch LLDB resources](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/LldbCallTracer.ts#L1005-L1015) and retries [that pending resource set](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/native/LldbCallTracer.ts#L381-L437). Discovery returns before creating it, while a preparation rejection occurs before a process supervisor exists. Both paths drop the typed producer's live cleanup state. A shared prelaunch failure/cleanup bridge at the tracer boundary could cover both outcomes while keeping shared-reader ownership distinct from tracer-owned command supervisors. ### Additional context [#6037](https://github.com/morluto/rea/issues/6037) covers discovery cancellation-cause loss and [#6040](https://github.com/morluto/rea/issues/6040) covers discovery timeout/I/O classification; neither reproduces cleanup-owner retention, and those classification claims are not new findings here. [#5913](https://github.com/morluto/rea/issues/5913) and its cleanup comment concern completed trace facts after launch. #5287/#5297 concern native command cancellation and stale state after an existing owner retry; #4522/#4526 concern compiler producer/output handling. These do not cover the LLDB prelaunch cleanup consumer. The fixed [#3185](https://github.com/morluto/rea/issues/3185) concerns false unresolved target cleanup after LLDB positively reports a pre-target integrity rejection; this report concerns real unresolved producer cleanup state lost before LLDB is spawned or discovery completes. The [existing foreign-owner regression](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/tests/process-global/darwinCompilerCleanup.test.ts#L293-L341) was also executed: both ordinary and late-cancellation cases passed. It verifies that native client close leaves foreign helper cleanup unresolved and reader close subsequently resolves the same error. This is why adopting the shared reader into every native command client is outside the expected repair. Fetched upstream main `1f92c773989585a7c0f3255b7a9264789394908c` leaves the relevant tracer, command runner, native provider, owned-command and Darwin reader files unchanged relative to the executed revision. The newer revision was source-compared, not separately executed.

bsakaguchi closed this issue

2026-10-31 00:43:11 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471