morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3185] [Bug] LLDB pre-launch integrity rejection leaves cleanup stuck and blocks tracer reuse #1124

New issue

Closed

opened 2026-10-15 18:31:17 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:17 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3185

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

When the selected executable changes after REA's host-side digest validation but before LLDB creates its target, observe-native-calls correctly rejects the changed file, then reports unresolved target cleanup and leaves its private runtime directory behind. The public CLI exits 1 with code: "cleanup_incomplete" and category: "integrity_mismatch". Its details preserve execution_failure: "evidence_integrity_mismatch" and the digest-change reason, but also report native-target:unknown, the missing-start-identity cleanup reason, and an additional shutdown cleanup_error.code: "cleanup_incomplete". The retained directory contains only config.json and result.json, with no PID file.

An independent verifier reproduced this through the public CLI and real LLDB with both a copied system Mach-O and a freshly compiled C fixture. A supplementary probe using the production tracer with its default launcher and resolver reproduced three failed close() attempts. After restoring the target, another observation on that same tracer returned the stale cleanup failure before starting another run. A fresh tracer did not inherit the pending owner and closed successfully. Its observation encountered a separate launch/environment failure, so that control does not establish successful native tracing on this host. The same-instance result is a production-tracer probe, not an MCP-session reproduction.

Reproduction

Use a clean macOS checkout at 58300885207133422af235e0f2b10658cf958b03. From its root, prepare the normal built CLI. These installation/build commands are clean-checkout prerequisites; local verification used the already prepared baseline without reinstalling or rebuilding it.

npm ci
npm run build:cached

Create a disposable working directory and the following driver. It compiles an inline C fixture, invokes the public observation command, and changes only that fixture when REA publishes its LLDB runtime configuration. HOME and all XDG directories used by the driver, compiler and CLI remain under the disposable directory.

export REPRO_ROOT="$(mktemp -d)"
mkdir -p "$REPRO_ROOT/home/config" "$REPRO_ROOT/home/cache" "$REPRO_ROOT/home/data"
cat > "$REPRO_ROOT/repro.mjs" <<'JS'
import { spawn, spawnSync } from 'node:child_process';
import { appendFileSync, existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';

const root = realpathSync(process.env.REPRO_ROOT);
const runtime = join(root, 'runtime');
mkdirSync(runtime, { recursive: true });
const target = join(root, 'target');
const compiled = spawnSync('xcrun', ['clang', '-x', 'c', '-', '-o', target], {
  input: 'int main(void) { return 0; }\n', encoding: 'utf8'
});
if (compiled.status !== 0) throw new Error(compiled.stderr);
const child = spawn(process.execPath, [
  'scripts/rea.mjs', 'observe-native-calls', target,
  JSON.stringify({ breakpoints: [{ kind: 'function', name: 'main' }], duration_ms: 100, max_events: 1 }),
  '--format', 'json'
], { env: { ...process.env, TMPDIR: runtime } });
let output = '', errors = '', mutated = false;
child.stdout.on('data', data => output += data);
child.stderr.on('data', data => errors += data);
const timer = setInterval(() => {
  for (const name of readdirSync(runtime)) {
    if (!name.startsWith('rea-lldb-') || mutated) continue;
    const config = join(runtime, name, 'config.json');
    try {
      if (JSON.parse(readFileSync(config, 'utf8')).executable === target) {
        appendFileSync(target, '\nchanged-after-host-digest\n');
        mutated = true;
      }
    } catch {}
  }
}, 1);
let exitCode;
try {
  exitCode = await new Promise((done, fail) => {
    child.once('error', fail);
    child.once('close', done);
  });
} finally { clearInterval(timer); }
writeFileSync(join(root, 'cli-output.json'), output);
writeFileSync(join(root, 'cli-stderr.txt'), errors);
const result = JSON.parse(output);
const retained = readdirSync(runtime).filter(name => name.startsWith('rea-lldb-')).map(name => {
  const directory = join(runtime, name);
  return {
    files: readdirSync(directory),
    pidExists: existsSync(join(directory, 'pid')),
    bridge: JSON.parse(readFileSync(join(directory, 'result.json'), 'utf8'))
  };
});
console.log(JSON.stringify({
  exitCode, mutated, code: result.code, category: result.category,
  reason: result.details?.reason, cleanup: result.details?.cleanup,
  cleanupReason: result.details?.cleanup_reason,
  closeCode: result.details?.cleanup_error?.code,
  unknownTarget: result.details?.resources?.includes('native-target:unknown'),
  retained, stderr: errors
}, null, 2));
JS
HOME="$REPRO_ROOT/home" XDG_CONFIG_HOME="$REPRO_ROOT/home/config" XDG_CACHE_HOME="$REPRO_ROOT/home/cache" XDG_DATA_HOME="$REPRO_ROOT/home/data" node "$REPRO_ROOT/repro.mjs"

Check that mutated is true and the retained bridge result says selected executable digest changed before LLDB target creation. The driver's compact output should report exitCode: 1, code: "cleanup_incomplete", category: "integrity_mismatch", closeCode: "cleanup_incomplete", unknownTarget: true, and a retained directory whose only files are config.json and result.json, with pidExists: false. Raw CLI output and stderr are saved inside the disposable directory for inspection.

The mutation uses a race window. If the bridge gives a different failure, that attempt did not establish this case; use a new disposable directory and rerun. The driver canonicalizes its working directory before comparing paths. The successful observed attempts were one copied system Mach-O and one newly compiled C fixture, two failures in two correctly timed CLI mutation attempts. An earlier watcher failed to account for canonical path spelling and never mutated the input; that attempt returned an attach-permission error and cleaned its runtime instead.

Expected behavior

docs/native-investigation.md:423-446 documents the owned LLDB observation and preserves uncertainty when ownership or cleanup cannot be verified. Once LLDB has exited and the bridge positively confirms that it rejected the digest before target creation, there is no target process to wait for. REA should preserve the integrity rejection, remove its owned runtime directory during cleanup, and allow later observations through the same tracer. The owned-root and retryable-cleanup contracts are also reflected in src/process/PrivateRuntimeRoot.ts:38-43,82-91 and src/native/LldbCallTracer.ts:160,289-296. The existing no-target treatment at src/native/LldbCallTracer.ts:426-433 supports this lifecycle expectation; the documentation does not promise eventual resolution of every uncertain outcome.

A missing PID alone, a missing result, or an integrity failure after launch must continue to preserve genuine ownership and termination uncertainty.

Impact and scope

P2. A bounded local executable-change race leaks the runtime directory, produces false unresolved-target cleanup reporting, and blocks subsequent observations through the affected tracer until that instance is discarded. The changed executable is correctly rejected. No foreign-process signaling, target data loss, or security boundary violation was observed.

After restoring the target, starting a fresh REA backend avoids the stale owner, as supported by a new CLI invocation and a fresh production tracer. Already leaked directories still need removal after confirming that their recorded rejection occurred before launch; restarting does not remove them. The two successfully timed CLI mutation attempts both reproduced the defect. This sample does not establish a frequency for normal use or a failure across every operation in a REA process. File disappearance and pre-launch module-identity disagreement are source-derived variants that were not executed.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node.js v26.10.0; macOS 27.0.1 build 26A434 arm64; Apple LLDB lldb-2103.0.34.103. Xcode Command Line Tools provide xcrun clang and xcrun lldb.

Target and artifact context

A disposable, locally compiled Mach-O executable from inline C source, with a function breakpoint on main, duration_ms: 100, and max_events: 1. The driver appends bytes only to that fixture after the LLDB runtime configuration appears. Its private runtime and HOME/XDG directories remain under the disposable working directory. The retained bridge result identifies digest rejection before LLDB target creation.

Interface

CLI

Earliest failure stage

Cleanup, cancellation, or lifecycle

Relevant output or evidence

exitCode: 1
mutated: true
code: cleanup_incomplete
category: integrity_mismatch
execution_failure: evidence_integrity_mismatch
bridge status: target-integrity-error
bridge error: selected executable digest changed before LLDB target creation
cleanup_reason: target process start identity was unavailable; survivor was left untouched
cleanup_error.code: cleanup_incomplete
resources include: native-target:unknown
retained files: config.json, result.json
pidExists: false

Typed outcome or recovery expected

Keep the underlying evidence_integrity_mismatch rejection and its digest-change reason. For this positively proven pre-launch case, complete cleanup without an unresolved-target or close-time cleanup error, remove the owned runtime directory, and let repeated close() calls succeed. Restoring the fixture should allow the same tracer's next observation to reach its own launch or result instead of returning the stale cleanup failure. A discriminating post-launch case with unverified process exit must still retain cleanup uncertainty and the resources needed for a safe retry. An unmodified target must continue to clean its runtime after a normal launch failure.

Analysis or suspected cause

Source-derived reasoning at baseline 58300885207133422af235e0f2b10658cf958b03: bridge/native/rea_lldb_tracer.py:566-575 emits the reproduced target-integrity-error before CreateTarget; PID publication occurs later at bridge/native/rea_lldb_tracer.py:635-636. Host digest checks precede the tracer at src/native/NativeCallObservation.ts:48-89.

src/native/LldbCallTracer.ts:847-855 marks the target as possibly launched whenever LLDB has a supervisor. The missing PID reaches ensureTerminated(..., true), which returns unavailable identity at src/native/LldbCallTracer.ts:961-963. The pre-launch exception at src/native/LldbCallTracer.ts:423-436 recognizes target-error and launch-error, but omits the reproduced pre-CreateTarget integrity rejection. The retained owner retries that same missing-PID condition at src/native/LldbCallTracer.ts:309-350 and prevents another trace at src/native/LldbCallTracer.ts:196-197. This source chain explains the observed stuck cleanup; it does not make every integrity error proof that no process launched.

Additional context

Related to #2397, which investigates event-limit mismatches and identity failure for an already launched target with a recorded PID; this report concerns cleanup after a confirmed digest rejection before target creation, with no PID and no target process.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3185 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior When the selected executable changes after REA's host-side digest validation but before LLDB creates its target, `observe-native-calls` correctly rejects the changed file, then reports unresolved target cleanup and leaves its private runtime directory behind. The public CLI exits 1 with `code: "cleanup_incomplete"` and `category: "integrity_mismatch"`. Its details preserve `execution_failure: "evidence_integrity_mismatch"` and the digest-change reason, but also report `native-target:unknown`, the missing-start-identity cleanup reason, and an additional shutdown `cleanup_error.code: "cleanup_incomplete"`. The retained directory contains only `config.json` and `result.json`, with no PID file. An independent verifier reproduced this through the public CLI and real LLDB with both a copied system Mach-O and a freshly compiled C fixture. A supplementary probe using the production tracer with its default launcher and resolver reproduced three failed `close()` attempts. After restoring the target, another observation on that same tracer returned the stale cleanup failure before starting another run. A fresh tracer did not inherit the pending owner and closed successfully. Its observation encountered a separate launch/environment failure, so that control does not establish successful native tracing on this host. The same-instance result is a production-tracer probe, not an MCP-session reproduction. ### Reproduction 1. Use a clean macOS checkout at `58300885207133422af235e0f2b10658cf958b03`. From its root, prepare the normal built CLI. These installation/build commands are clean-checkout prerequisites; local verification used the already prepared baseline without reinstalling or rebuilding it. ```sh npm ci npm run build:cached ``` 2. Create a disposable working directory and the following driver. It compiles an inline C fixture, invokes the public observation command, and changes only that fixture when REA publishes its LLDB runtime configuration. HOME and all XDG directories used by the driver, compiler and CLI remain under the disposable directory. ```sh export REPRO_ROOT="$(mktemp -d)" mkdir -p "$REPRO_ROOT/home/config" "$REPRO_ROOT/home/cache" "$REPRO_ROOT/home/data" cat > "$REPRO_ROOT/repro.mjs" <<'JS' import { spawn, spawnSync } from 'node:child_process'; import { appendFileSync, existsSync, mkdirSync, readdirSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; import { join, resolve } from 'node:path'; const root = realpathSync(process.env.REPRO_ROOT); const runtime = join(root, 'runtime'); mkdirSync(runtime, { recursive: true }); const target = join(root, 'target'); const compiled = spawnSync('xcrun', ['clang', '-x', 'c', '-', '-o', target], { input: 'int main(void) { return 0; }\n', encoding: 'utf8' }); if (compiled.status !== 0) throw new Error(compiled.stderr); const child = spawn(process.execPath, [ 'scripts/rea.mjs', 'observe-native-calls', target, JSON.stringify({ breakpoints: [{ kind: 'function', name: 'main' }], duration_ms: 100, max_events: 1 }), '--format', 'json' ], { env: { ...process.env, TMPDIR: runtime } }); let output = '', errors = '', mutated = false; child.stdout.on('data', data => output += data); child.stderr.on('data', data => errors += data); const timer = setInterval(() => { for (const name of readdirSync(runtime)) { if (!name.startsWith('rea-lldb-') || mutated) continue; const config = join(runtime, name, 'config.json'); try { if (JSON.parse(readFileSync(config, 'utf8')).executable === target) { appendFileSync(target, '\nchanged-after-host-digest\n'); mutated = true; } } catch {} } }, 1); let exitCode; try { exitCode = await new Promise((done, fail) => { child.once('error', fail); child.once('close', done); }); } finally { clearInterval(timer); } writeFileSync(join(root, 'cli-output.json'), output); writeFileSync(join(root, 'cli-stderr.txt'), errors); const result = JSON.parse(output); const retained = readdirSync(runtime).filter(name => name.startsWith('rea-lldb-')).map(name => { const directory = join(runtime, name); return { files: readdirSync(directory), pidExists: existsSync(join(directory, 'pid')), bridge: JSON.parse(readFileSync(join(directory, 'result.json'), 'utf8')) }; }); console.log(JSON.stringify({ exitCode, mutated, code: result.code, category: result.category, reason: result.details?.reason, cleanup: result.details?.cleanup, cleanupReason: result.details?.cleanup_reason, closeCode: result.details?.cleanup_error?.code, unknownTarget: result.details?.resources?.includes('native-target:unknown'), retained, stderr: errors }, null, 2)); JS HOME="$REPRO_ROOT/home" XDG_CONFIG_HOME="$REPRO_ROOT/home/config" XDG_CACHE_HOME="$REPRO_ROOT/home/cache" XDG_DATA_HOME="$REPRO_ROOT/home/data" node "$REPRO_ROOT/repro.mjs" ``` 3. Check that `mutated` is true and the retained bridge result says `selected executable digest changed before LLDB target creation`. The driver's compact output should report `exitCode: 1`, `code: "cleanup_incomplete"`, `category: "integrity_mismatch"`, `closeCode: "cleanup_incomplete"`, `unknownTarget: true`, and a retained directory whose only files are `config.json` and `result.json`, with `pidExists: false`. Raw CLI output and stderr are saved inside the disposable directory for inspection. The mutation uses a race window. If the bridge gives a different failure, that attempt did not establish this case; use a new disposable directory and rerun. The driver canonicalizes its working directory before comparing paths. The successful observed attempts were one copied system Mach-O and one newly compiled C fixture, two failures in two correctly timed CLI mutation attempts. An earlier watcher failed to account for canonical path spelling and never mutated the input; that attempt returned an attach-permission error and cleaned its runtime instead. ### Expected behavior `docs/native-investigation.md:423-446` documents the owned LLDB observation and preserves uncertainty when ownership or cleanup cannot be verified. Once LLDB has exited and the bridge positively confirms that it rejected the digest before target creation, there is no target process to wait for. REA should preserve the integrity rejection, remove its owned runtime directory during cleanup, and allow later observations through the same tracer. The owned-root and retryable-cleanup contracts are also reflected in `src/process/PrivateRuntimeRoot.ts:38-43,82-91` and `src/native/LldbCallTracer.ts:160,289-296`. The existing no-target treatment at `src/native/LldbCallTracer.ts:426-433` supports this lifecycle expectation; the documentation does not promise eventual resolution of every uncertain outcome. A missing PID alone, a missing result, or an integrity failure after launch must continue to preserve genuine ownership and termination uncertainty. ### Impact and scope P2. A bounded local executable-change race leaks the runtime directory, produces false unresolved-target cleanup reporting, and blocks subsequent observations through the affected tracer until that instance is discarded. The changed executable is correctly rejected. No foreign-process signaling, target data loss, or security boundary violation was observed. After restoring the target, starting a fresh REA backend avoids the stale owner, as supported by a new CLI invocation and a fresh production tracer. Already leaked directories still need removal after confirming that their recorded rejection occurred before launch; restarting does not remove them. The two successfully timed CLI mutation attempts both reproduced the defect. This sample does not establish a frequency for normal use or a failure across every operation in a REA process. File disappearance and pre-launch module-identity disagreement are source-derived variants that were not executed. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node.js v26.10.0; macOS 27.0.1 build 26A434 arm64; Apple LLDB lldb-2103.0.34.103. Xcode Command Line Tools provide `xcrun clang` and `xcrun lldb`. ### Target and artifact context A disposable, locally compiled Mach-O executable from inline C source, with a function breakpoint on `main`, `duration_ms: 100`, and `max_events: 1`. The driver appends bytes only to that fixture after the LLDB runtime configuration appears. Its private runtime and HOME/XDG directories remain under the disposable working directory. The retained bridge result identifies digest rejection before LLDB target creation. ### Interface CLI ### Earliest failure stage Cleanup, cancellation, or lifecycle ### Relevant output or evidence ```text exitCode: 1 mutated: true code: cleanup_incomplete category: integrity_mismatch execution_failure: evidence_integrity_mismatch bridge status: target-integrity-error bridge error: selected executable digest changed before LLDB target creation cleanup_reason: target process start identity was unavailable; survivor was left untouched cleanup_error.code: cleanup_incomplete resources include: native-target:unknown retained files: config.json, result.json pidExists: false ``` ### Typed outcome or recovery expected Keep the underlying `evidence_integrity_mismatch` rejection and its digest-change reason. For this positively proven pre-launch case, complete cleanup without an unresolved-target or close-time cleanup error, remove the owned runtime directory, and let repeated `close()` calls succeed. Restoring the fixture should allow the same tracer's next observation to reach its own launch or result instead of returning the stale cleanup failure. A discriminating post-launch case with unverified process exit must still retain cleanup uncertainty and the resources needed for a safe retry. An unmodified target must continue to clean its runtime after a normal launch failure. ### Analysis or suspected cause Source-derived reasoning at baseline `58300885207133422af235e0f2b10658cf958b03`: `bridge/native/rea_lldb_tracer.py:566-575` emits the reproduced `target-integrity-error` before `CreateTarget`; PID publication occurs later at `bridge/native/rea_lldb_tracer.py:635-636`. Host digest checks precede the tracer at `src/native/NativeCallObservation.ts:48-89`. `src/native/LldbCallTracer.ts:847-855` marks the target as possibly launched whenever LLDB has a supervisor. The missing PID reaches `ensureTerminated(..., true)`, which returns unavailable identity at `src/native/LldbCallTracer.ts:961-963`. The pre-launch exception at `src/native/LldbCallTracer.ts:423-436` recognizes `target-error` and `launch-error`, but omits the reproduced pre-CreateTarget integrity rejection. The retained owner retries that same missing-PID condition at `src/native/LldbCallTracer.ts:309-350` and prevents another trace at `src/native/LldbCallTracer.ts:196-197`. This source chain explains the observed stuck cleanup; it does not make every integrity error proof that no process launched. ### Additional context Related to #2397, which investigates event-limit mismatches and identity failure for an already launched target with a recorded PID; this report concerns cleanup after a confirmed digest rejection before target creation, with no PID and no target process.

bsakaguchi

2026-10-15 18:31:17 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:18 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Resolved on main by 45b6a7fe8.

Positively verified pre-launch rejection is distinguished from unknown/post-launch ownership, allowing safe cleanup and reuse without discarding uncertain resources. These are controlled boundary tests, not a fresh real-LLDB run.

Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: tests/boundary/providers/native/lldbCallTracerFailures.test.ts.

@morluto commented on GitHub (Oct 14, 2026): Resolved on main by 45b6a7fe8. Positively verified pre-launch rejection is distinguished from unknown/post-launch ownership, allowing safe cleanup and reuse without discarding uncertain resources. These are controlled boundary tests, not a fresh real-LLDB run. Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: tests/boundary/providers/native/lldbCallTracerFailures.test.ts.

bsakaguchi referenced this issue

2026-10-16 21:02:34 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:09:00 +00:00

[PR #1145] [MERGED] fix(android): validate engine archive prerequisites #2681