morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4575] [Bug] capture-process fails when an observed symlink disappears during a filesystem snapshot #2375

New issue

Closed

opened 2026-10-17 21:03:53 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-17 21:03:53 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4575

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

capture-process with filesystem_observation_paths fails the entire command when a symlink disappears after its entry lookup and before the follow-up readlink. The captured program prints capture succeeded and exits 0, but REA exits 1 with process_capture_failed. The error retains the initial snapshot, terminal output, successful target exit and cleanup report in partial_observation; the final filesystem snapshot and manifest are unavailable, and no normal capture Evidence is returned.

This reproduced in 3/3 deterministic removal trials on each tested build. All 3/3 stable controls on each build returned normal Evidence with truncated=false and the readable sibling in the final snapshot.

Reproduction

Use the builds listed under REA version. For the source build, run npm ci and npm run build:cached with the pinned toolchain first. For the published package, install it in an isolated directory. Start in the checkout or installed package root, where scripts/rea.mjs exists. Use the same Node executable for REA and the target.

Set these two variables once, while still in the checkout or installed package root:

rea_entry="$PWD/scripts/rea.mjs"
node_executable="$(command -v node)"

Create a fresh fixture and isolated HOME for every invocation, including each control and removal trial:

fixture="$(mktemp -d /tmp/rea-snapshot-repro.XXXXXX)"
mkdir -p "$fixture/state" "$fixture/home" "$fixture/tmp"
printf '%s' 'retain me' > "$fixture/state/a-sibling"
ln -s nonexistent-target "$fixture/state/z-link"
cat > "$fixture/scenario.json" <<JSON
{"executable":"$node_executable","arguments":["-e","console.log('capture succeeded');"],"working_directory":"$fixture","filesystem_observation_paths":["$fixture/state"],"timeout_ms":5000,"settle_ms":100}
JSON

Write this preload. It performs an actual unlink immediately before calling the real second readlink for the selected symlink. The first read belongs to the initial snapshot; the second belongs to the final snapshot. The helper runs in REA's CLI process, does not synthesize an error, and does not transform REA modules.

cat > "$fixture/hook.mjs" <<'HOOK'
import fs from 'node:fs/promises';
import { syncBuiltinESMExports } from 'node:module';
const original = fs.readlink;
let reads = 0;
fs.readlink = async function (path, ...args) {
  if (path === process.env.REPRO_LEAF && ++reads === 2) {
    await fs.unlink(path);
    process.stderr.write('REPRO: deleted symlink before final readlink\n');
  }
  return original.call(this, path, ...args);
};
syncBuiltinESMExports();
HOOK

Run the stable control without the preload:

cd "$fixture"
env -i PATH="$PATH" HOME="$fixture/home" TMPDIR="$fixture/tmp" REPRO_LEAF="$fixture/state/z-link" "$node_executable" "$rea_entry" capture-process "$fixture/scenario.json" --json

With a fresh fixture, HOME and symlink from steps 1–2, run the deterministic removal case:

cd "$fixture"
env -i PATH="$PATH" HOME="$fixture/home" TMPDIR="$fixture/tmp" REPRO_LEAF="$fixture/state/z-link" "$node_executable" --import "$fixture/hook.mjs" "$rea_entry" capture-process "$fixture/scenario.json" --json

I ran three controls before three removal trials on each build, with a fresh fixture per invocation. The observed removal signal is the marker on stderr followed by CLI exit 1 and the error projection below. These counts describe this deterministic fixture, not production incidence.

Expected behavior

Return usable partial filesystem Evidence that retains unaffected entries, records incomplete observation with a reason, and classifies effects that cannot be established as unknown. The documented filesystem coverage contract preserves observations around unavailable enumeration and does not infer absence from incomplete coverage. Existing identity-change handling also marks a snapshot incomplete rather than failing the checkpoint. A disappearance after entry lookup should preserve these partial facts.

A path already absent at its initial lookup is a different case and is already handled as observed absence. This report concerns an entry that was observed and then disappeared during a follow-up read.

Impact and scope

The reproduced scope is symlink disappearance during the final snapshot. It prevents normal capture completion and discards the requested final checkpoint even when the target succeeds. Initial state, output, target exit and cleanup remain available in the error's partial observation. All six removal trials reported the owned process group, terminal renderer and temporary root as cleaned. No target data loss or security-boundary crossing was observed.

Directory disappearance at the later directory revalidation is source-derived related scope only; no directory-disappearance CLI trial was executed in this verification. No claim about its incidence is made.

Impact level: P2
Assessment status: supported
Impact basis: A bounded concurrent filesystem change causes command failure and loss of the requested final filesystem Evidence while the successful target exit and earlier observations remain available.
Workaround status: available
Workaround basis: When writers are controllable, keep the selected filesystem state stable during capture; all six stable controls completed with the same selected paths. No universal workaround was established for mandatory concurrent churn. Omitting required observation paths loses evidence, and rerunning arbitrary target programs is not generally safe.

REA version

Current main source: f22e8e7268, built from the locked dependencies; --version returned 6.5.0.
Latest published npm package: rea-agents@6.5.0, installed separately; --version returned 6.5.0. The npm registry and latest GitHub release agreed on this version.

Runtime and platform

Node v24.18.0 on Darwin arm64 for both builds. The source build used its pinned npm 11.16.0. Every CLI trial ran from its fixture with a fresh HOME and temporary directory, in an environment containing only PATH, HOME, TMPDIR and the helper's REPRO_LEAF. No external analysis provider was involved.

Target and artifact context

A Node program that prints one line and exits successfully. The selected directory contains the readable regular file a-sibling and the dangling symlink z-link. Requested artifacts are process capture Evidence and filesystem before/after snapshots. The preload only coordinates the timing of a real removal in this disposable fixture.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Expected source: docs/process-capture.md:228–242 describes selected filesystem snapshots and unknown effects under incomplete enumeration; docs/process-capture.md:404–419 preserves entries around unavailable enumeration. AGENTS.md, Architecture and Cleanup, requires preservation of partial facts and meaningful failure reasons. These passages also exist at the release tag. src/process/capture/FilesystemSnapshot.ts:230–233 preserves incomplete observations on identity changes.
Failure source: The public CLI trials above produced the error projection below on both tested builds. src/process/capture/FilesystemSnapshot.ts:219–243 performs the entry lookup and unguarded follow-up readlink/lstat; src/process/capture/ProcessHarness.ts:515–528 marks the final checkpoint unavailable and rethrows. Source locators refer to the main build unless stated otherwise.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The following projection preserves the common result from all six removal trials. Only the generated fixture path is replaced with <fixture>:

stderr: REPRO: deleted symlink before final readlink
CLI exit: 1
code: process_capture_failed
category: execution_failure
details.execution_failure: ENOENT: no such file or directory, readlink '<fixture>/state/z-link'
details.partial_observation.observations.frames.value[0].data: capture succeeded\r\n
details.partial_observation.observations.exit.value: {code:0,signal:0,reason:"exited"}
details.partial_observation.observations.filesystem_snapshots.before.state: available
details.partial_observation.observations.filesystem_snapshots.after.state: unavailable
details.partial_observation.observations.filesystem_snapshots.after.reason: Final filesystem snapshot failed: ENOENT: no such file or directory, readlink '<fixture>/state/z-link'
details.partial_observation.observations.manifest.state: unavailable

Source-derived boundary: FilesystemSnapshot.ts:50–72 handles absence during initial lookup, :300–315 preserves omissions around opendir failures, and :333 performs another unguarded directory revalidation. The latter directory case is unexecuted scope, not a second observed reproduction.

Typed outcome or recovery expected

Proposed restoration check: repeat the same deterministic final-readlink removal and receive normal capture Evidence with an available partial final checkpoint, the readable sibling retained, and explicit incomplete coverage and unknown affected effects with reasons. The stable control must continue to return a complete snapshot with truncated=false. These are proposed acceptance checks, not a claim that a fix has been implemented.

Analysis or suspected cause

Source-derived reasoning: after entry lookup succeeds, the later symlink readlink and lstat can reject with ENOENT. Existing handling covers initial absence, unavailable directory opens and detected identity changes, but does not contain disappearance errors from those follow-up reads. The final checkpoint error then propagates to command failure. The report leaves the recovery representation and implementation to maintainers.

Additional context

Related but distinct: #3238 fixed a previously observed selected root becoming unreachable through ENOTDIR; its contributor comment explicitly notes remaining mid-traversal ENOENT races. #2545 fixed unavailable directory opens, rather than these later follow-up reads. #3303 fixed literal symlink-target retention, rather than disappearance. Searches of open and closed issues and PRs found no report covering this reproduced follow-up-read failure. No last-known-good version was established.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4575 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `capture-process` with `filesystem_observation_paths` fails the entire command when a symlink disappears after its entry lookup and before the follow-up `readlink`. The captured program prints `capture succeeded` and exits 0, but REA exits 1 with `process_capture_failed`. The error retains the initial snapshot, terminal output, successful target exit and cleanup report in `partial_observation`; the final filesystem snapshot and manifest are unavailable, and no normal capture Evidence is returned. This reproduced in 3/3 deterministic removal trials on each tested build. All 3/3 stable controls on each build returned normal Evidence with `truncated=false` and the readable sibling in the final snapshot. ### Reproduction Use the builds listed under REA version. For the source build, run `npm ci` and `npm run build:cached` with the pinned toolchain first. For the published package, install it in an isolated directory. Start in the checkout or installed package root, where `scripts/rea.mjs` exists. Use the same Node executable for REA and the target. Set these two variables once, while still in the checkout or installed package root: ```sh rea_entry="$PWD/scripts/rea.mjs" node_executable="$(command -v node)" ``` 1. Create a fresh fixture and isolated HOME for every invocation, including each control and removal trial: ```sh fixture="$(mktemp -d /tmp/rea-snapshot-repro.XXXXXX)" mkdir -p "$fixture/state" "$fixture/home" "$fixture/tmp" printf '%s' 'retain me' > "$fixture/state/a-sibling" ln -s nonexistent-target "$fixture/state/z-link" cat > "$fixture/scenario.json" <<JSON {"executable":"$node_executable","arguments":["-e","console.log('capture succeeded');"],"working_directory":"$fixture","filesystem_observation_paths":["$fixture/state"],"timeout_ms":5000,"settle_ms":100} JSON ``` 2. Write this preload. It performs an actual unlink immediately before calling the real second `readlink` for the selected symlink. The first read belongs to the initial snapshot; the second belongs to the final snapshot. The helper runs in REA's CLI process, does not synthesize an error, and does not transform REA modules. ```sh cat > "$fixture/hook.mjs" <<'HOOK' import fs from 'node:fs/promises'; import { syncBuiltinESMExports } from 'node:module'; const original = fs.readlink; let reads = 0; fs.readlink = async function (path, ...args) { if (path === process.env.REPRO_LEAF && ++reads === 2) { await fs.unlink(path); process.stderr.write('REPRO: deleted symlink before final readlink\n'); } return original.call(this, path, ...args); }; syncBuiltinESMExports(); HOOK ``` 3. Run the stable control without the preload: ```sh cd "$fixture" env -i PATH="$PATH" HOME="$fixture/home" TMPDIR="$fixture/tmp" REPRO_LEAF="$fixture/state/z-link" "$node_executable" "$rea_entry" capture-process "$fixture/scenario.json" --json ``` 4. With a fresh fixture, HOME and symlink from steps 1–2, run the deterministic removal case: ```sh cd "$fixture" env -i PATH="$PATH" HOME="$fixture/home" TMPDIR="$fixture/tmp" REPRO_LEAF="$fixture/state/z-link" "$node_executable" --import "$fixture/hook.mjs" "$rea_entry" capture-process "$fixture/scenario.json" --json ``` I ran three controls before three removal trials on each build, with a fresh fixture per invocation. The observed removal signal is the marker on stderr followed by CLI exit 1 and the error projection below. These counts describe this deterministic fixture, not production incidence. ### Expected behavior Return usable partial filesystem Evidence that retains unaffected entries, records incomplete observation with a reason, and classifies effects that cannot be established as unknown. The documented filesystem coverage contract preserves observations around unavailable enumeration and does not infer absence from incomplete coverage. Existing identity-change handling also marks a snapshot incomplete rather than failing the checkpoint. A disappearance after entry lookup should preserve these partial facts. A path already absent at its initial lookup is a different case and is already handled as observed absence. This report concerns an entry that was observed and then disappeared during a follow-up read. ### Impact and scope The reproduced scope is symlink disappearance during the final snapshot. It prevents normal capture completion and discards the requested final checkpoint even when the target succeeds. Initial state, output, target exit and cleanup remain available in the error's partial observation. All six removal trials reported the owned process group, terminal renderer and temporary root as cleaned. No target data loss or security-boundary crossing was observed. Directory disappearance at the later directory revalidation is source-derived related scope only; no directory-disappearance CLI trial was executed in this verification. No claim about its incidence is made. - Impact level: P2 - Assessment status: supported - Impact basis: A bounded concurrent filesystem change causes command failure and loss of the requested final filesystem Evidence while the successful target exit and earlier observations remain available. - Workaround status: available - Workaround basis: When writers are controllable, keep the selected filesystem state stable during capture; all six stable controls completed with the same selected paths. No universal workaround was established for mandatory concurrent churn. Omitting required observation paths loses evidence, and rerunning arbitrary target programs is not generally safe. ### REA version - Current main source: f22e8e72685c05e372c62582c7bc5fcd085b2fa6, built from the locked dependencies; `--version` returned `6.5.0`. - Latest published npm package: `rea-agents@6.5.0`, installed separately; `--version` returned `6.5.0`. The npm registry and latest GitHub release agreed on this version. ### Runtime and platform Node `v24.18.0` on `Darwin arm64` for both builds. The source build used its pinned npm `11.16.0`. Every CLI trial ran from its fixture with a fresh HOME and temporary directory, in an environment containing only PATH, HOME, TMPDIR and the helper's REPRO_LEAF. No external analysis provider was involved. ### Target and artifact context A Node program that prints one line and exits successfully. The selected directory contains the readable regular file `a-sibling` and the dangling symlink `z-link`. Requested artifacts are process capture Evidence and filesystem before/after snapshots. The preload only coordinates the timing of a real removal in this disposable fixture. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence - Expected source: `docs/process-capture.md:228–242` describes selected filesystem snapshots and unknown effects under incomplete enumeration; `docs/process-capture.md:404–419` preserves entries around unavailable enumeration. `AGENTS.md`, Architecture and Cleanup, requires preservation of partial facts and meaningful failure reasons. These passages also exist at the release tag. `src/process/capture/FilesystemSnapshot.ts:230–233` preserves incomplete observations on identity changes. - Failure source: The public CLI trials above produced the error projection below on both tested builds. `src/process/capture/FilesystemSnapshot.ts:219–243` performs the entry lookup and unguarded follow-up `readlink`/`lstat`; `src/process/capture/ProcessHarness.ts:515–528` marks the final checkpoint unavailable and rethrows. Source locators refer to the main build unless stated otherwise. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The following projection preserves the common result from all six removal trials. Only the generated fixture path is replaced with `<fixture>`: ```text stderr: REPRO: deleted symlink before final readlink CLI exit: 1 code: process_capture_failed category: execution_failure details.execution_failure: ENOENT: no such file or directory, readlink '<fixture>/state/z-link' details.partial_observation.observations.frames.value[0].data: capture succeeded\r\n details.partial_observation.observations.exit.value: {code:0,signal:0,reason:"exited"} details.partial_observation.observations.filesystem_snapshots.before.state: available details.partial_observation.observations.filesystem_snapshots.after.state: unavailable details.partial_observation.observations.filesystem_snapshots.after.reason: Final filesystem snapshot failed: ENOENT: no such file or directory, readlink '<fixture>/state/z-link' details.partial_observation.observations.manifest.state: unavailable ``` Source-derived boundary: `FilesystemSnapshot.ts:50–72` handles absence during initial lookup, `:300–315` preserves omissions around `opendir` failures, and `:333` performs another unguarded directory revalidation. The latter directory case is unexecuted scope, not a second observed reproduction. ### Typed outcome or recovery expected Proposed restoration check: repeat the same deterministic final-readlink removal and receive normal capture Evidence with an available partial final checkpoint, the readable sibling retained, and explicit incomplete coverage and unknown affected effects with reasons. The stable control must continue to return a complete snapshot with `truncated=false`. These are proposed acceptance checks, not a claim that a fix has been implemented. ### Analysis or suspected cause Source-derived reasoning: after entry lookup succeeds, the later symlink `readlink` and `lstat` can reject with ENOENT. Existing handling covers initial absence, unavailable directory opens and detected identity changes, but does not contain disappearance errors from those follow-up reads. The final checkpoint error then propagates to command failure. The report leaves the recovery representation and implementation to maintainers. ### Additional context Related but distinct: [#3238](https://github.com/morluto/rea/issues/3238) fixed a previously observed selected root becoming unreachable through ENOTDIR; its contributor comment explicitly notes remaining mid-traversal ENOENT races. [#2545](https://github.com/morluto/rea/issues/2545) fixed unavailable directory opens, rather than these later follow-up reads. [#3303](https://github.com/morluto/rea/issues/3303) fixed literal symlink-target retention, rather than disappearance. Searches of open and closed issues and PRs found no report covering this reproduced follow-up-read failure. No last-known-good version was established.

bsakaguchi commented

2026-10-17 21:03:54 +00:00

Author

Owner

@vikas468368-star commented on GitHub (Oct 16, 2026):

hey @coygeek , Can I work on this issue.

@vikas468368-star commented on GitHub (Oct 16, 2026): hey @coygeek , Can I work on this issue.

bsakaguchi commented

2026-10-17 21:03:55 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 16, 2026):

Sure, go ahead.

@coygeek commented on GitHub (Oct 16, 2026): Sure, go ahead.

bsakaguchi commented

2026-10-17 21:03:55 +00:00

Author

Owner

@vikas468368-star commented on GitHub (Oct 16, 2026):

thanks.

@vikas468368-star commented on GitHub (Oct 16, 2026): thanks.

bsakaguchi referenced this issue

2026-10-31 00:29:41 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi closed this issue

2026-10-31 00:33:13 +00:00

bsakaguchi commented

2026-10-31 00:33:14 +00:00

Author

Owner

@morluto commented on GitHub (Oct 19, 2026):

Fixed on main in c85b44e1b.

The root cause was unguarded follow-up readlink/lstat after successful metadata admission. The snapshot owner now returns typed partial observations: entry_unavailable retains path and errno, while a successfully observed identity mismatch records identity_changed with no invented errno. Unstable symlinks are omitted, observed directory metadata and siblings survive, and incomplete roots keep absence unknown. Cancellation and unexpected I/O failures remain terminal. Root admission metadata is reused; the affected cancellation regression now follows descriptor state instead of counting lstat calls.

Validation: 125 passing focused checks across 13 capture/schema/CLI/MCP files, 4 Windows-only cases skipped on macOS; full typecheck/lint/module and architecture guards, docs checks (5 tests), and discovery of all 1604 test files passed. The compiled public CLI regression performs a real symlink removal during final observation after a successful controlled Node target. Additional production-boundary tests cover real removal/replacement errno, directory revalidation, cancellation including after iterator close, released name budgets, and unexpected-error propagation. Scheduling is controlled; some permission/cancellation faults are injected. The unchanged baseline failed the regression cases before this fix.

Six equivalent stable SDK workflows (catalog discovery, actual controlled Node capture, residual Unknown lookup, awaited cleanup) preserved the filesystem/exit/cleanup oracle. Repeated-process medians: elapsed 3.320→3.295 s, CPU 4.057→3.940 s, sampled process-family RSS 893.6→892.9 MB. Requests remained 109 cl100k-estimated tokens; full serialized SDK replies including discovery were 814161→814065, varying with ambient unrelated-process diagnostics. The 207-tool catalog increased by 14 estimated tokens. These small controls establish no speedup or actual model-context saving; true cold, large/fault costs and other hosts remain unmeasured.

Adapted the symlink-race proposal in #4587 at fae4e25539969671c82a4d56cfb0b50761b833a3, with contributor credit in the commit; the PR was not merged. The commit marks the expanded omission-reason contract as breaking for strict old-schema clients.

The nearby-consumer scan also found and fixed #5135 and #5136. Remaining scope: this does not convert directory iterator/close failures into partial success; their cleanup ownership must be established separately before broadening that catch. No actual failed native directory-close reproduction, Windows/Linux race qualification, or native analysis-engine qualification is claimed.

@morluto commented on GitHub (Oct 19, 2026): Fixed on main in [c85b44e1b](https://github.com/morluto/rea/commit/c85b44e1b14851c54e65327a4807f79ad5cd5a61). The root cause was unguarded follow-up `readlink`/`lstat` after successful metadata admission. The snapshot owner now returns typed partial observations: `entry_unavailable` retains path and errno, while a successfully observed identity mismatch records `identity_changed` with no invented errno. Unstable symlinks are omitted, observed directory metadata and siblings survive, and incomplete roots keep absence unknown. Cancellation and unexpected I/O failures remain terminal. Root admission metadata is reused; the affected cancellation regression now follows descriptor state instead of counting `lstat` calls. Validation: **125 passing focused checks across 13 capture/schema/CLI/MCP files, 4 Windows-only cases skipped on macOS**; full typecheck/lint/module and architecture guards, docs checks (5 tests), and discovery of all 1604 test files passed. The compiled public CLI regression performs a real symlink removal during final observation after a successful controlled Node target. Additional production-boundary tests cover real removal/replacement errno, directory revalidation, cancellation including after iterator close, released name budgets, and unexpected-error propagation. Scheduling is controlled; some permission/cancellation faults are injected. The unchanged baseline failed the regression cases before this fix. Six equivalent stable SDK workflows (catalog discovery, actual controlled Node capture, residual Unknown lookup, awaited cleanup) preserved the filesystem/exit/cleanup oracle. Repeated-process medians: elapsed 3.320→3.295 s, CPU 4.057→3.940 s, sampled process-family RSS 893.6→892.9 MB. Requests remained 109 cl100k-estimated tokens; full serialized SDK replies including discovery were 814161→814065, varying with ambient unrelated-process diagnostics. The 207-tool catalog increased by 14 estimated tokens. These small controls establish no speedup or actual model-context saving; true cold, large/fault costs and other hosts remain unmeasured. Adapted the symlink-race proposal in #4587 at `fae4e25539969671c82a4d56cfb0b50761b833a3`, with contributor credit in the commit; the PR was not merged. The commit marks the expanded omission-reason contract as breaking for strict old-schema clients. The nearby-consumer scan also found and fixed #5135 and #5136. Remaining scope: this does not convert directory iterator/close failures into partial success; their cleanup ownership must be established separately before broadening that catch. No actual failed native directory-close reproduction, Windows/Linux race qualification, or native analysis-engine qualification is claimed.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2375

No description provided.

Rows
Columns