morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3180] [Bug] Passing filesystem traces still report ignored checkpoint timestamp changes #1119

New issue

Closed

opened 2026-10-15 18:31:13 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:13 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3180

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

compare-process-captures reports a filesystem change when both complete input captures satisfy a trace that explicitly ignores filesystem checkpoint timestamps. The same passing trace suppresses a terminal timestamp difference, but the ignored filesystem timestamp remains the first divergence and keeps the overall result changed. The compiled CLI exits successfully with trace.verdict: equivalent and both trace sides passing.

Reproduction

Start from a clean checkout at 58300885207133422af235e0f2b10658cf958b03. Install its locked dependencies and build the CLI. These preparation commands are included for a fresh checkout; verification used an already installed and built checkout.

npm ci
npm run build:cached

From the checkout root, create the two input Evidence files and two trace specifications inline. The right input changes only the terminal timestamp from 900 to 1000 and the after-settlement checkpoint timestamp from 50 to 60.

node --input-type=module <<'JS'
import { writeFileSync, mkdirSync } from 'node:fs';
import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from './dist/domain/process/processCaptureExample.js';
import { parseProcessCapture } from './dist/domain/process/processCaptureParsing.js';
import { createEvidence } from './dist/domain/evidence.js';
import { PROCESS_PROVIDER } from './dist/domain/process/processEvidenceProvider.js';
const directory = process.env.REPRO_DIR || 'repro-data';
mkdirSync(directory, { recursive: true });
function makeCapture(checkpointTime, terminalTime) {
  const capture = structuredClone(EMPTY_PROCESS_CAPTURE_EXAMPLE);
  capture.frames = [{ sequence: 0, at_ms: terminalTime, data: 'Ready' }];
  capture.filesystem_checkpoints[1].at_ms = checkpointTime;
  Object.assign(capture.truncation_details.raw_terminal, {
    observed_bytes: 5, retained_bytes: 5, observed_frames: 1, retained_frames: 1,
  });
  capture.event_journal = [
    ['frames', 0], ['filesystem_checkpoints', 0], ['lifecycle', 0],
    ['lifecycle', 1], ['filesystem_checkpoints', 1],
  ].map(([collection, index], capture_order) => ({ collection, index, capture_order }));
  return parseProcessCapture(capture);
}
function wrap(result) {
  return createEvidence(undefined, PROCESS_PROVIDER, {
    predicateType: 'rea.process-capture', operation: 'capture_process_scenario',
    parameters: {
      executable_name: 'node', argument_count: 0, event_count: 0,
      filesystem_observation_path_count: 0,
    },
    result, confidence: 'observed', authority: 'controlled-replay', limitations: [],
  });
}
function save(name, value) {
  writeFileSync(`${directory}/${name}.json`, JSON.stringify(value, null, 2));
}
const left = makeCapture(50, 900);
const right = makeCapture(60, 1000);
save('left', wrap(left));
save('right', wrap(right));
const ready = {
  id: 'ready', source: 'terminal_raw', exact: { data: 'Ready' },
  ignore_fields: ['sequence', 'at_ms'], cardinality: { kind: 'required' },
};
const checkpoints = left.filesystem_checkpoints.map(({ at_ms, ...exact }, index) => ({
  id: `fs${index}`, source: 'filesystem', exact,
  ignore_fields: ['at_ms'], cardinality: { kind: 'required' },
}));
save('trace', {
  events: [ready, ...checkpoints],
  language: { kind: 'finite_traces', variants: [{ id: 'accepted', trace: ['ready', 'fs0', 'fs1'] }] },
});
save('terminal-only', {
  events: [ready],
  language: { kind: 'finite_traces', variants: [{ id: 'accepted', trace: ['ready'] }] },
});
JS

Run the comparison with all terminal and filesystem observations declared in the trace, then run the controls.

node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/right.json repro-data/trace.json --format json
node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/right.json --format json
node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/left.json repro-data/trace.json --format json
node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/right.json repro-data/terminal-only.json --format json

Inspect each output's normalized_result. All commands complete successfully. The first comparison has an equivalent trace with both sides passing, but its filesystem and overall status remain changed.

Expected behavior

docs/process-capture.md:76-88 documents comparison of terminal, interaction, exit, filesystem, and process observations under a shared contract, with an optional exact-event trace specification and caller-selected ordering constraints. Missing journals, relevant residual unknowns, or truncated observations must prevent complete equivalence. The trace schema permits at_ms in ignore_fields at src/domain/process/processTraceSpecification.ts:42-60.

For these complete captures, the trace declares every terminal and filesystem observation and both sides pass. All other dimensions are identical. The result should report terminal: unchanged, filesystem: unchanged, overall status: unchanged, and first_divergence.status: none, while retaining trace.verdict: equivalent.

Without a trace, both timestamp differences should remain changes. A trace declaring only terminal observations should retain the filesystem difference. Identical captures should remain unchanged. Unknown or truncated filesystem observations must continue to prevent an equivalence claim.

Impact and scope

This produces a false difference for callers using an accepted trace specification to compare checkpoint timing variations. The verified case uses complete synthetic process-capture Evidence with compatible commitments, complete journals, no residual unknowns, and no truncation. The offline comparison preserves the input Evidence and does not mutate or lose data.

For the demonstrated complete case, callers can inspect the passing trace, check the unchanged other dimensions, and confirm that the retained filesystem divergence contains only the explicitly ignored timestamp. The output retains the evidence needed for that manual interpretation. Verification exercised the compiled offline CLI on macOS arm64; it did not exercise live PTY capture, MCP transport, or another platform.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node.js v26.10.0; macOS arm64. No OS version was recorded in the verifier's environment description.

Target and artifact context

Two synthetic complete process-capture Evidence JSON files generated from the public capture example and current process Evidence provider, plus finite-trace specifications created inline. The right input changes only the raw terminal timestamp from 900 to 1000 and the after-settlement checkpoint timestamp from 50 to 60. No target process or external analysis provider is required.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The full trace comparison exits 0 and returns this excerpt from normalized_result. Both trace sides pass and match the declared finite trace.

{
  "status": "changed",
  "terminal": "unchanged",
  "interaction": "unchanged",
  "exit": "unchanged",
  "filesystem": "changed",
  "process": "unchanged",
  "first_divergence": {
    "status": "found",
    "dimension": "filesystem",
    "index": 1,
    "left_at_ms": 50,
    "right_at_ms": 60
  },
  "trace": { "verdict": "equivalent" }
}

The no-trace control reports terminal and filesystem changed. The identical-input control reports both dimensions and overall status unchanged with no divergence. The terminal-only trace reports terminal unchanged and filesystem changed. A partial-order specification covering the same observations also reproduced the filesystem failure. The verifier reran the inline fixture generator and all four comparison commands against the prepared compiled CLI; dependency installation and the clean-checkout build prerequisites were not rerun during verification.

Typed outcome or recovery expected

Return an unchanged comparison with an equivalent trace and no first divergence for the full trace case. Preserve the exact timestamp differences in the no-trace comparison and the filesystem difference in the terminal-only comparison. Preserve unknown outcomes when filesystem observations are incomplete, unknown, or truncated.

Analysis or suspected cause

Source-derived analysis at baseline 58300885207133422af235e0f2b10658cf958b03: src/domain/process/processTraceDimensionProjection.ts:20 maps the filesystem trace source to the filesystem comparison dimension, and line 26 defines coverage as every observed source in a dimension. Its filesystem branch at lines 48-49 nevertheless always returns false. The equivalent-verdict projection at src/domain/process/processComparison.ts:317-377 therefore retains the exact filesystem status even when both trace sides pass.

The filesystem observation projection at src/domain/process/processObservation.ts:174-177 uses the checkpoint records consumed by the filesystem comparison at src/domain/process/processComparison.ts:126-150. The production CLI validates Evidence, the process provider, captures, and the trace specification before comparison at src/application/process/ProcessCli.ts:98-200; the verified inputs passed those checks. These locators support the suspected cause, separately from the reproduced CLI output.

Additional context

Related #2616 concerns a failing trace assigning changes to equal observation dimensions, while this report concerns a passing trace retaining an explicitly ignored filesystem timing difference. Related #724 requests explanations of content, outcome, and timing facets, while this report requires the existing dimension status to honor a complete accepted trace.

P2 reflects a bounded consequential comparison defect with a practical manual workaround for the demonstrated complete case. No data loss, mutation, or security boundary crossing was observed.

Related: #3181 reports a separate failure in the same area; the two have independent fixes.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3180 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `compare-process-captures` reports a filesystem change when both complete input captures satisfy a trace that explicitly ignores filesystem checkpoint timestamps. The same passing trace suppresses a terminal timestamp difference, but the ignored filesystem timestamp remains the first divergence and keeps the overall result changed. The compiled CLI exits successfully with `trace.verdict: equivalent` and both trace sides passing. ### Reproduction 1. Start from a clean checkout at `58300885207133422af235e0f2b10658cf958b03`. Install its locked dependencies and build the CLI. These preparation commands are included for a fresh checkout; verification used an already installed and built checkout. ```sh npm ci npm run build:cached ``` 2. From the checkout root, create the two input Evidence files and two trace specifications inline. The right input changes only the terminal timestamp from 900 to 1000 and the after-settlement checkpoint timestamp from 50 to 60. ```sh node --input-type=module <<'JS' import { writeFileSync, mkdirSync } from 'node:fs'; import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from './dist/domain/process/processCaptureExample.js'; import { parseProcessCapture } from './dist/domain/process/processCaptureParsing.js'; import { createEvidence } from './dist/domain/evidence.js'; import { PROCESS_PROVIDER } from './dist/domain/process/processEvidenceProvider.js'; const directory = process.env.REPRO_DIR || 'repro-data'; mkdirSync(directory, { recursive: true }); function makeCapture(checkpointTime, terminalTime) { const capture = structuredClone(EMPTY_PROCESS_CAPTURE_EXAMPLE); capture.frames = [{ sequence: 0, at_ms: terminalTime, data: 'Ready' }]; capture.filesystem_checkpoints[1].at_ms = checkpointTime; Object.assign(capture.truncation_details.raw_terminal, { observed_bytes: 5, retained_bytes: 5, observed_frames: 1, retained_frames: 1, }); capture.event_journal = [ ['frames', 0], ['filesystem_checkpoints', 0], ['lifecycle', 0], ['lifecycle', 1], ['filesystem_checkpoints', 1], ].map(([collection, index], capture_order) => ({ collection, index, capture_order })); return parseProcessCapture(capture); } function wrap(result) { return createEvidence(undefined, PROCESS_PROVIDER, { predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: { executable_name: 'node', argument_count: 0, event_count: 0, filesystem_observation_path_count: 0, }, result, confidence: 'observed', authority: 'controlled-replay', limitations: [], }); } function save(name, value) { writeFileSync(`${directory}/${name}.json`, JSON.stringify(value, null, 2)); } const left = makeCapture(50, 900); const right = makeCapture(60, 1000); save('left', wrap(left)); save('right', wrap(right)); const ready = { id: 'ready', source: 'terminal_raw', exact: { data: 'Ready' }, ignore_fields: ['sequence', 'at_ms'], cardinality: { kind: 'required' }, }; const checkpoints = left.filesystem_checkpoints.map(({ at_ms, ...exact }, index) => ({ id: `fs${index}`, source: 'filesystem', exact, ignore_fields: ['at_ms'], cardinality: { kind: 'required' }, })); save('trace', { events: [ready, ...checkpoints], language: { kind: 'finite_traces', variants: [{ id: 'accepted', trace: ['ready', 'fs0', 'fs1'] }] }, }); save('terminal-only', { events: [ready], language: { kind: 'finite_traces', variants: [{ id: 'accepted', trace: ['ready'] }] }, }); JS ``` 3. Run the comparison with all terminal and filesystem observations declared in the trace, then run the controls. ```sh node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/right.json repro-data/trace.json --format json node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/right.json --format json node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/left.json repro-data/trace.json --format json node scripts/rea.mjs compare-process-captures repro-data/left.json repro-data/right.json repro-data/terminal-only.json --format json ``` 4. Inspect each output's `normalized_result`. All commands complete successfully. The first comparison has an equivalent trace with both sides passing, but its filesystem and overall status remain changed. ### Expected behavior `docs/process-capture.md:76-88` documents comparison of terminal, interaction, exit, filesystem, and process observations under a shared contract, with an optional exact-event trace specification and caller-selected ordering constraints. Missing journals, relevant residual unknowns, or truncated observations must prevent complete equivalence. The trace schema permits `at_ms` in `ignore_fields` at `src/domain/process/processTraceSpecification.ts:42-60`. For these complete captures, the trace declares every terminal and filesystem observation and both sides pass. All other dimensions are identical. The result should report `terminal: unchanged`, `filesystem: unchanged`, overall `status: unchanged`, and `first_divergence.status: none`, while retaining `trace.verdict: equivalent`. Without a trace, both timestamp differences should remain changes. A trace declaring only terminal observations should retain the filesystem difference. Identical captures should remain unchanged. Unknown or truncated filesystem observations must continue to prevent an equivalence claim. ### Impact and scope This produces a false difference for callers using an accepted trace specification to compare checkpoint timing variations. The verified case uses complete synthetic process-capture Evidence with compatible commitments, complete journals, no residual unknowns, and no truncation. The offline comparison preserves the input Evidence and does not mutate or lose data. For the demonstrated complete case, callers can inspect the passing trace, check the unchanged other dimensions, and confirm that the retained filesystem divergence contains only the explicitly ignored timestamp. The output retains the evidence needed for that manual interpretation. Verification exercised the compiled offline CLI on macOS arm64; it did not exercise live PTY capture, MCP transport, or another platform. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node.js v26.10.0; macOS arm64. No OS version was recorded in the verifier's environment description. ### Target and artifact context Two synthetic complete process-capture Evidence JSON files generated from the public capture example and current process Evidence provider, plus finite-trace specifications created inline. The right input changes only the raw terminal timestamp from 900 to 1000 and the after-settlement checkpoint timestamp from 50 to 60. No target process or external analysis provider is required. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The full trace comparison exits 0 and returns this excerpt from `normalized_result`. Both trace sides pass and match the declared finite trace. ```text { "status": "changed", "terminal": "unchanged", "interaction": "unchanged", "exit": "unchanged", "filesystem": "changed", "process": "unchanged", "first_divergence": { "status": "found", "dimension": "filesystem", "index": 1, "left_at_ms": 50, "right_at_ms": 60 }, "trace": { "verdict": "equivalent" } } ``` The no-trace control reports terminal and filesystem changed. The identical-input control reports both dimensions and overall status unchanged with no divergence. The terminal-only trace reports terminal unchanged and filesystem changed. A partial-order specification covering the same observations also reproduced the filesystem failure. The verifier reran the inline fixture generator and all four comparison commands against the prepared compiled CLI; dependency installation and the clean-checkout build prerequisites were not rerun during verification. ### Typed outcome or recovery expected Return an unchanged comparison with an equivalent trace and no first divergence for the full trace case. Preserve the exact timestamp differences in the no-trace comparison and the filesystem difference in the terminal-only comparison. Preserve unknown outcomes when filesystem observations are incomplete, unknown, or truncated. ### Analysis or suspected cause Source-derived analysis at baseline `58300885207133422af235e0f2b10658cf958b03`: `src/domain/process/processTraceDimensionProjection.ts:20` maps the filesystem trace source to the filesystem comparison dimension, and line 26 defines coverage as every observed source in a dimension. Its filesystem branch at lines 48-49 nevertheless always returns false. The equivalent-verdict projection at `src/domain/process/processComparison.ts:317-377` therefore retains the exact filesystem status even when both trace sides pass. The filesystem observation projection at `src/domain/process/processObservation.ts:174-177` uses the checkpoint records consumed by the filesystem comparison at `src/domain/process/processComparison.ts:126-150`. The production CLI validates Evidence, the process provider, captures, and the trace specification before comparison at `src/application/process/ProcessCli.ts:98-200`; the verified inputs passed those checks. These locators support the suspected cause, separately from the reproduced CLI output. ### Additional context Related [#2616](https://github.com/morluto/rea/issues/2616) concerns a failing trace assigning changes to equal observation dimensions, while this report concerns a passing trace retaining an explicitly ignored filesystem timing difference. Related [#724](https://github.com/morluto/rea/issues/724) requests explanations of content, outcome, and timing facets, while this report requires the existing dimension status to honor a complete accepted trace. P2 reflects a bounded consequential comparison defect with a practical manual workaround for the demonstrated complete case. No data loss, mutation, or security boundary crossing was observed. Related: #3181 reports a separate failure in the same area; the two have independent fixes.

bsakaguchi

2026-10-15 18:31:13 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:14 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Delivered on main in fffb07419.

Filesystem trace projection never covered an exact timestamp-only difference. Trace-aware comparison now honors the caller timestamp rule while retaining actual file-content changes and incomplete coverage as blockers.

Validation: Process trace tests cover ignored timestamps, real changes and the no-trace control. Full exact-revision qualification passed.

@morluto commented on GitHub (Oct 14, 2026): Delivered on main in [fffb07419](https://github.com/morluto/rea/commit/fffb0741936067530d1ce9777461cd53e87398c2). Filesystem trace projection never covered an exact timestamp-only difference. Trace-aware comparison now honors the caller timestamp rule while retaining actual file-content changes and incomplete coverage as blockers. Validation: Process trace tests cover ignored timestamps, real changes and the no-trace control. Full exact-revision [qualification](https://github.com/morluto/rea/actions/runs/38629313022) passed.

bsakaguchi referenced this issue

2026-10-17 21:08:47 +00:00

[PR #1119] [MERGED] fix(hopper): enforce caller-supplied request deadlines #2664

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1119

No description provided.

Rows
Columns