[GH-ISSUE #2645] fix(process): later process differences hide earlier interaction outcomes #948

New issue

Closed

opened 2026-10-15 18:29:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:29:48 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2645

Summary

When two compatible recorded captures differ in an interaction outcome at 6 ms and in a process observation at 500 ms, the comparison identifies the process observation as first_divergence. The earlier interaction is correctly classified as changed, but its recorded dispatch timestamp does not participate in the first-difference selection. This gives callers the wrong starting location for diagnosis while retaining the underlying observations.

Steps to reproduce

Use the source baseline and development toolchain stated below with its locked dependencies available. Save the following test as src/domain/process/interaction-first-divergence.test.ts, then run npm run test:focused -- src/domain/process/interaction-first-divergence.test.ts. The test uses owned synthetic recorded captures and does not launch a target. This reduced test was executed. A separate source application-workflow run also admitted canonical capture Evidence files and returned the same result through compareProcessEvidenceFiles.

import { expect, it } from "vitest";
import { compareProcessCaptures } from "./processComparison.js";
import { parseProcessCapture } from "./processCaptureParsing.js";
import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "./processCaptureExample.js";
import { accountFullyObservedProcessCapture } from "./processCapture.fixture.js";

const makeCapture = (side: "left" | "right", processTime: number) => {
  const base = EMPTY_PROCESS_CAPTURE_EXAMPLE;
  const journal = [
    { collection: "filesystem_checkpoints" as const, index: 0 },
    ...(processTime < 6 ? [{ collection: "process_samples" as const, index: 0 }] : []),
    { collection: "interaction_events" as const, index: 0 },
    ...(processTime >= 6 ? [{ collection: "process_samples" as const, index: 0 }] : []),
    { collection: "lifecycle" as const, index: 0 },
    { collection: "lifecycle" as const, index: 1 },
    { collection: "filesystem_checkpoints" as const, index: 1 },
  ].map((entry, capture_order) => ({ ...entry, capture_order }));
  return parseProcessCapture(accountFullyObservedProcessCapture({
    ...base,
    manifest: { ...base.manifest, completed_at: "2026-01-01T00:00:01.000Z" },
    settlement: { ...base.settlement, elapsed_ms: 1000 },
    filesystem_checkpoints: base.filesystem_checkpoints.map((c, i) => ({ ...c, at_ms: i === 0 ? 0 : 1000 })),
    interaction_events: [{ sequence: 0, scheduled_at_ms: 5, dispatched_at_ms: 6, type: "input", data: "hello", outcome: side === "left" ? "dispatched" : "failed" }],
    process_samples: [{ at_ms: processTime, pid: 2, parent_pid: 1, command: side === "left" ? "worker-a" : "worker-b", process_group_id: 1, session_id: 1 }],
    event_journal: journal,
  }));
};
it("reports the earlier interaction difference", () => {
  const comparison = compareProcessCaptures(makeCapture("left", 500), makeCapture("right", 500));
  console.log("OBSERVED", JSON.stringify(comparison));
  expect(comparison).toMatchObject({ interaction: "changed", process: "changed", first_divergence: { status: "found", dimension: "interaction", left_at_ms: 6, right_at_ms: 6 } });
});

it("keeps an earlier process difference ahead of the interaction", () => {
  const comparison = compareProcessCaptures(makeCapture("left", 4), makeCapture("right", 4));
  console.log("CONTROL", JSON.stringify(comparison));
  expect(comparison.first_divergence).toMatchObject({ status: "found", dimension: "process", left_at_ms: 4, right_at_ms: 4 });
});

The first case expects the interaction difference at 6 ms. The second case moves the process difference to 4 ms and checks that an earlier process observation still wins.

Expected behavior

The existing timestamp-based first-difference selection should consider the recorded dispatch time of an interaction. For the first case, first_divergence should identify the changed interaction outcome with left_at_ms: 6 and right_at_ms: 6. The control should continue to identify the process observation at 4 ms. These expectations concern recorded time and make no claim about causal order.

Actual behavior

The first case failed its assertion. The control passed. The source application-workflow run and the reduced domain run both returned the following result for the first case:

{"status":"changed","terminal":"unchanged","interaction":"changed","exit":"unchanged","filesystem":"unchanged","process":"changed","first_divergence":{"status":"found","dimension":"process","index":0,"left_at_ms":500,"right_at_ms":500,"left":{"at_ms":500,"pid":2,"parent_pid":1,"command":"worker-a","process_group_id":1,"session_id":1},"right":{"at_ms":500,"pid":2,"parent_pid":1,"command":"worker-b","process_group_id":1,"session_id":1}},"limitations":[]}

Each focused run exited with status 1, with one failed assertion and one passing control. All input captures passed the production parser. No trace specification, truncation, residual unknown or normalization mismatch was involved.

Evidence

Expected source: docs/process-capture.md:70-80 describes supported interaction/process comparison and difference locations; docs/process-capture.md:393-395 describes cross-dimension first-divergence localization. The concrete existing selection rule in src/domain/process/processComparison.ts:207-228 orders candidates by the smallest available side timestamp. src/domain/process/processObservation.ts:104-108 identifies dispatched_at_ms as available observation time.
Failure source: The executed source application-workflow and inline reduced test return process/500 for the interaction/6 case. src/domain/process/processComparison.ts:168-178 reads only at_ms; lines 198-199 therefore assign null times to the interaction candidate, which loses to a later timed candidate.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The contrary output and failing assertion come from independent local runs. The timestamp omission and selector behavior were also traced in source. The source application workflow and parsed domain comparison were executed; a compiled CLI subprocess and MCP transport were not run.

The interaction schema at src/domain/process/processCapture.ts:498-510 requires the dispatch timestamp. The producer at src/process/capture/ProcessCaptureJournal.ts:55-102 records that timestamp and supports both dispatched and failed outcomes for the same scheduled input. The source CLI workflow at src/application/process/ProcessCli.ts:99-125 validates the Evidence and capture before calling the comparison. Matching commitments and full coverage let these fixtures reach the affected selector.

Issue #724 discusses explanations of content, exit and timing facets. This report concerns the current first-difference selector using an interaction's recorded timestamp, with the existing dimension classifications preserved.

Related: #2616 concerns trace-derived changes assigned to equal observation dimensions. This case uses no trace specification and concerns selection of the first recorded difference.

Restoration check

Run the inline test through the production capture parser and comparison. The interaction/6 versus process/500 case must return interaction/6, while the process/4 control must retain process/4. The source CLI comparison workflow should preserve the same result when the captures are supplied as valid capture Evidence files. Both changed dimensions must remain changed, and the original observations must remain available.

Runtime or environment

Source revision: a2d8b94dd5. Tests used Node 24.18.0 and npm 11.16.0 with the repository's locked dependencies. The latest published package was rea-agents 6.4.0 when checked; that released runtime was not tested. Inputs were synthetic recorded process captures, with no engine or live target required.

Current-upstream source recheck at commit 3ae826ceba found the comparison selector, capture validation, interaction timestamp projection and direct CLI/MCP comparison consumers unchanged. Shared Evidence metadata and regular-file admission changed, but the report's canonical recorded-capture inputs remain admitted by their current rules. Runtime reproduction was not rerun at this revision.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: For the demonstrated compatible captures, callers receive a later observation as the first difference. This can misdirect comparison diagnosis, but the overall changed status, dimension statuses and recorded observations remain intact.
Workaround status: available
Workaround basis: Inspect the retained interaction outcome and dispatched_at_ms values alongside the process sample timestamps. The fixture Evidence retains those values, so manual inspection recovers the earlier recorded difference without modifying data or normalization.

Originally created by @coygeek on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2645 ## Summary When two compatible recorded captures differ in an interaction outcome at 6 ms and in a process observation at 500 ms, the comparison identifies the process observation as `first_divergence`. The earlier interaction is correctly classified as changed, but its recorded dispatch timestamp does not participate in the first-difference selection. This gives callers the wrong starting location for diagnosis while retaining the underlying observations. ## Steps to reproduce Use the source baseline and development toolchain stated below with its locked dependencies available. Save the following test as `src/domain/process/interaction-first-divergence.test.ts`, then run `npm run test:focused -- src/domain/process/interaction-first-divergence.test.ts`. The test uses owned synthetic recorded captures and does not launch a target. This reduced test was executed. A separate source application-workflow run also admitted canonical capture Evidence files and returned the same result through `compareProcessEvidenceFiles`. ```ts import { expect, it } from "vitest"; import { compareProcessCaptures } from "./processComparison.js"; import { parseProcessCapture } from "./processCaptureParsing.js"; import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "./processCaptureExample.js"; import { accountFullyObservedProcessCapture } from "./processCapture.fixture.js"; const makeCapture = (side: "left" | "right", processTime: number) => { const base = EMPTY_PROCESS_CAPTURE_EXAMPLE; const journal = [ { collection: "filesystem_checkpoints" as const, index: 0 }, ...(processTime < 6 ? [{ collection: "process_samples" as const, index: 0 }] : []), { collection: "interaction_events" as const, index: 0 }, ...(processTime >= 6 ? [{ collection: "process_samples" as const, index: 0 }] : []), { collection: "lifecycle" as const, index: 0 }, { collection: "lifecycle" as const, index: 1 }, { collection: "filesystem_checkpoints" as const, index: 1 }, ].map((entry, capture_order) => ({ ...entry, capture_order })); return parseProcessCapture(accountFullyObservedProcessCapture({ ...base, manifest: { ...base.manifest, completed_at: "2026-01-01T00:00:01.000Z" }, settlement: { ...base.settlement, elapsed_ms: 1000 }, filesystem_checkpoints: base.filesystem_checkpoints.map((c, i) => ({ ...c, at_ms: i === 0 ? 0 : 1000 })), interaction_events: [{ sequence: 0, scheduled_at_ms: 5, dispatched_at_ms: 6, type: "input", data: "hello", outcome: side === "left" ? "dispatched" : "failed" }], process_samples: [{ at_ms: processTime, pid: 2, parent_pid: 1, command: side === "left" ? "worker-a" : "worker-b", process_group_id: 1, session_id: 1 }], event_journal: journal, })); }; it("reports the earlier interaction difference", () => { const comparison = compareProcessCaptures(makeCapture("left", 500), makeCapture("right", 500)); console.log("OBSERVED", JSON.stringify(comparison)); expect(comparison).toMatchObject({ interaction: "changed", process: "changed", first_divergence: { status: "found", dimension: "interaction", left_at_ms: 6, right_at_ms: 6 } }); }); it("keeps an earlier process difference ahead of the interaction", () => { const comparison = compareProcessCaptures(makeCapture("left", 4), makeCapture("right", 4)); console.log("CONTROL", JSON.stringify(comparison)); expect(comparison.first_divergence).toMatchObject({ status: "found", dimension: "process", left_at_ms: 4, right_at_ms: 4 }); }); ``` The first case expects the interaction difference at 6 ms. The second case moves the process difference to 4 ms and checks that an earlier process observation still wins. ## Expected behavior The existing timestamp-based first-difference selection should consider the recorded dispatch time of an interaction. For the first case, `first_divergence` should identify the changed interaction outcome with `left_at_ms: 6` and `right_at_ms: 6`. The control should continue to identify the process observation at 4 ms. These expectations concern recorded time and make no claim about causal order. ## Actual behavior The first case failed its assertion. The control passed. The source application-workflow run and the reduced domain run both returned the following result for the first case: ```json {"status":"changed","terminal":"unchanged","interaction":"changed","exit":"unchanged","filesystem":"unchanged","process":"changed","first_divergence":{"status":"found","dimension":"process","index":0,"left_at_ms":500,"right_at_ms":500,"left":{"at_ms":500,"pid":2,"parent_pid":1,"command":"worker-a","process_group_id":1,"session_id":1},"right":{"at_ms":500,"pid":2,"parent_pid":1,"command":"worker-b","process_group_id":1,"session_id":1}},"limitations":[]} ``` Each focused run exited with status 1, with one failed assertion and one passing control. All input captures passed the production parser. No trace specification, truncation, residual unknown or normalization mismatch was involved. ## Evidence - Expected source: `docs/process-capture.md:70-80` describes supported interaction/process comparison and difference locations; `docs/process-capture.md:393-395` describes cross-dimension first-divergence localization. The concrete existing selection rule in `src/domain/process/processComparison.ts:207-228` orders candidates by the smallest available side timestamp. `src/domain/process/processObservation.ts:104-108` identifies `dispatched_at_ms` as available observation time. - Failure source: The executed source application-workflow and inline reduced test return process/500 for the interaction/6 case. `src/domain/process/processComparison.ts:168-178` reads only `at_ms`; lines 198-199 therefore assign null times to the interaction candidate, which loses to a later timed candidate. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The contrary output and failing assertion come from independent local runs. The timestamp omission and selector behavior were also traced in source. The source application workflow and parsed domain comparison were executed; a compiled CLI subprocess and MCP transport were not run. The interaction schema at `src/domain/process/processCapture.ts:498-510` requires the dispatch timestamp. The producer at `src/process/capture/ProcessCaptureJournal.ts:55-102` records that timestamp and supports both dispatched and failed outcomes for the same scheduled input. The source CLI workflow at `src/application/process/ProcessCli.ts:99-125` validates the Evidence and capture before calling the comparison. Matching commitments and full coverage let these fixtures reach the affected selector. [Issue #724](https://github.com/morluto/rea/issues/724) discusses explanations of content, exit and timing facets. This report concerns the current first-difference selector using an interaction's recorded timestamp, with the existing dimension classifications preserved. Related: [#2616](https://github.com/morluto/rea/issues/2616) concerns trace-derived changes assigned to equal observation dimensions. This case uses no trace specification and concerns selection of the first recorded difference. ## Restoration check Run the inline test through the production capture parser and comparison. The interaction/6 versus process/500 case must return interaction/6, while the process/4 control must retain process/4. The source CLI comparison workflow should preserve the same result when the captures are supplied as valid capture Evidence files. Both changed dimensions must remain changed, and the original observations must remain available. ## Runtime or environment Source revision: a2d8b94dd50dab8d966239dd63ab84718aa8b4b4. Tests used Node 24.18.0 and npm 11.16.0 with the repository's locked dependencies. The latest published package was rea-agents 6.4.0 when checked; that released runtime was not tested. Inputs were synthetic recorded process captures, with no engine or live target required. Current-upstream source recheck at commit 3ae826ceba6f25b08bce50bcb51001209bc16b57 found the comparison selector, capture validation, interaction timestamp projection and direct CLI/MCP comparison consumers unchanged. Shared Evidence metadata and regular-file admission changed, but the report's canonical recorded-capture inputs remain admitted by their current rules. Runtime reproduction was not rerun at this revision. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: For the demonstrated compatible captures, callers receive a later observation as the first difference. This can misdirect comparison diagnosis, but the overall changed status, dimension statuses and recorded observations remain intact. - Workaround status: available - Workaround basis: Inspect the retained interaction outcome and dispatched_at_ms values alongside the process sample timestamps. The fixture Evidence retains those values, so manual inspection recovers the earlier recorded difference without modifying data or normalization.

bsakaguchi

2026-10-15 18:29:48 +00:00