morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #2616] fix(process): trace mismatch reports changes in an equal observation dimension #931

New issue

Closed

opened 2026-10-15 18:29:41 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:29:41 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2616

Summary

Comparing two compatible captures with a trace specification can report a change in an observation dimension whose values are identical. In the reproduced case, the captures differ only in terminal text and the specification declares terminal and process events. The comparison correctly detects the terminal difference but also reports the identical process samples as changed. A process-only difference produces the reverse misattribution. This makes the per-dimension result unreliable for locating the changed behavior.

Steps to reproduce

Use the source baseline and installed dependencies described under Runtime or environment. The following domain test uses the repository's validated capture fixture, which supplies matching comparison commitments, complete event journals, and no truncation or residual unknowns. Both captures contain the same process sample and event order. The only differing observation is the terminal frame's text.

Save the following test as src/domain/process/traceDimensionRegression.test.ts in a disposable checkout.

import { expect, it } from "vitest";
import { compareProcessCaptures } from "./processComparison.js";
import {
  capture,
  partialSpecification,
  processStarted,
  terminal,
} from "./processTraceComparison.fixture.js";

it("attributes a terminal mismatch to the terminal dimension", () => {
  const makeCapture = (data: string) => capture({
    frames: [{ ...terminal, data }],
    process_samples: [processStarted],
    event_journal: [
      { collection: "frames", index: 0 },
      { collection: "process_samples", index: 0 },
    ],
  });
  const left = makeCapture("Ready");
  const right = makeCapture("Different");
  const exact = compareProcessCaptures(left, right);
  const traced = compareProcessCaptures(left, right, {
    traceSpecification: partialSpecification(),
  });
  expect(exact).toMatchObject({
    terminal: "changed", process: "unchanged",
  });
  expect(traced.trace).toMatchObject({ verdict: "different" });
  expect(traced).toMatchObject({
    terminal: "changed", process: "unchanged",
  });
});

Run the focused test.

npm run test:focused -- src/domain/process/traceDimensionRegression.test.ts

Observe that the exact comparison assertion passes and the final assertion fails because the traced comparison returns process: "changed".

The verifier executed this capture construction and comparison sequence in a temporary domain test, together with the reverse and identical-input controls. The filename above is a suggested reproduction filename rather than an existing committed regression test.

Expected behavior

The comparison should attribute changes to the dimensions whose observations or applicable ordering constraints differ. When both captures have identical process samples in the same journal positions and differ only in terminal text, process should remain unchanged and terminal should be changed. The combined trace should still be different because the right terminal event fails its declared exact predicate.

Actual behavior

The focused reproduction returned the following dimension results. The final process assertion failed and the focused command exited with code 1.

Without trace specification: terminal=changed, process=unchanged
With trace specification:    terminal=changed, process=changed
Trace: different; left=pass, right=fail
First divergence: terminal frame 0, Ready versus Different

The reverse control retained identical terminal frames and changed only the process command from worker to other-worker. The exact result was terminal unchanged and process changed, but the traced result marked both changed. Comparing an identical capture to itself with the same specification passed with terminal and process unchanged and trace equivalent. The overall changed status and first divergence remained correct for the differing-input cases.

Runtime or environment

Failure verified at a2d8b94dd50dab8d966239dd63ab84718aa8b4b4 and still present at 8b181716aca2cb467be499cba226abfb5ae19141. Package version: 6.4.0. Node.js: v26.10.0. npm: 11.19.1. Vitest: 5.0.3. Dependencies were already installed from the repository lockfile. Verification ran the source domain comparison with synthetic parsed captures on macOS; compiled CLI and MCP delivery were inspected in source rather than executed.

Evidence

Expected source: docs/process-capture.md:76-87 documents comparison of each observation dimension under a shared contract, observed differences with locations, and optional exact-event trace specifications. src/domain/process/processComparison.ts:59-99 defines separate dimension results, and its comparison contract comment at line 400 distinguishes changes in an individual dimension.
Failure source: The focused domain probe observed an extra changed process dimension for equal process samples. src/domain/process/processComparison.ts:294-329 projects the combined trace verdict onto covered dimensions; the different-verdict branch at line 315 returns changed before the per-dimension coverage check. src/domain/process/processTraceComparison.ts:29-47 establishes the different verdict when one side passes and the other fails, and src/domain/process/processTraceDimensionProjection.ts:12-23 maps the declared terminal and process sources to both dimensions.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

The terminal mismatch is an admitted observation that fails the trace predicate, rather than invalid capture data or an invalid specification. The shared comparison commitments pass the compatibility guard, no capture-age restriction is requested, both journals are complete, and neither side carries relevant unknowns. One side passes and the other fails, so neither the nonconforming nor unknown trace guard applies. The false process change follows the global different-verdict branch.

Source inspection shows that the CLI and MCP adapters call the same comparison. The MCP scope question includes every non-unchanged dimension, so the extra changed process dimension would also appear among disagreeing scopes. This downstream result was not executed; the implementation creates an aggregate contradiction question rather than a separate contradiction for each dimension.

Related reference: the open comparison-explanation discussion includes another contributor's observation of a terminal mismatch also changing equal lifecycle observations. That discussion proposes additional explanations; this report concerns the reproduced attribution failure in existing dimension results.

Restoration check

Repeat the terminal-only input above. The traced result must retain terminal changed, process unchanged, a different trace verdict, and terminal as the first divergence. Repeat the process-only control with the terminal frame unchanged and the process command changed to other-worker; terminal must remain unchanged and process must be changed. An identical-capture comparison must remain unchanged in both dimensions with an equivalent trace. Preserve genuine ordering differences under explicit trace constraints and do not suppress the aggregate different verdict to make these checks pass.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The reproduced result assigns a change to identical observations and can misdirect analysis of which behavior changed. The aggregate changed result remains correct, and the defect affects comparisons that request a trace specification covering multiple dimensions.
Workaround status: available
Workaround basis: For the reproduced inputs, omitting the trace specification directly returned the correct per-dimension observation comparison. Callers can retain that exact comparison alongside the traced result when they also need trace conformance; the additional comparison does not replace trace-order analysis.

Originally created by @coygeek on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2616 ## Summary Comparing two compatible captures with a trace specification can report a change in an observation dimension whose values are identical. In the reproduced case, the captures differ only in terminal text and the specification declares terminal and process events. The comparison correctly detects the terminal difference but also reports the identical process samples as changed. A process-only difference produces the reverse misattribution. This makes the per-dimension result unreliable for locating the changed behavior. ## Steps to reproduce Use the source baseline and installed dependencies described under Runtime or environment. The following domain test uses the repository's validated capture fixture, which supplies matching comparison commitments, complete event journals, and no truncation or residual unknowns. Both captures contain the same process sample and event order. The only differing observation is the terminal frame's text. 1. Save the following test as `src/domain/process/traceDimensionRegression.test.ts` in a disposable checkout. ```ts import { expect, it } from "vitest"; import { compareProcessCaptures } from "./processComparison.js"; import { capture, partialSpecification, processStarted, terminal, } from "./processTraceComparison.fixture.js"; it("attributes a terminal mismatch to the terminal dimension", () => { const makeCapture = (data: string) => capture({ frames: [{ ...terminal, data }], process_samples: [processStarted], event_journal: [ { collection: "frames", index: 0 }, { collection: "process_samples", index: 0 }, ], }); const left = makeCapture("Ready"); const right = makeCapture("Different"); const exact = compareProcessCaptures(left, right); const traced = compareProcessCaptures(left, right, { traceSpecification: partialSpecification(), }); expect(exact).toMatchObject({ terminal: "changed", process: "unchanged", }); expect(traced.trace).toMatchObject({ verdict: "different" }); expect(traced).toMatchObject({ terminal: "changed", process: "unchanged", }); }); ``` 2. Run the focused test. ```sh npm run test:focused -- src/domain/process/traceDimensionRegression.test.ts ``` 3. Observe that the exact comparison assertion passes and the final assertion fails because the traced comparison returns `process: "changed"`. The verifier executed this capture construction and comparison sequence in a temporary domain test, together with the reverse and identical-input controls. The filename above is a suggested reproduction filename rather than an existing committed regression test. ## Expected behavior The comparison should attribute changes to the dimensions whose observations or applicable ordering constraints differ. When both captures have identical process samples in the same journal positions and differ only in terminal text, process should remain unchanged and terminal should be changed. The combined trace should still be different because the right terminal event fails its declared exact predicate. ## Actual behavior The focused reproduction returned the following dimension results. The final process assertion failed and the focused command exited with code 1. ```text Without trace specification: terminal=changed, process=unchanged With trace specification: terminal=changed, process=changed Trace: different; left=pass, right=fail First divergence: terminal frame 0, Ready versus Different ``` The reverse control retained identical terminal frames and changed only the process command from `worker` to `other-worker`. The exact result was terminal unchanged and process changed, but the traced result marked both changed. Comparing an identical capture to itself with the same specification passed with terminal and process unchanged and trace equivalent. The overall changed status and first divergence remained correct for the differing-input cases. ## Runtime or environment Failure verified at `a2d8b94dd50dab8d966239dd63ab84718aa8b4b4` and still present at `8b181716aca2cb467be499cba226abfb5ae19141`. Package version: 6.4.0. Node.js: v26.10.0. npm: 11.19.1. Vitest: 5.0.3. Dependencies were already installed from the repository lockfile. Verification ran the source domain comparison with synthetic parsed captures on macOS; compiled CLI and MCP delivery were inspected in source rather than executed. ## Evidence - Expected source: `docs/process-capture.md:76-87` documents comparison of each observation dimension under a shared contract, observed differences with locations, and optional exact-event trace specifications. `src/domain/process/processComparison.ts:59-99` defines separate dimension results, and its comparison contract comment at line 400 distinguishes changes in an individual dimension. - Failure source: The focused domain probe observed an extra changed process dimension for equal process samples. `src/domain/process/processComparison.ts:294-329` projects the combined trace verdict onto covered dimensions; the different-verdict branch at line 315 returns changed before the per-dimension coverage check. `src/domain/process/processTraceComparison.ts:29-47` establishes the different verdict when one side passes and the other fails, and `src/domain/process/processTraceDimensionProjection.ts:12-23` maps the declared terminal and process sources to both dimensions. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete The terminal mismatch is an admitted observation that fails the trace predicate, rather than invalid capture data or an invalid specification. The shared comparison commitments pass the compatibility guard, no capture-age restriction is requested, both journals are complete, and neither side carries relevant unknowns. One side passes and the other fails, so neither the nonconforming nor unknown trace guard applies. The false process change follows the global different-verdict branch. Source inspection shows that the CLI and MCP adapters call the same comparison. The MCP scope question includes every non-unchanged dimension, so the extra changed process dimension would also appear among disagreeing scopes. This downstream result was not executed; the implementation creates an aggregate contradiction question rather than a separate contradiction for each dimension. Related reference: [the open comparison-explanation discussion](https://github.com/morluto/rea/issues/724) includes another contributor's observation of a terminal mismatch also changing equal lifecycle observations. That discussion proposes additional explanations; this report concerns the reproduced attribution failure in existing dimension results. ## Restoration check Repeat the terminal-only input above. The traced result must retain terminal changed, process unchanged, a different trace verdict, and terminal as the first divergence. Repeat the process-only control with the terminal frame unchanged and the process command changed to `other-worker`; terminal must remain unchanged and process must be changed. An identical-capture comparison must remain unchanged in both dimensions with an equivalent trace. Preserve genuine ordering differences under explicit trace constraints and do not suppress the aggregate different verdict to make these checks pass. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The reproduced result assigns a change to identical observations and can misdirect analysis of which behavior changed. The aggregate changed result remains correct, and the defect affects comparisons that request a trace specification covering multiple dimensions. - Workaround status: available - Workaround basis: For the reproduced inputs, omitting the trace specification directly returned the correct per-dimension observation comparison. Callers can retain that exact comparison alongside the traced result when they also need trace conformance; the additional comparison does not replace trace-order analysis.

bsakaguchi

2026-10-15 18:29:41 +00:00

closed this issue
added the
area: runtime

bug
labels

bsakaguchi commented

2026-10-15 18:29:42 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Process-trace comparison now attributes a difference only to observation dimensions that differ, while preserving aggregate trace mismatch and explicit ordering attribution. The dimension-isolation regression and controls pass on current main.

Root cause: A globally different trace verdict was projected as changed onto every covered dimension, including dimensions with equal observations.

Validation: src/domain/process/processTraceComparison.test.ts: 'keeps equal dimensions unchanged when one side fails a predicate' covers terminal-only and process-only changes; neighboring ordering test verifies both dimensions change for an ordering difference. These assertions passed in CI at 500e32399; the reviewed implementation is present on current main. Fix: 599a8ee11de6b1a7fb270e669b88d78b6be982d2.

Verification scope: Synthetic process captures only; adapter delivery was source-reviewed, not separately invoked.

@morluto commented on GitHub (Oct 14, 2026): Process-trace comparison now attributes a difference only to observation dimensions that differ, while preserving aggregate trace mismatch and explicit ordering attribution. The dimension-isolation regression and controls pass on current main. Root cause: A globally different trace verdict was projected as changed onto every covered dimension, including dimensions with equal observations. Validation: src/domain/process/processTraceComparison.test.ts: 'keeps equal dimensions unchanged when one side fails a predicate' covers terminal-only and process-only changes; neighboring ordering test verifies both dimensions change for an ordering difference. These assertions passed in [CI at 500e32399](https://github.com/morluto/rea/actions/runs/38514789942); the reviewed implementation is present on current main. Fix: [599a8ee11de6b1a7fb270e669b88d78b6be982d2](https://github.com/morluto/rea/commit/599a8ee11de6b1a7fb270e669b88d78b6be982d2). Verification scope: Synthetic process captures only; adapter delivery was source-reviewed, not separately invoked.

bsakaguchi referenced this issue

2026-10-17 21:06:57 +00:00

[PR #931] [MERGED] fix(apple): pair bridge hypotheses within one application root #2508

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#931

No description provided.

Rows
Columns