morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7294] [Bug] Reversing two failing process traces changes the attributed dimensions #4330

New issue

Closed

opened 2026-10-31 00:48:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:48:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 26, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7294

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

The public compare-process-captures CLI reports different changed dimensions for the same pair when the operand order is reversed. Both imported captures fail an explicit partial-order specification, but on different edges. In the forward comparison terminal/interaction are changed and filesystem/exit are unchanged; in reverse filesystem/exit are changed and terminal/interaction are unchanged. Both runs report trace different, both trace sides fail, and overall changed.

The per-source observations, normalization and comparison commitments are identical. Only cross-source journal order differs. The no-trace control reports all four dimensions unchanged. These are parser-validated, caller-authored synthetic capture Evidence files; this is an offline import/comparison reproduction, not a claim that a real capture producer emitted these journals.

Reproduction

Run the following from a prepared source checkout, with a supported Node.js on PATH, locked dependencies and its compiled dist/ runtime already present. Preparation was supplied for verification; installation and build were not rerun. The script derives three capture records from the public empty-capture example, writes neutral JSON fixtures into a disposable directory, and invokes the registered CLI six times with an explicit environment and isolated home/config/cache/state/temp paths. It launches no target or analysis provider. The fixture deliberately supplies two different journal orderings to the supported import boundary: left violates ready -> typed, right violates before -> settled; every other event pair is explicitly unordered.

node --input-type=module <<'JS'
import assert from "node:assert/strict";
import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import { spawnSync } from "node:child_process";
const repo = resolve(process.env.REA_REPO ?? process.cwd());
const work = process.env.REPRO_WORK ?? mkdtempSync(join(tmpdir(), "rea-trace-"));
const load = (file) => import(pathToFileURL(join(repo, "dist", file)).href);
const { EMPTY_PROCESS_CAPTURE_EXAMPLE: base } = await load("domain/process/processCaptureExample.js");
const { parseProcessCapture } = await load("domain/process/processCaptureParsing.js");
const { processTraceSpecificationSchema } = await load("domain/process/processTraceSpecification.js");
const { createEvidence } = await load("domain/evidence.js");
const { PROCESS_PROVIDER } = await load("domain/process/processEvidenceProvider.js");
const frames = [{ sequence: 0, at_ms: 10, data: "Ready" }];
const input = { sequence: 0, scheduled_at_ms: 10, dispatched_at_ms: 10, type: "input", data: "go", outcome: "dispatched" };
const fsBefore = base.filesystem_checkpoints[0];
const settlement = { event: "settlement", ...base.settlement };
const definitions = [
  ["ready", "terminal_raw", frames[0], "frames", 0],
  ["typed", "interaction", input, "interaction_events", 0],
  ["before", "filesystem", fsBefore, "filesystem_checkpoints", 0],
  ["settled", "lifecycle", settlement, "lifecycle", 1],
  ["exited", "lifecycle", { event: "exit", ...base.exit }, "lifecycle", 0],
  ["after", "filesystem", base.filesystem_checkpoints[1], "filesystem_checkpoints", 1],
];
const edges = [{ before: "ready", after: "typed" }, { before: "before", after: "settled" }];
const ids = definitions.map(([id]) => id);
const specification = processTraceSpecificationSchema.parse({
  events: definitions.map(([id, source, exact]) => ({ id, source, exact, cardinality: { kind: "required" } })),
  language: { kind: "partial_order", happens_before: edges, not_before: [], prefix: [], suffix: [],
    unordered_groups: ids.flatMap((id, i) => ids.slice(i + 1).filter((other) => !edges.some((e) => e.before === id && e.after === other)).map((other) => ({ events: [id, other] }))),
  },
});
const save = (file, value) => { writeFileSync(join(work, file), JSON.stringify(value)); return join(work, file); };
const make = (name, order) => {
  const capture = structuredClone(base);
  capture.frames = frames;
  capture.interaction_events = [input];
  capture.process_samples = [];
  capture.residual_unknowns = [{ scope: "process", reason: "Process trees are sampled and may omit short-lived descendants." }];
  const observations = order.map((id) => { const d = definitions.find(([key]) => key === id); return { collection: d[3], index: d[4] }; });
  capture.event_journal = observations.map((entry, capture_order) => ({ ...entry, capture_order }));
  const bytes = frames.reduce((n, frame) => n + Buffer.byteLength(frame.data), 0);
  Object.assign(capture.truncation_details.raw_terminal, { observed_bytes: bytes, retained_bytes: bytes, observed_frames: 1, retained_frames: 1 });
  capture.truncation_details.process.retained_samples = 0;
  return save(name + ".json", createEvidence(undefined, PROCESS_PROVIDER, {
    predicateType: "rea.process-capture", operation: "capture_process_scenario", parameters: { fixture: name },
    result: parseProcessCapture(capture), confidence: "observed", authority: "controlled-replay",
    environment: { id: "synthetic-fixture", platform: "fixture", architecture: "fixture", isolation: "process" },
  }));
};
const left = make("left", ["before", "typed", "ready", "exited", "settled", "after"]);
const right = make("right", ["ready", "typed", "exited", "settled", "before", "after"]);
const passing = make("passing", ["before", "ready", "typed", "exited", "settled", "after"]);
const tracePath = save("trace.json", specification);
const env = { PATH: dirname(process.execPath) + ":/usr/bin:/bin", NO_COLOR: "1" };
for (const key of ["HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_STATE_HOME", "TMPDIR"]) { env[key] = join(work, key.toLowerCase()); mkdirSync(env[key], { recursive: true }); }
const run = (label, a, b, traced = true) => {
  const args = [join(repo, "scripts/rea.mjs"), "compare-process-captures", a, b, ...(traced ? [tracePath] : []), "--format", "json"];
  const result = spawnSync(process.execPath, args, { env, cwd: work, encoding: "utf8", timeout: 30000 });
  assert.equal(result.status, 0, result.stdout + result.stderr);
  const evidence = JSON.parse(result.stdout);
  assert.equal(evidence.operation, "compare_process_captures");
  const c = evidence.normalized_result;
  const row = { label, status: c.status, terminal: c.terminal, interaction: c.interaction, exit: c.exit, filesystem: c.filesystem, process: c.process, trace: c.trace ?? null };
  save(label + "-result.json", evidence);
  console.log(JSON.stringify({ label, status: row.status, terminal: row.terminal, interaction: row.interaction, exit: row.exit, filesystem: row.filesystem, process: row.process, verdict: c.trace?.verdict, sides: c.trace && [c.trace.left.status, c.trace.right.status], diagnostic: c.trace?.diagnostic?.event_ids }));
  return row;
};
const forward = run("forward", left, right);
const reverse = run("reverse", right, left);
assert.equal(forward.trace.left.status, "fail"); assert.equal(forward.trace.right.status, "fail");
assert.equal(reverse.trace.left.status, "fail"); assert.equal(reverse.trace.right.status, "fail");
assert.equal(forward.interaction, "changed"); assert.equal(reverse.interaction, "unchanged");
assert.equal(forward.filesystem, "unchanged"); assert.equal(reverse.filesystem, "changed");
run("exact", left, right, false);
run("one-failure", left, passing);
run("one-failure-reversed", passing, left);
run("passing", passing, passing);
JS

The final replay completed with all six CLI children exiting 0. A separate source-loaded comparison returned exactly the same normalized results for both failing orientations and independently exposed each side's distinct edge diagnostic.

Expected behavior

Swapping the selected captures may relabel sides and choose a different headline diagnostic, but must preserve the set of changed dimensions for this pair. Both observed edge failures must contribute their dimensions: terminal, interaction, filesystem and exit changed in either orientation, with process remaining unknown from its sampling residual. The no-trace control must remain unchanged in those four dimensions. A pair with only the terminal/interaction edge failure must continue changing just those dimensions in either orientation.

docs/process-capture.md:529-537 says trace differences apply to implicated dimensions, unrelated dimensions retain their comparisons, and observed ordering violations remain changes. The partial-order regression at src/domain/process/processTraceComparison.test.ts:668-733 also requires attribution of every independent failed ordering edge. This report extends that existing rule to independent failures on opposite sides rather than losing one side during pair comparison.

Impact and scope

P2: a successful public comparison emits operand-dependent dimension results, suppressing a different pair of ordering changes in each orientation while retaining the correct aggregate changed verdict. The verified scope is the synthetic imported pair and explicit specification above. User prevalence, naturally recorded examples, other platforms and MCP transport were not measured. No security boundary is crossed.

A demonstrated workaround for this pair is to run both operand orders and take the union of their changed dimensions, retaining both results and the same trace specification. Omitting trace validation alone does not preserve the required ordering information. Source inspection shows that find_changed_behavior emits findings only for non-unchanged process dimensions (src/domain/changedBehavior.ts:222-252) and dimension-specific verify_reconstruction claims read these statuses (src/domain/reconstructionVerification.ts:282-287); those downstream workflows were not executed here.

REA version

rea-agents 6.8.0; source revision 0d1a454e25db825567782e461da21530bad4b158.

Runtime and platform

Node.js v24.22.1; macOS arm64. The compiled CLI and direct source comparison were checked against the same checkout; no native analysis engine was used.

Target and artifact context

Synthetic process-capture Evidence based on src/domain/process/processCaptureExample.ts, with one raw terminal frame, one input interaction, two lifecycle observations and two empty filesystem checkpoints. Both journals reference every observation exactly once with contiguous capture order. The captures have matching canonical commitments, no truncation, equal observation values and a process sampling residual outside the trace-selected sources. The arbitrary imported journal orders are intentional; no live process recording is claimed.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Selected fields from the real CLI output; all six calls exited 0:

Run	terminal	interaction	exit	filesystem	process	trace verdict	trace sides	diagnostic event IDs
forward	changed	changed	unchanged	unchanged	unknown	different	fail / fail	ready, typed
reverse	unchanged	unchanged	changed	changed	unknown	different	fail / fail	before, settled
exact, without trace	unchanged	unchanged	unchanged	unchanged	unknown	absent	absent	absent
one-failure	changed	changed	unchanged	unchanged	unknown	different	fail / pass	ready, typed
one-failure-reversed	changed	changed	unchanged	unchanged	unknown	different	pass / fail	ready, typed
passing	unchanged	unchanged	unchanged	unchanged	unknown	equivalent	pass / pass	absent

The forward and reverse overall statuses are changed. The exact and passing controls have overall unknown because process coverage remains unknown. The source-loaded side evaluations returned ready must happen before typed. for left and before must happen before settled. for right. The compiled CLI's normalized result equaled the direct current-source comparison in each orientation.

Expected source: docs/process-capture.md:529-537 and src/domain/process/processTraceComparison.test.ts:668-733, requiring dimension-specific attribution while retaining independent observed ordering failures.
Failure source: the six executed CLI results above; src/domain/process/processTraceComparison.ts:20-27 evaluates both sides but retains the left diagnostic when both exist, and src/domain/process/processComparison.ts:339-365 attributes changes using only that retained diagnostic.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Public reachability was exercised through src/cli/processCommands.ts:28-52 and src/application/process/ProcessCli.ts:107-183: file reads, Evidence integrity/workflow/provider admission, capture semantic validation and trace-specification parsing precede compareProcessCaptureEvidence. That owner invokes compareProcessCaptures, whose compatible-contract guard passes for these equal commitments; no capture-age restriction is requested. Both side evaluations are fail, so unknown/equivalent/nonconforming branches do not explain the result. There is no test-only bypass at the CLI boundary.

Typed outcome or recovery expected

Return derived comparison Evidence with the same four changed dimensions in both operand orders, preserving trace verdict different, both side failures and process unknown. A single presentation diagnostic can remain side-specific without discarding the other side's dimension attribution.

Analysis or suspected cause

Source-derived and corroborated by direct side evaluation: both failures exist before pair construction. Pair construction prefers the left diagnostic, while dimension projection derives its event/source set solely from trace.diagnostic. In the forward order this set contains terminal/interaction; reversing the files replaces it with filesystem/lifecycle. Each excluded dimension falls back to its unchanged exact observation comparison. The raw observations are equal, but their explicitly constrained cross-source order is not.

Additional context

This is one comparison-attribution defect, independent of rendered-terminal truncation: neither input is truncated and both trace sides fail rather than pass. NOT_ENOUGH_INFO: last-known-good release, live-producer frequency, effects of other multi-failure shapes and executed downstream consumer behavior were not established. No implementation change or post-fix restoration is claimed. The final source-freshness comparison found later documentation/CI and DMG-recovery changes on live main. The trace/comparison implementation, capture/trace validators, CLI command and issue form were unchanged. The shared ArtifactReader changes only add/preserve failure-recovery user messages; they do not change this successful JSON-read path. Current main was source-compared, not rebuilt or runtime-tested.

References

#2616, closed as completed, and #2929, closed, concern false attribution of a predicate mismatch to equal unrelated dimensions. A review on that PR also identified independent ordering failures within one side; the current evaluator aggregates those failures at src/domain/process/processTraceEvaluation.ts:398-419. This case instead loses a failure across sides when both sides fail differently, after each side has been evaluated correctly.

#3931, closed as completed, and #4178, closed, preserve unrelated incomplete coverage. This fixture has no truncation in any dimension and its trace-selected sources are complete.

#6514, closed as completed, and #3181, closed as completed, concern finite-language omissions; this report uses complete equal observation collections and partial-order edge failures. #5923, closed as completed, concerns first-divergence localization, a separate result field. #724 remains open for additive content/timing explanations and explicitly preserves existing verdict semantics; it does not report this two-failure operand-order dependence.

Restoration check

Replay the same pair and specification in both orientations. Require terminal/interaction/filesystem/exit changed, process unknown, trace different and both trace sides fail in each. Preserve the exact control's four unchanged dimensions and the passing trace control. Preserve one-failure and one-failure-reversed as terminal/interaction changed with exit/filesystem unchanged, so marking every declared dimension changed cannot satisfy the check. The script records/asserts current failing behavior; restoration requires replacing its asymmetry assertions with the symmetric expected outcomes above.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The executed offline CLI returns different changed-dimension sets for the same admitted pair when the operands are reversed; the correct aggregate changed result and both trace failures survive, bounding the observed impact to comparison attribution.
Workaround status: available
Workaround basis: Both operand orders were executed with the same captures and trace; combining their changed sets recovers all four implicated dimensions for this pair without changing inputs, relaxing the language or discarding trace validation.

Originally created by @coygeek on GitHub (Oct 26, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7294 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Process capture or runtime observation ## Observed behavior The public `compare-process-captures` CLI reports different changed dimensions for the same pair when the operand order is reversed. Both imported captures fail an explicit partial-order specification, but on different edges. In the forward comparison terminal/interaction are changed and filesystem/exit are unchanged; in reverse filesystem/exit are changed and terminal/interaction are unchanged. Both runs report trace `different`, both trace sides `fail`, and overall `changed`. The per-source observations, normalization and comparison commitments are identical. Only cross-source journal order differs. The no-trace control reports all four dimensions unchanged. These are parser-validated, caller-authored synthetic capture Evidence files; this is an offline import/comparison reproduction, not a claim that a real capture producer emitted these journals. ## Reproduction Run the following from a prepared source checkout, with a supported Node.js on PATH, locked dependencies and its compiled `dist/` runtime already present. Preparation was supplied for verification; installation and build were not rerun. The script derives three capture records from the public empty-capture example, writes neutral JSON fixtures into a disposable directory, and invokes the registered CLI six times with an explicit environment and isolated home/config/cache/state/temp paths. It launches no target or analysis provider. The fixture deliberately supplies two different journal orderings to the supported import boundary: left violates `ready -> typed`, right violates `before -> settled`; every other event pair is explicitly unordered. ```sh node --input-type=module <<'JS' import assert from "node:assert/strict"; import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { pathToFileURL } from "node:url"; import { spawnSync } from "node:child_process"; const repo = resolve(process.env.REA_REPO ?? process.cwd()); const work = process.env.REPRO_WORK ?? mkdtempSync(join(tmpdir(), "rea-trace-")); const load = (file) => import(pathToFileURL(join(repo, "dist", file)).href); const { EMPTY_PROCESS_CAPTURE_EXAMPLE: base } = await load("domain/process/processCaptureExample.js"); const { parseProcessCapture } = await load("domain/process/processCaptureParsing.js"); const { processTraceSpecificationSchema } = await load("domain/process/processTraceSpecification.js"); const { createEvidence } = await load("domain/evidence.js"); const { PROCESS_PROVIDER } = await load("domain/process/processEvidenceProvider.js"); const frames = [{ sequence: 0, at_ms: 10, data: "Ready" }]; const input = { sequence: 0, scheduled_at_ms: 10, dispatched_at_ms: 10, type: "input", data: "go", outcome: "dispatched" }; const fsBefore = base.filesystem_checkpoints[0]; const settlement = { event: "settlement", ...base.settlement }; const definitions = [ ["ready", "terminal_raw", frames[0], "frames", 0], ["typed", "interaction", input, "interaction_events", 0], ["before", "filesystem", fsBefore, "filesystem_checkpoints", 0], ["settled", "lifecycle", settlement, "lifecycle", 1], ["exited", "lifecycle", { event: "exit", ...base.exit }, "lifecycle", 0], ["after", "filesystem", base.filesystem_checkpoints[1], "filesystem_checkpoints", 1], ]; const edges = [{ before: "ready", after: "typed" }, { before: "before", after: "settled" }]; const ids = definitions.map(([id]) => id); const specification = processTraceSpecificationSchema.parse({ events: definitions.map(([id, source, exact]) => ({ id, source, exact, cardinality: { kind: "required" } })), language: { kind: "partial_order", happens_before: edges, not_before: [], prefix: [], suffix: [], unordered_groups: ids.flatMap((id, i) => ids.slice(i + 1).filter((other) => !edges.some((e) => e.before === id && e.after === other)).map((other) => ({ events: [id, other] }))), }, }); const save = (file, value) => { writeFileSync(join(work, file), JSON.stringify(value)); return join(work, file); }; const make = (name, order) => { const capture = structuredClone(base); capture.frames = frames; capture.interaction_events = [input]; capture.process_samples = []; capture.residual_unknowns = [{ scope: "process", reason: "Process trees are sampled and may omit short-lived descendants." }]; const observations = order.map((id) => { const d = definitions.find(([key]) => key === id); return { collection: d[3], index: d[4] }; }); capture.event_journal = observations.map((entry, capture_order) => ({ ...entry, capture_order })); const bytes = frames.reduce((n, frame) => n + Buffer.byteLength(frame.data), 0); Object.assign(capture.truncation_details.raw_terminal, { observed_bytes: bytes, retained_bytes: bytes, observed_frames: 1, retained_frames: 1 }); capture.truncation_details.process.retained_samples = 0; return save(name + ".json", createEvidence(undefined, PROCESS_PROVIDER, { predicateType: "rea.process-capture", operation: "capture_process_scenario", parameters: { fixture: name }, result: parseProcessCapture(capture), confidence: "observed", authority: "controlled-replay", environment: { id: "synthetic-fixture", platform: "fixture", architecture: "fixture", isolation: "process" }, })); }; const left = make("left", ["before", "typed", "ready", "exited", "settled", "after"]); const right = make("right", ["ready", "typed", "exited", "settled", "before", "after"]); const passing = make("passing", ["before", "ready", "typed", "exited", "settled", "after"]); const tracePath = save("trace.json", specification); const env = { PATH: dirname(process.execPath) + ":/usr/bin:/bin", NO_COLOR: "1" }; for (const key of ["HOME", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_STATE_HOME", "TMPDIR"]) { env[key] = join(work, key.toLowerCase()); mkdirSync(env[key], { recursive: true }); } const run = (label, a, b, traced = true) => { const args = [join(repo, "scripts/rea.mjs"), "compare-process-captures", a, b, ...(traced ? [tracePath] : []), "--format", "json"]; const result = spawnSync(process.execPath, args, { env, cwd: work, encoding: "utf8", timeout: 30000 }); assert.equal(result.status, 0, result.stdout + result.stderr); const evidence = JSON.parse(result.stdout); assert.equal(evidence.operation, "compare_process_captures"); const c = evidence.normalized_result; const row = { label, status: c.status, terminal: c.terminal, interaction: c.interaction, exit: c.exit, filesystem: c.filesystem, process: c.process, trace: c.trace ?? null }; save(label + "-result.json", evidence); console.log(JSON.stringify({ label, status: row.status, terminal: row.terminal, interaction: row.interaction, exit: row.exit, filesystem: row.filesystem, process: row.process, verdict: c.trace?.verdict, sides: c.trace && [c.trace.left.status, c.trace.right.status], diagnostic: c.trace?.diagnostic?.event_ids })); return row; }; const forward = run("forward", left, right); const reverse = run("reverse", right, left); assert.equal(forward.trace.left.status, "fail"); assert.equal(forward.trace.right.status, "fail"); assert.equal(reverse.trace.left.status, "fail"); assert.equal(reverse.trace.right.status, "fail"); assert.equal(forward.interaction, "changed"); assert.equal(reverse.interaction, "unchanged"); assert.equal(forward.filesystem, "unchanged"); assert.equal(reverse.filesystem, "changed"); run("exact", left, right, false); run("one-failure", left, passing); run("one-failure-reversed", passing, left); run("passing", passing, passing); JS ``` The final replay completed with all six CLI children exiting 0. A separate source-loaded comparison returned exactly the same normalized results for both failing orientations and independently exposed each side's distinct edge diagnostic. ## Expected behavior Swapping the selected captures may relabel sides and choose a different headline diagnostic, but must preserve the set of changed dimensions for this pair. Both observed edge failures must contribute their dimensions: terminal, interaction, filesystem and exit changed in either orientation, with process remaining unknown from its sampling residual. The no-trace control must remain unchanged in those four dimensions. A pair with only the terminal/interaction edge failure must continue changing just those dimensions in either orientation. `docs/process-capture.md:529-537` says trace differences apply to implicated dimensions, unrelated dimensions retain their comparisons, and observed ordering violations remain changes. The partial-order regression at `src/domain/process/processTraceComparison.test.ts:668-733` also requires attribution of every independent failed ordering edge. This report extends that existing rule to independent failures on opposite sides rather than losing one side during pair comparison. ## Impact and scope P2: a successful public comparison emits operand-dependent dimension results, suppressing a different pair of ordering changes in each orientation while retaining the correct aggregate changed verdict. The verified scope is the synthetic imported pair and explicit specification above. User prevalence, naturally recorded examples, other platforms and MCP transport were not measured. No security boundary is crossed. A demonstrated workaround for this pair is to run both operand orders and take the union of their changed dimensions, retaining both results and the same trace specification. Omitting trace validation alone does not preserve the required ordering information. Source inspection shows that `find_changed_behavior` emits findings only for non-unchanged process dimensions (`src/domain/changedBehavior.ts:222-252`) and dimension-specific `verify_reconstruction` claims read these statuses (`src/domain/reconstructionVerification.ts:282-287`); those downstream workflows were not executed here. ## REA version rea-agents 6.8.0; source revision `0d1a454e25db825567782e461da21530bad4b158`. ## Runtime and platform Node.js v24.22.1; macOS arm64. The compiled CLI and direct source comparison were checked against the same checkout; no native analysis engine was used. ## Target and artifact context Synthetic process-capture Evidence based on `src/domain/process/processCaptureExample.ts`, with one raw terminal frame, one input interaction, two lifecycle observations and two empty filesystem checkpoints. Both journals reference every observation exactly once with contiguous capture order. The captures have matching canonical commitments, no truncation, equal observation values and a process sampling residual outside the trace-selected sources. The arbitrary imported journal orders are intentional; no live process recording is claimed. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence Selected fields from the real CLI output; all six calls exited 0: | Run | terminal | interaction | exit | filesystem | process | trace verdict | trace sides | diagnostic event IDs | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | forward | changed | changed | unchanged | unchanged | unknown | different | fail / fail | ready, typed | | reverse | unchanged | unchanged | changed | changed | unknown | different | fail / fail | before, settled | | exact, without trace | unchanged | unchanged | unchanged | unchanged | unknown | absent | absent | absent | | one-failure | changed | changed | unchanged | unchanged | unknown | different | fail / pass | ready, typed | | one-failure-reversed | changed | changed | unchanged | unchanged | unknown | different | pass / fail | ready, typed | | passing | unchanged | unchanged | unchanged | unchanged | unknown | equivalent | pass / pass | absent | The forward and reverse overall statuses are changed. The exact and passing controls have overall unknown because process coverage remains unknown. The source-loaded side evaluations returned `ready must happen before typed.` for left and `before must happen before settled.` for right. The compiled CLI's normalized result equaled the direct current-source comparison in each orientation. - Expected source: `docs/process-capture.md:529-537` and `src/domain/process/processTraceComparison.test.ts:668-733`, requiring dimension-specific attribution while retaining independent observed ordering failures. - Failure source: the six executed CLI results above; `src/domain/process/processTraceComparison.ts:20-27` evaluates both sides but retains the left diagnostic when both exist, and `src/domain/process/processComparison.ts:339-365` attributes changes using only that retained diagnostic. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete Public reachability was exercised through `src/cli/processCommands.ts:28-52` and `src/application/process/ProcessCli.ts:107-183`: file reads, Evidence integrity/workflow/provider admission, capture semantic validation and trace-specification parsing precede `compareProcessCaptureEvidence`. That owner invokes `compareProcessCaptures`, whose compatible-contract guard passes for these equal commitments; no capture-age restriction is requested. Both side evaluations are fail, so unknown/equivalent/nonconforming branches do not explain the result. There is no test-only bypass at the CLI boundary. ## Typed outcome or recovery expected Return derived comparison Evidence with the same four changed dimensions in both operand orders, preserving trace verdict different, both side failures and process unknown. A single presentation diagnostic can remain side-specific without discarding the other side's dimension attribution. ## Analysis or suspected cause Source-derived and corroborated by direct side evaluation: both failures exist before pair construction. Pair construction prefers the left diagnostic, while dimension projection derives its event/source set solely from `trace.diagnostic`. In the forward order this set contains terminal/interaction; reversing the files replaces it with filesystem/lifecycle. Each excluded dimension falls back to its unchanged exact observation comparison. The raw observations are equal, but their explicitly constrained cross-source order is not. ## Additional context This is one comparison-attribution defect, independent of rendered-terminal truncation: neither input is truncated and both trace sides fail rather than pass. NOT_ENOUGH_INFO: last-known-good release, live-producer frequency, effects of other multi-failure shapes and executed downstream consumer behavior were not established. No implementation change or post-fix restoration is claimed. The final source-freshness comparison found later documentation/CI and DMG-recovery changes on live main. The trace/comparison implementation, capture/trace validators, CLI command and issue form were unchanged. The shared ArtifactReader changes only add/preserve failure-recovery user messages; they do not change this successful JSON-read path. Current main was source-compared, not rebuilt or runtime-tested. ## References [#2616](https://github.com/morluto/rea/issues/2616), closed as completed, and [#2929](https://github.com/morluto/rea/pull/2929), closed, concern false attribution of a predicate mismatch to equal unrelated dimensions. A review on that PR also identified independent ordering failures within one side; the current evaluator aggregates those failures at `src/domain/process/processTraceEvaluation.ts:398-419`. This case instead loses a failure across sides when both sides fail differently, after each side has been evaluated correctly. [#3931](https://github.com/morluto/rea/issues/3931), closed as completed, and [#4178](https://github.com/morluto/rea/pull/4178), closed, preserve unrelated incomplete coverage. This fixture has no truncation in any dimension and its trace-selected sources are complete. [#6514](https://github.com/morluto/rea/issues/6514), closed as completed, and [#3181](https://github.com/morluto/rea/issues/3181), closed as completed, concern finite-language omissions; this report uses complete equal observation collections and partial-order edge failures. [#5923](https://github.com/morluto/rea/issues/5923), closed as completed, concerns first-divergence localization, a separate result field. [#724](https://github.com/morluto/rea/issues/724) remains open for additive content/timing explanations and explicitly preserves existing verdict semantics; it does not report this two-failure operand-order dependence. ## Restoration check Replay the same pair and specification in both orientations. Require terminal/interaction/filesystem/exit changed, process unknown, trace different and both trace sides fail in each. Preserve the exact control's four unchanged dimensions and the passing trace control. Preserve one-failure and one-failure-reversed as terminal/interaction changed with exit/filesystem unchanged, so marking every declared dimension changed cannot satisfy the check. The script records/asserts current failing behavior; restoration requires replacing its asymmetry assertions with the symmetric expected outcomes above. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The executed offline CLI returns different changed-dimension sets for the same admitted pair when the operands are reversed; the correct aggregate changed result and both trace failures survive, bounding the observed impact to comparison attribution. - Workaround status: available - Workaround basis: Both operand orders were executed with the same captures and trace; combining their changed sets recovers all four implicated dimensions for this pair without changing inputs, relaxing the language or discarding trace validation.

bsakaguchi closed this issue

2026-10-31 00:48:58 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:53 +00:00

[GH-ISSUE #3582] [Tracker] Ready to close or merge: issues and pull requests (October 2026) #1330

bsakaguchi referenced this issue

2026-11-01 21:10:54 +00:00

[GH-ISSUE #3795] [Tracker] Open documentation issues and pull requests (October 2026) #1468

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue