morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3181] [Bug] A finite-trace omission marks identical terminal observations as changed #1118

New issue

Closed

opened 2026-10-15 18:31:13 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:13 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3181

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

compare-process-captures left.json right.json finite.json --json reports terminal: changed when the captures have identical terminal frames and differ only by a process sample present on the right. The finite specification accepts only [ready, worker]. The left capture correctly fails that language and the right passes, but the finite-mismatch diagnostic identifies the matching ready terminal event rather than the omitted process event. The comparison exits with code 0 and reports overall changed, process changed, and trace different. Its first divergence correctly identifies process sample 0.

Both captures have complete event journals, matching normalization and comparison commitments, no truncation, and no residual unknowns. The process predicate has optional cardinality, which permits evaluation to reach the finite language. That cardinality does not make the process event optional in the sole declared finite variant. No pair of shared events changes relative order in this omission case.

Reproduction

Start in a clean checkout of 58300885207133422af235e0f2b10658cf958b03 with the repository's supported toolchain. Prepare dependencies and the compiled runtime:

npm ci
npm run build:cached

The verifier used the prepared dependencies and compiled runtime at this baseline. These installation and build prerequisites were not rerun during verification.

Create the fixture generator inline. It uses the public empty-capture example and production parser to create the capture Evidence and specifications:

cat > repro-inputs.mjs <<'JS'
import { writeFileSync } from 'node:fs';
import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from './dist/domain/process/processCaptureExample.js';
import { parseProcessCapture } from './dist/domain/process/processCaptureParsing.js';
import { createEvidence } from './dist/domain/evidence.js';
import { PROCESS_PROVIDER } from './dist/domain/process/processEvidenceProvider.js';

const terminal = { sequence: 0, at_ms: 900, data: 'Ready' };
const worker = { at_ms: 1, pid: 1, parent_pid: 0, command: 'worker', process_group_id: 1, session_id: 1 };
function capture(samples, data = 'Ready', reverse = false) {
  const value = structuredClone(EMPTY_PROCESS_CAPTURE_EXAMPLE);
  value.frames = [{ ...terminal, data }];
  value.process_samples = samples;
  const bytes = Buffer.byteLength(data);
  Object.assign(value.truncation_details.raw_terminal, { observed_bytes: bytes, retained_bytes: bytes, observed_frames: 1, retained_frames: 1 });
  value.truncation_details.process.retained_samples = samples.length;
  const journal = [{ collection: 'frames', index: 0 }, ...samples.map((_, index) => ({ collection: 'process_samples', index }))];
  if (reverse) journal.reverse();
  journal.push({collection: 'lifecycle', index: 0}, {collection: 'lifecycle', index: 1}, {collection: 'filesystem_checkpoints', index: 0}, {collection: 'filesystem_checkpoints', index: 1});
  value.event_journal = journal.map((entry, capture_order) => ({ capture_order, ...entry }));
  const result = parseProcessCapture(value);
  return createEvidence(undefined, PROCESS_PROVIDER, { predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: { executable_name: 'node', argument_count: 0, event_count: 0, filesystem_observation_path_count: 0 }, result, confidence: 'observed', authority: 'controlled-replay', limitations: [] });
}
const events = [
  { id: 'ready', source: 'terminal_raw', exact: { data: 'Ready' }, ignore_fields: ['sequence', 'at_ms'], cardinality: { kind: 'required' } },
  { id: 'worker', source: 'process', exact: { pid: 1, parent_pid: 0, command: 'worker', process_group_id: 1, session_id: 1 }, ignore_fields: ['at_ms'], cardinality: { kind: 'optional' } }
];
const fixtures = {
  'left.json': capture([]),
  'right.json': capture([worker]),
  'changed-terminal.json': capture([worker], 'Different'),
  'reordered.json': capture([worker], 'Ready', true),
  'finite.json': { events, language: {kind: 'finite_traces', variants: [{id: 'both', trace: ['ready', 'worker']}] } },
  'partial.json': { events, language: {kind: 'partial_order', happens_before: [], not_before: [], unordered_groups: [{ events: ['ready', 'worker']}], prefix: [], suffix: [] } }
};
for (const [name, value] of Object.entries(fixtures)) writeFileSync(name, JSON.stringify(value));
console.log(JSON.stringify({equal_terminal: JSON.stringify(fixtures['left.json'].normalized_result.frames) === JSON.stringify(fixtures['right.json'].normalized_result.frames), files: Object.keys(fixtures)}));
JS
node repro-inputs.mjs

Compare the omission inputs against the finite specification:

node scripts/rea.mjs compare-process-captures left.json right.json finite.json --json

Observe terminal: changed despite the generator reporting equal_terminal: true.

Compare the same capture Evidence without a trace specification:

node scripts/rea.mjs compare-process-captures left.json right.json --json

This observed control reports terminal unchanged, process added, and overall changed.

Check the true terminal-change and cross-source ordering controls:

node scripts/rea.mjs compare-process-captures right.json changed-terminal.json finite.json --json
node scripts/rea.mjs compare-process-captures right.json reordered.json finite.json --json

The terminal-change control reports terminal changed, process unchanged, and trace different. The ordering control reports terminal and process changed, with trace different. All comparison commands exited with code 0 in verification.

Expected behavior

docs/process-capture.md:76-87 documents comparison of the five observation dimensions with difference locations, plus an explicit optional trace specification. The attribution rule in src/domain/process/processComparison.ts:332-333 says a different verdict should name the failing events and observations so equal unrelated observations retain their exact status.

For the omission inputs, terminal must remain unchanged, process must remain changed, overall status must remain changed, and trace.verdict must remain different. The first divergence must still identify the process sample. Preserve the true terminal-change and ordering controls. Equal per-source collections can legitimately be classified changed for an ordering difference involving both dimensions; this report concerns the omission case without such a shared-event ordering change.

Impact and scope

The reproduced error falsely identifies an additional affected dimension and can misdirect investigation of which behavior differs. The observed scope is a valid finite-trace omission comparison using locally constructed capture Evidence. The overall changed result and actual process divergence remain available, and no user data is modified or lost. No security boundary is crossed.

A demonstrated workaround is to retain the finite comparison for conformance and run a second comparison without a trace specification to inspect exact per-dimension observations. The first divergence is also useful in this case, but it cannot replace all dimension statuses in general. The generated partial.json is only a diagnostic control. It was observed to return trace equivalent and all dimensions unchanged because that different language accepts the omission; changing to it would alter the caller's accepted language.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node 26.10.0; macOS 27.0.1 arm64

Target and artifact context

Locally constructed canonical process-capture Evidence based on EMPTY_PROCESS_CAPTURE_EXAMPLE, with one identical raw terminal frame per capture and a process sample present only on the right. No analysis provider, PTY, or target execution is required.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

The failure first occurs in domain comparison and finite-trace evaluation after the CLI accepts the Evidence and specification. It precedes result projection.

Relevant output or evidence

Fixture assertion: equal_terminal=true
Finite comparison: exit=0, status=changed, terminal=changed, process=changed
first_divergence={"status":"found","dimension":"process","index":0}
trace.verdict=different; left=fail; right=pass
trace.diagnostic={"kind":"trace","side":"left","message":"Observed events do not match any finite trace variant.","event_ids":["ready"],"locations":[{"collection":"frames","index":0,"capture_order":0}]}
Exact control: terminal=unchanged, process=added, status=changed
Terminal-change control: terminal=changed, process=unchanged, trace=different
Ordering control: terminal=changed, process=changed, trace=different
Existing focused trace-comparison tests: 10 passed

Typed outcome or recovery expected

Return comparison Evidence that keeps terminal unchanged, process and overall status changed, and trace verdict different for the unchanged omission inputs. Preserve the real process divergence and finite-language conformance failure. A failure diagnostic should identify the events or constraints responsible for the mismatch without attributing the matching terminal event as a terminal behavior change.

Analysis or suspected cause

Source-derived at baseline 58300885207133422af235e0f2b10658cf958b03: src/domain/process/processTraceEvaluation.ts:393-400 constructs a finite-language mismatch diagnostic from every matched event ID and location, including the unchanged terminal. src/domain/process/processComparison.ts:334-355 maps those diagnostic sources to dimensions and changes the otherwise exact-unchanged terminal status. The supported command is registered at src/cli/processCommands.ts:42, and src/application/process/ProcessCli.ts:99-125 parses the capture Evidence and specification before invoking the comparison.

src/domain/process/processTraceComparison.test.ts:305 covers unrelated-dimension preservation for predicate failures. Its neighboring ordering test permits both participating dimensions to change when their order differs. The independently executed omission comparison supplies the missing finite-language case; all 10 existing tests passed during verification.

Additional context

Related issues #2616 and #724 concern predicate-based trace attribution and broader content/timing explanations, respectively; this report concerns a remaining finite-variant omission diagnostic that falsely implicates the unchanged terminal even with the predicate-attribution change already present at the verified baseline.

P2 is supported by the bounded false attribution, retained correct overall/process result, and demonstrated second-comparison workaround that preserves the original finite-language conformance result.

Related: #3180 reports a separate failure in the same area; the two have independent fixes.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3181 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `compare-process-captures left.json right.json finite.json --json` reports `terminal: changed` when the captures have identical terminal frames and differ only by a process sample present on the right. The finite specification accepts only `[ready, worker]`. The left capture correctly fails that language and the right passes, but the finite-mismatch diagnostic identifies the matching `ready` terminal event rather than the omitted process event. The comparison exits with code 0 and reports overall `changed`, process `changed`, and trace `different`. Its first divergence correctly identifies process sample 0. Both captures have complete event journals, matching normalization and comparison commitments, no truncation, and no residual unknowns. The process predicate has optional cardinality, which permits evaluation to reach the finite language. That cardinality does not make the process event optional in the sole declared finite variant. No pair of shared events changes relative order in this omission case. ### Reproduction 1. Start in a clean checkout of `58300885207133422af235e0f2b10658cf958b03` with the repository's supported toolchain. Prepare dependencies and the compiled runtime: ```sh npm ci npm run build:cached ``` The verifier used the prepared dependencies and compiled runtime at this baseline. These installation and build prerequisites were not rerun during verification. 2. Create the fixture generator inline. It uses the public empty-capture example and production parser to create the capture Evidence and specifications: ```sh cat > repro-inputs.mjs <<'JS' import { writeFileSync } from 'node:fs'; import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from './dist/domain/process/processCaptureExample.js'; import { parseProcessCapture } from './dist/domain/process/processCaptureParsing.js'; import { createEvidence } from './dist/domain/evidence.js'; import { PROCESS_PROVIDER } from './dist/domain/process/processEvidenceProvider.js'; const terminal = { sequence: 0, at_ms: 900, data: 'Ready' }; const worker = { at_ms: 1, pid: 1, parent_pid: 0, command: 'worker', process_group_id: 1, session_id: 1 }; function capture(samples, data = 'Ready', reverse = false) { const value = structuredClone(EMPTY_PROCESS_CAPTURE_EXAMPLE); value.frames = [{ ...terminal, data }]; value.process_samples = samples; const bytes = Buffer.byteLength(data); Object.assign(value.truncation_details.raw_terminal, { observed_bytes: bytes, retained_bytes: bytes, observed_frames: 1, retained_frames: 1 }); value.truncation_details.process.retained_samples = samples.length; const journal = [{ collection: 'frames', index: 0 }, ...samples.map((_, index) => ({ collection: 'process_samples', index }))]; if (reverse) journal.reverse(); journal.push({collection: 'lifecycle', index: 0}, {collection: 'lifecycle', index: 1}, {collection: 'filesystem_checkpoints', index: 0}, {collection: 'filesystem_checkpoints', index: 1}); value.event_journal = journal.map((entry, capture_order) => ({ capture_order, ...entry })); const result = parseProcessCapture(value); return createEvidence(undefined, PROCESS_PROVIDER, { predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: { executable_name: 'node', argument_count: 0, event_count: 0, filesystem_observation_path_count: 0 }, result, confidence: 'observed', authority: 'controlled-replay', limitations: [] }); } const events = [ { id: 'ready', source: 'terminal_raw', exact: { data: 'Ready' }, ignore_fields: ['sequence', 'at_ms'], cardinality: { kind: 'required' } }, { id: 'worker', source: 'process', exact: { pid: 1, parent_pid: 0, command: 'worker', process_group_id: 1, session_id: 1 }, ignore_fields: ['at_ms'], cardinality: { kind: 'optional' } } ]; const fixtures = { 'left.json': capture([]), 'right.json': capture([worker]), 'changed-terminal.json': capture([worker], 'Different'), 'reordered.json': capture([worker], 'Ready', true), 'finite.json': { events, language: {kind: 'finite_traces', variants: [{id: 'both', trace: ['ready', 'worker']}] } }, 'partial.json': { events, language: {kind: 'partial_order', happens_before: [], not_before: [], unordered_groups: [{ events: ['ready', 'worker']}], prefix: [], suffix: [] } } }; for (const [name, value] of Object.entries(fixtures)) writeFileSync(name, JSON.stringify(value)); console.log(JSON.stringify({equal_terminal: JSON.stringify(fixtures['left.json'].normalized_result.frames) === JSON.stringify(fixtures['right.json'].normalized_result.frames), files: Object.keys(fixtures)})); JS node repro-inputs.mjs ``` 3. Compare the omission inputs against the finite specification: ```sh node scripts/rea.mjs compare-process-captures left.json right.json finite.json --json ``` Observe `terminal: changed` despite the generator reporting `equal_terminal: true`. 4. Compare the same capture Evidence without a trace specification: ```sh node scripts/rea.mjs compare-process-captures left.json right.json --json ``` This observed control reports terminal `unchanged`, process `added`, and overall `changed`. 5. Check the true terminal-change and cross-source ordering controls: ```sh node scripts/rea.mjs compare-process-captures right.json changed-terminal.json finite.json --json node scripts/rea.mjs compare-process-captures right.json reordered.json finite.json --json ``` The terminal-change control reports terminal `changed`, process `unchanged`, and trace `different`. The ordering control reports terminal and process `changed`, with trace `different`. All comparison commands exited with code 0 in verification. ### Expected behavior `docs/process-capture.md:76-87` documents comparison of the five observation dimensions with difference locations, plus an explicit optional trace specification. The attribution rule in `src/domain/process/processComparison.ts:332-333` says a different verdict should name the failing events and observations so equal unrelated observations retain their exact status. For the omission inputs, terminal must remain `unchanged`, process must remain `changed`, overall status must remain `changed`, and `trace.verdict` must remain `different`. The first divergence must still identify the process sample. Preserve the true terminal-change and ordering controls. Equal per-source collections can legitimately be classified changed for an ordering difference involving both dimensions; this report concerns the omission case without such a shared-event ordering change. ### Impact and scope The reproduced error falsely identifies an additional affected dimension and can misdirect investigation of which behavior differs. The observed scope is a valid finite-trace omission comparison using locally constructed capture Evidence. The overall changed result and actual process divergence remain available, and no user data is modified or lost. No security boundary is crossed. A demonstrated workaround is to retain the finite comparison for conformance and run a second comparison without a trace specification to inspect exact per-dimension observations. The first divergence is also useful in this case, but it cannot replace all dimension statuses in general. The generated `partial.json` is only a diagnostic control. It was observed to return trace `equivalent` and all dimensions `unchanged` because that different language accepts the omission; changing to it would alter the caller's accepted language. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node 26.10.0; macOS 27.0.1 arm64 ### Target and artifact context Locally constructed canonical process-capture Evidence based on `EMPTY_PROCESS_CAPTURE_EXAMPLE`, with one identical raw terminal frame per capture and a process sample present only on the right. No analysis provider, PTY, or target execution is required. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction The failure first occurs in domain comparison and finite-trace evaluation after the CLI accepts the Evidence and specification. It precedes result projection. ### Relevant output or evidence ```text Fixture assertion: equal_terminal=true Finite comparison: exit=0, status=changed, terminal=changed, process=changed first_divergence={"status":"found","dimension":"process","index":0} trace.verdict=different; left=fail; right=pass trace.diagnostic={"kind":"trace","side":"left","message":"Observed events do not match any finite trace variant.","event_ids":["ready"],"locations":[{"collection":"frames","index":0,"capture_order":0}]} Exact control: terminal=unchanged, process=added, status=changed Terminal-change control: terminal=changed, process=unchanged, trace=different Ordering control: terminal=changed, process=changed, trace=different Existing focused trace-comparison tests: 10 passed ``` ### Typed outcome or recovery expected Return comparison Evidence that keeps terminal `unchanged`, process and overall status `changed`, and trace verdict `different` for the unchanged omission inputs. Preserve the real process divergence and finite-language conformance failure. A failure diagnostic should identify the events or constraints responsible for the mismatch without attributing the matching terminal event as a terminal behavior change. ### Analysis or suspected cause Source-derived at baseline `58300885207133422af235e0f2b10658cf958b03`: `src/domain/process/processTraceEvaluation.ts:393-400` constructs a finite-language mismatch diagnostic from every matched event ID and location, including the unchanged terminal. `src/domain/process/processComparison.ts:334-355` maps those diagnostic sources to dimensions and changes the otherwise exact-unchanged terminal status. The supported command is registered at `src/cli/processCommands.ts:42`, and `src/application/process/ProcessCli.ts:99-125` parses the capture Evidence and specification before invoking the comparison. `src/domain/process/processTraceComparison.test.ts:305` covers unrelated-dimension preservation for predicate failures. Its neighboring ordering test permits both participating dimensions to change when their order differs. The independently executed omission comparison supplies the missing finite-language case; all 10 existing tests passed during verification. ### Additional context Related issues #2616 and #724 concern predicate-based trace attribution and broader content/timing explanations, respectively; this report concerns a remaining finite-variant omission diagnostic that falsely implicates the unchanged terminal even with the predicate-attribution change already present at the verified baseline. P2 is supported by the bounded false attribution, retained correct overall/process result, and demonstrated second-comparison workaround that preserves the original finite-language conformance result. Related: #3180 reports a separate failure in the same area; the two have independent fixes.

bsakaguchi

2026-10-15 18:31:13 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:14 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Delivered on main in fffb07419.

Finite-trace omission diagnostics attributed every observed event to the mismatch, contaminating unchanged dimensions. Evaluation now attributes missing/unexpected members once, including unordered multiset counts and actual locations.

Validation: Process trace/comparison regressions for process-only divergence and unchanged unrelated dimensions. Full exact-revision qualification passed.

@morluto commented on GitHub (Oct 14, 2026): Delivered on main in [fffb07419](https://github.com/morluto/rea/commit/fffb0741936067530d1ce9777461cd53e87398c2). Finite-trace omission diagnostics attributed every observed event to the mismatch, contaminating unchanged dimensions. Evaluation now attributes missing/unexpected members once, including unordered multiset counts and actual locations. Validation: Process trace/comparison regressions for process-only divergence and unchanged unrelated dimensions. Full exact-revision [qualification](https://github.com/morluto/rea/actions/runs/38629313022) passed.

bsakaguchi referenced this issue

2026-10-17 21:08:46 +00:00

[PR #1118] [MERGED] fix(hopper): preserve owned basic-block endpoint instructions #2663

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1118

No description provided.

Rows
Columns