morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7295] [Bug] A passing raw-only trace erases uncertainty from omitted rendered terminal frames #4331

New issue

Closed

opened 2026-10-31 00:48:58 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:48:58 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 26, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7295

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

compare-process-captures changes terminal from unknown to unchanged when a passing raw-only trace is added to captures whose rendered snapshots were omitted. The raw trace correctly reports equivalent; it does not cover the omitted rendered observations. The comparison's overall status and first_divergence.status remain unknown.

The incorrect dimension result survives serialization and affects reconstruction verification: importing the captures and CLI comparison as an Evidence bundle through MCP, then calling verify_reconstruction with dimension: "terminal", returns claim and overall pass. Selecting dimension: "overall" instead returns unknown.

Reproduction

The following fixture-backed reproduction was executed against the prepared baseline checkout, using its locked dependencies and existing dist/; no build or installation was rerun. It invokes the actual terminal renderer and bounded normalizer to produce omitted and complete rendered observations, wraps them in validated synthetic capture Evidence, and compares files through the registered CLI. It does not launch a target or record a live PTY session.

Run from the checkout root. The script writes only to a disposable directory, isolates the CLI's home/config/cache/state/temp directories, and generates bundle.json containing the two omitted-rendered captures and their actual traced CLI comparison.

export REA_REPO="$PWD"
export REPRO_DIR="$(mktemp -d)"
node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import {mkdirSync, readFileSync, writeFileSync} from 'node:fs';
import {join, dirname, resolve} from 'node:path';
import {pathToFileURL} from 'node:url';
import {spawnSync} from 'node:child_process';

const repo = resolve(process.env.REA_REPO);
const work = resolve(process.env.REPRO_DIR);
const load = path => import(pathToFileURL(join(repo, 'dist', path)));
const {EMPTY_PROCESS_CAPTURE_EXAMPLE: example} = await load('domain/process/processCaptureExample.js');
const {parseProcessCapture} = await load('domain/process/processCaptureParsing.js');
const {hasCaptureTruncation} = await load('domain/process/processCaptureCoverage.js');
const {processScenarioSchema} = await load('domain/process/processScenario.js');
const {createBoundedProcessTextNormalizer} = await load('process/capture/ProcessNormalization.js');
const {TerminalRenderer} = await load('process/capture/TerminalRenderer.js');
const {createEvidence} = await load('domain/evidence.js');
const {PROCESS_PROVIDER} = await load('domain/process/processEvidenceProvider.js');
const frame = {sequence: 0, at_ms: 10, data: 'Ready'};
const scenario = processScenarioSchema.parse({executable: '/fixture/program', working_directory: '/fixture', limits: {output_bytes: 5}, terminal: {columns: 5, rows: 1, scrollback: 0}});
const normalize = createBoundedProcessTextNormalizer(scenario, '/fixture/temporary', 100);
async function observe(budget) {
  const renderer = new TerminalRenderer({columns: 5, rows: 1, scrollback: 0, maxBytes: budget, normalize});
  renderer.write(frame.data, frame.at_ms);
  const frames = await renderer.frames();
  const retention = renderer.retention();
  await renderer.dispose();
  return {frames, retention};
}
const omitted = await observe(5);
const complete = await observe(100);
assert.equal(omitted.frames.length, 0);
assert.equal(complete.frames.length, 1);
console.log('producer', JSON.stringify(omitted));
const residuals = [
  {scope: 'process', reason: 'Process trees are sampled and may omit short-lived descendants.'},
  {scope: 'process', reason: 'The executable digest is a prelaunch file sample; operating-system image binding is not atomically observed.'},
  {scope: 'environment', reason: 'Inherited host environment variables are not recorded.'},
  {scope: 'network', reason: 'Network activity is not observed by this capture.'}
];
function capture(observation, rawOmitted = false) {
  const c = structuredClone(example);
  c.frames = rawOmitted ? [] : [frame];
  c.rendered_frames = rawOmitted ? [] : observation.frames;
  c.truncation_details.raw_terminal = {budget_bytes: rawOmitted ? 1 : observation.retention.budget_bytes, observed_bytes: 5, retained_bytes: rawOmitted ? 0 : 5, observed_frames: 1, retained_frames: rawOmitted ? 0 : 1};
  c.truncation_details.rendered_terminal = rawOmitted ? {budget_bytes: 1, observed_bytes: 0, retained_bytes: 0, observed_frames: 0, retained_frames: 0} : observation.retention;
  c.truncated = hasCaptureTruncation(c.truncation_details);
  c.residual_unknowns = residuals;
  c.limitations = c.truncated ? [rawOmitted ? 'Raw PTY chunks exceeded output_bytes and were omitted; rendered states also lack those inputs. See truncation_details.raw_terminal.' : 'Cumulative rendered state and visible-line bytes exceeded output_bytes; whole rendered frames were omitted. Raw PTY coverage is reported separately. See truncation_details.rendered_terminal.'] : [];
  const entries = [['filesystem_checkpoints', 0], ...c.frames.map((_, i) => ['frames', i]), ...c.rendered_frames.map((_, i) => ['rendered_frames', i]), ['lifecycle', 0], ['lifecycle', 1], ['filesystem_checkpoints', 1]];
  c.event_journal = entries.map(([collection, index], capture_order) => ({collection, index, capture_order}));
  return parseProcessCapture(c);
}
function evidence(name, value) {
  return createEvidence(undefined, PROCESS_PROVIDER, {predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: {fixture: name}, result: value, confidence: 'observed', authority: 'controlled-replay', environment: {id: 'fixture', platform: 'fixture', architecture: 'fixture', isolation: 'process'}, limitations: value.limitations});
}
const save = (name, value) => {writeFileSync(join(work, name), JSON.stringify(value, null, 2)); return name;};
for (const [name, observation, rawOmitted] of [['omitted', omitted, false], ['complete', complete, false], ['raw-omitted', omitted, true]]) {
  for (const side of ['left', 'right']) save(`${name}-${side}.json`, evidence(`${name}-${side}`, capture(observation, rawOmitted)));
}
save('trace.json', {events: [{id: 'ready', source: 'terminal_raw', exact: frame, cardinality: {kind: 'required'}}], language: {kind: 'finite_traces', variants: [{id: 'ready-only', trace: ['ready']}]}});
const env = {PATH: `${dirname(process.execPath)}:/usr/bin:/bin`, NO_COLOR: '1'};
for (const key of ['HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_STATE_HOME', 'TMPDIR']) {env[key] = join(work, key.toLowerCase()); mkdirSync(env[key], {recursive: true});}
function compare(name, traced) {
  const run = spawnSync(process.execPath, [join(repo, 'scripts/rea.mjs'), 'compare-process-captures', `${name}-left.json`, `${name}-right.json`, ...(traced ? ['trace.json'] : []), '--format', 'json'], {cwd: work, env, encoding: 'utf8', timeout: 20000});
  assert.equal(run.status, 0, run.stdout + run.stderr);
  const value = JSON.parse(run.stdout);
  save(`${name}-${traced ? 'traced' : 'exact'}-comparison.json`, value);
  const r = value.normalized_result;
  console.log(JSON.stringify({input: name, traced, terminal: r.terminal, trace: r.trace?.verdict ?? null, status: r.status, first_divergence: r.first_divergence.status}));
  return value;
}
assert.equal(compare('omitted', false).normalized_result.terminal, 'unknown');
assert.equal(compare('omitted', true).normalized_result.terminal, 'unchanged');
assert.equal(compare('complete', true).normalized_result.terminal, 'unchanged');
assert.equal(compare('raw-omitted', true).normalized_result.terminal, 'unknown');

const {createEvidenceBundle} = await load('domain/evidenceBundle.js');
const records = ['omitted-left.json', 'omitted-right.json', 'omitted-traced-comparison.json'].map(name => JSON.parse(readFileSync(join(work, name), 'utf8')));
save('bundle.json', createEvidenceBundle(records));
console.log('comparison_evidence_id', records[2].evidence_id);
JS

For the downstream check, call MCP import_evidence_bundle with the absolute path to the generated bundle.json. Then call verify_reconstruction with the request below, substituting the comparison_evidence_id printed by the script. This route was independently executed against the production MCP server over linked in-memory SDK transports, using an empty analysis-provider registry. Import and both verification calls succeeded without opening a binary or launching an analysis provider.

{
  "specification": {
    "name": "Terminal coverage",
    "claims": [{
      "kind": "behavioral",
      "claim_id": "terminal",
      "title": "Replay remains equivalent",
      "comparison_evidence_id": "<printed comparison_evidence_id>",
      "dimension": "terminal"
    }]
  }
}

The terminal claim returns pass; changing the claim ID and dimension to overall returns unknown. The public fixture script was replayed successfully, and the imported-bundle MCP check was rerun using those newly generated records.

Expected behavior

A complete raw-terminal assertion may pass even when rendered snapshots are omitted. The terminal comparison dimension must nevertheless remain unknown for those omitted observations, and a terminal-scoped reconstruction claim must not pass on that dimension result. Complete rendered coverage must still permit terminal unchanged; actual raw truncation must keep the raw assertion unknown.

Impact and scope

P2: this is a bounded result-integrity defect for complete raw chunks with omitted rendered snapshots and an equivalent raw-only trace. The observed consequence includes a false terminal-scoped reconstruction pass. The aggregate comparison remains unknown, the input Evidence remains intact, and the comparison retains the omission limitation. Population frequency and live-capture/platform prevalence were not measured. No security boundary is involved.

A verified workaround is to preserve a second comparison without the trace specification: the same omitted-rendered inputs then retain terminal unknown. Keep the original trace result separately to preserve its raw conformance information. Inspecting the retained truncation metadata also exposes the gap. The higher-budget renderer control retained its snapshot, but recapturing an arbitrary live workload with a larger budget was not tested.

REA version

6.8.0; source revision 0d1a454e25db825567782e461da21530bad4b158. The submission freshness comparison against upstream main 3233992733c33dabf2de09d22ee4e6d9409f797a found no changes in the cited process comparison, renderer or reconstruction semantics, their contracts, documentation or dependencies. Other desktop, artifact, cleanup, browser and CI changes were inspected separately. The runtime reproduction was not rerun at that submission head. 6.8.0 was the latest published release checked.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1 arm64. Existing compiled runtime and locked dependencies; no Hopper, Ghidra, IDA, debugger, or external analysis service used.

Target and artifact context

Synthetic process-capture Evidence based on the public empty-capture example, with one raw Ready frame and renderer-produced retention metadata. Two distinct capture Evidence records share an explicit fixture environment and controlled-replay authority. The rendered producer is real; the surrounding captures are fixtures, not observations of an executed program.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

All four CLI subprocesses exited 0. The renderer and CLI results were:

producer: frames=[]; retention={"budget_bytes":5,"observed_bytes":0,"retained_bytes":0,"observed_frames":1,"retained_frames":0,"unknown_size_frames":1}
omitted, no trace: terminal=unknown; trace absent; status=unknown; first_divergence=unknown
omitted, raw-only trace: terminal=unchanged; trace=equivalent; status=unknown; first_divergence=unknown
complete, raw-only trace: terminal=unchanged; trace=equivalent; status=unknown; first_divergence=unknown
raw-omitted, raw-only trace: terminal=unknown; trace=unknown; status=unknown; first_divergence=unknown
MCP terminal claim: status=pass; claim=pass; observed_status=unchanged
MCP overall claim: status=unknown; claim=unknown; observed_status=unknown

The omitted-rendered captures have truncated: true, complete raw retention, and no terminal-scope residual unknown, matching the producer's deliberate filtering of terminal coverage reasons. Their limitation says: Cumulative rendered state and visible-line bytes exceeded output_bytes; whole rendered frames were omitted. Raw PTY coverage is reported separately. See truncation_details.rendered_terminal. The comparison retains that limitation and the first-divergence reason Incomplete observations prevent locating the first divergence; unaffected dimensions retain their comparisons.

Expected source: docs/process-capture.md:80-95,526-537 requires affected incomplete dimensions to remain unknown and allows trace assertions to use only their selected sources' coverage; src/process/capture/TerminalRenderer.ts:32-37 explicitly retains both representations because neither subsumes the other.
Failure source: The independently executed registered CLI and imported-bundle MCP results above; src/domain/process/processTraceDimensionProjection.ts:21-46 checks only retained arrays when deciding terminal coverage, and src/domain/process/processComparison.ts:366-373 uses that answer to substitute the equivalent trace's dimension status.
Evidence provenance: observed; the source explanation and producer reachability below are separately source-derived.
Local verification: reproduced through serialized Evidence at the CLI and MCP boundaries; real target capture and stdio transport were not exercised.
Reproduction completeness: complete for the fixture-backed supported comparison/import/verification route.

Typed outcome or recovery expected

For the omitted-rendered case, retain trace.verdict: "equivalent", terminal unknown, overall unknown, and unknown first divergence. A terminal reconstruction claim based on that result must remain unknown. Keep the complete-coverage control unchanged and raw-truncated trace control unknown.

Analysis or suspected cause

Source-derived: the exact comparison recognizes rendered truncation through processSourceTruncated, but trace dimension coverage tests retained rendered_frames.length rather than producer-reported omissions. An empty retained array therefore makes a raw-only specification appear to cover the terminal dimension. The passing trace's unchanged status then replaces the exact unknown result. src/domain/reconstructionVerification.ts:282-287 consumes the selected dimension directly.

Producer reachability is supported by the executed TerminalRenderer and bounded normalizer plus the production wiring: ProcessHarness.ts:240-253 supplies limits.output_bytes as the rendered budget, and ProcessCaptureLifecycle.ts:433-435 excludes terminal coverage reasons from residual_unknowns while retaining their accounting and limitations. Five retained raw bytes can exhaust the independent rendered budget because rendered retention counts visible lines plus serialized state. The reproduction uses the actual returned zero-byte/unknown-size omission accounting, rather than inventing an observed rendered byte count.

Additional context

Prepared with AI assistance and independently verified. Last known good version: NOT_ENOUGH_INFO; no regression boundary was established. This report concerns an existing terminal-dimension status, not a request for new timing explanations or separate comparison facets.

References

Closed #3931 and its merged fix #4178 addressed a failing trace promoting unrelated unknown terminal coverage to changed. This current case instead uses an equivalent trace to promote affected unknown coverage to unchanged; the existing fix is present in the tested baseline.
Closed #3180 addressed complete filesystem traces retaining ignored timestamp differences. Here the missing rendered coverage must block terminal equivalence.
Open #724 proposes additive content/outcome/timing explanations while preserving statuses. Its discussion does not report this passing raw-only trace erasing rendered-truncation uncertainty.

Restoration check

Replay the same omitted-rendered Evidence and raw-only specification: raw trace equivalence must survive, terminal must stay unknown, and terminal-scoped reconstruction verification must return unknown after serialized bundle import. The no-trace case must keep terminal unknown, the complete rendered control must keep terminal unchanged, and raw-omitted input must keep its trace and terminal unknown. Preserve overall status, first-divergence uncertainty, retained observations, omission metadata and Evidence provenance.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: The executed CLI falsely reports terminal unchanged for omitted rendered observations, and the imported serialized comparison produces a terminal-scoped MCP reconstruction pass. The aggregate comparison and overall verification retain unknown, limiting the observed failure's scope.
Workaround status: available
Workaround basis: The executed no-trace comparison of the same Evidence keeps terminal unknown; retaining it alongside the original trace result preserves both coverage uncertainty and the requested raw conformance evidence.

Originally created by @coygeek on GitHub (Oct 26, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7295 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior `compare-process-captures` changes `terminal` from `unknown` to `unchanged` when a passing raw-only trace is added to captures whose rendered snapshots were omitted. The raw trace correctly reports `equivalent`; it does not cover the omitted rendered observations. The comparison's overall `status` and `first_divergence.status` remain `unknown`. The incorrect dimension result survives serialization and affects reconstruction verification: importing the captures and CLI comparison as an Evidence bundle through MCP, then calling `verify_reconstruction` with `dimension: "terminal"`, returns claim and overall `pass`. Selecting `dimension: "overall"` instead returns `unknown`. ### Reproduction The following fixture-backed reproduction was executed against the prepared baseline checkout, using its locked dependencies and existing `dist/`; no build or installation was rerun. It invokes the actual terminal renderer and bounded normalizer to produce omitted and complete rendered observations, wraps them in validated synthetic capture Evidence, and compares files through the registered CLI. It does not launch a target or record a live PTY session. Run from the checkout root. The script writes only to a disposable directory, isolates the CLI's home/config/cache/state/temp directories, and generates `bundle.json` containing the two omitted-rendered captures and their actual traced CLI comparison. ```sh export REA_REPO="$PWD" export REPRO_DIR="$(mktemp -d)" node --input-type=module <<'JS' import assert from 'node:assert/strict'; import {mkdirSync, readFileSync, writeFileSync} from 'node:fs'; import {join, dirname, resolve} from 'node:path'; import {pathToFileURL} from 'node:url'; import {spawnSync} from 'node:child_process'; const repo = resolve(process.env.REA_REPO); const work = resolve(process.env.REPRO_DIR); const load = path => import(pathToFileURL(join(repo, 'dist', path))); const {EMPTY_PROCESS_CAPTURE_EXAMPLE: example} = await load('domain/process/processCaptureExample.js'); const {parseProcessCapture} = await load('domain/process/processCaptureParsing.js'); const {hasCaptureTruncation} = await load('domain/process/processCaptureCoverage.js'); const {processScenarioSchema} = await load('domain/process/processScenario.js'); const {createBoundedProcessTextNormalizer} = await load('process/capture/ProcessNormalization.js'); const {TerminalRenderer} = await load('process/capture/TerminalRenderer.js'); const {createEvidence} = await load('domain/evidence.js'); const {PROCESS_PROVIDER} = await load('domain/process/processEvidenceProvider.js'); const frame = {sequence: 0, at_ms: 10, data: 'Ready'}; const scenario = processScenarioSchema.parse({executable: '/fixture/program', working_directory: '/fixture', limits: {output_bytes: 5}, terminal: {columns: 5, rows: 1, scrollback: 0}}); const normalize = createBoundedProcessTextNormalizer(scenario, '/fixture/temporary', 100); async function observe(budget) { const renderer = new TerminalRenderer({columns: 5, rows: 1, scrollback: 0, maxBytes: budget, normalize}); renderer.write(frame.data, frame.at_ms); const frames = await renderer.frames(); const retention = renderer.retention(); await renderer.dispose(); return {frames, retention}; } const omitted = await observe(5); const complete = await observe(100); assert.equal(omitted.frames.length, 0); assert.equal(complete.frames.length, 1); console.log('producer', JSON.stringify(omitted)); const residuals = [ {scope: 'process', reason: 'Process trees are sampled and may omit short-lived descendants.'}, {scope: 'process', reason: 'The executable digest is a prelaunch file sample; operating-system image binding is not atomically observed.'}, {scope: 'environment', reason: 'Inherited host environment variables are not recorded.'}, {scope: 'network', reason: 'Network activity is not observed by this capture.'} ]; function capture(observation, rawOmitted = false) { const c = structuredClone(example); c.frames = rawOmitted ? [] : [frame]; c.rendered_frames = rawOmitted ? [] : observation.frames; c.truncation_details.raw_terminal = {budget_bytes: rawOmitted ? 1 : observation.retention.budget_bytes, observed_bytes: 5, retained_bytes: rawOmitted ? 0 : 5, observed_frames: 1, retained_frames: rawOmitted ? 0 : 1}; c.truncation_details.rendered_terminal = rawOmitted ? {budget_bytes: 1, observed_bytes: 0, retained_bytes: 0, observed_frames: 0, retained_frames: 0} : observation.retention; c.truncated = hasCaptureTruncation(c.truncation_details); c.residual_unknowns = residuals; c.limitations = c.truncated ? [rawOmitted ? 'Raw PTY chunks exceeded output_bytes and were omitted; rendered states also lack those inputs. See truncation_details.raw_terminal.' : 'Cumulative rendered state and visible-line bytes exceeded output_bytes; whole rendered frames were omitted. Raw PTY coverage is reported separately. See truncation_details.rendered_terminal.'] : []; const entries = [['filesystem_checkpoints', 0], ...c.frames.map((_, i) => ['frames', i]), ...c.rendered_frames.map((_, i) => ['rendered_frames', i]), ['lifecycle', 0], ['lifecycle', 1], ['filesystem_checkpoints', 1]]; c.event_journal = entries.map(([collection, index], capture_order) => ({collection, index, capture_order})); return parseProcessCapture(c); } function evidence(name, value) { return createEvidence(undefined, PROCESS_PROVIDER, {predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: {fixture: name}, result: value, confidence: 'observed', authority: 'controlled-replay', environment: {id: 'fixture', platform: 'fixture', architecture: 'fixture', isolation: 'process'}, limitations: value.limitations}); } const save = (name, value) => {writeFileSync(join(work, name), JSON.stringify(value, null, 2)); return name;}; for (const [name, observation, rawOmitted] of [['omitted', omitted, false], ['complete', complete, false], ['raw-omitted', omitted, true]]) { for (const side of ['left', 'right']) save(`${name}-${side}.json`, evidence(`${name}-${side}`, capture(observation, rawOmitted))); } save('trace.json', {events: [{id: 'ready', source: 'terminal_raw', exact: frame, cardinality: {kind: 'required'}}], language: {kind: 'finite_traces', variants: [{id: 'ready-only', trace: ['ready']}]}}); const env = {PATH: `${dirname(process.execPath)}:/usr/bin:/bin`, NO_COLOR: '1'}; for (const key of ['HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_STATE_HOME', 'TMPDIR']) {env[key] = join(work, key.toLowerCase()); mkdirSync(env[key], {recursive: true});} function compare(name, traced) { const run = spawnSync(process.execPath, [join(repo, 'scripts/rea.mjs'), 'compare-process-captures', `${name}-left.json`, `${name}-right.json`, ...(traced ? ['trace.json'] : []), '--format', 'json'], {cwd: work, env, encoding: 'utf8', timeout: 20000}); assert.equal(run.status, 0, run.stdout + run.stderr); const value = JSON.parse(run.stdout); save(`${name}-${traced ? 'traced' : 'exact'}-comparison.json`, value); const r = value.normalized_result; console.log(JSON.stringify({input: name, traced, terminal: r.terminal, trace: r.trace?.verdict ?? null, status: r.status, first_divergence: r.first_divergence.status})); return value; } assert.equal(compare('omitted', false).normalized_result.terminal, 'unknown'); assert.equal(compare('omitted', true).normalized_result.terminal, 'unchanged'); assert.equal(compare('complete', true).normalized_result.terminal, 'unchanged'); assert.equal(compare('raw-omitted', true).normalized_result.terminal, 'unknown'); const {createEvidenceBundle} = await load('domain/evidenceBundle.js'); const records = ['omitted-left.json', 'omitted-right.json', 'omitted-traced-comparison.json'].map(name => JSON.parse(readFileSync(join(work, name), 'utf8'))); save('bundle.json', createEvidenceBundle(records)); console.log('comparison_evidence_id', records[2].evidence_id); JS ``` For the downstream check, call MCP `import_evidence_bundle` with the absolute path to the generated `bundle.json`. Then call `verify_reconstruction` with the request below, substituting the `comparison_evidence_id` printed by the script. This route was independently executed against the production MCP server over linked in-memory SDK transports, using an empty analysis-provider registry. Import and both verification calls succeeded without opening a binary or launching an analysis provider. ```json { "specification": { "name": "Terminal coverage", "claims": [{ "kind": "behavioral", "claim_id": "terminal", "title": "Replay remains equivalent", "comparison_evidence_id": "<printed comparison_evidence_id>", "dimension": "terminal" }] } } ``` The terminal claim returns `pass`; changing the claim ID and dimension to `overall` returns `unknown`. The public fixture script was replayed successfully, and the imported-bundle MCP check was rerun using those newly generated records. ### Expected behavior A complete raw-terminal assertion may pass even when rendered snapshots are omitted. The terminal comparison dimension must nevertheless remain `unknown` for those omitted observations, and a terminal-scoped reconstruction claim must not pass on that dimension result. Complete rendered coverage must still permit terminal `unchanged`; actual raw truncation must keep the raw assertion `unknown`. ### Impact and scope P2: this is a bounded result-integrity defect for complete raw chunks with omitted rendered snapshots and an equivalent raw-only trace. The observed consequence includes a false terminal-scoped reconstruction pass. The aggregate comparison remains `unknown`, the input Evidence remains intact, and the comparison retains the omission limitation. Population frequency and live-capture/platform prevalence were not measured. No security boundary is involved. A verified workaround is to preserve a second comparison without the trace specification: the same omitted-rendered inputs then retain terminal `unknown`. Keep the original trace result separately to preserve its raw conformance information. Inspecting the retained truncation metadata also exposes the gap. The higher-budget renderer control retained its snapshot, but recapturing an arbitrary live workload with a larger budget was not tested. ### REA version 6.8.0; source revision `0d1a454e25db825567782e461da21530bad4b158`. The submission freshness comparison against upstream main `3233992733c33dabf2de09d22ee4e6d9409f797a` found no changes in the cited process comparison, renderer or reconstruction semantics, their contracts, documentation or dependencies. Other desktop, artifact, cleanup, browser and CI changes were inspected separately. The runtime reproduction was not rerun at that submission head. 6.8.0 was the latest published release checked. ### Runtime and platform Node.js v24.22.1; macOS 27.0.1 arm64. Existing compiled runtime and locked dependencies; no Hopper, Ghidra, IDA, debugger, or external analysis service used. ### Target and artifact context Synthetic process-capture Evidence based on the public empty-capture example, with one raw `Ready` frame and renderer-produced retention metadata. Two distinct capture Evidence records share an explicit fixture environment and controlled-replay authority. The rendered producer is real; the surrounding captures are fixtures, not observations of an executed program. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence All four CLI subprocesses exited 0. The renderer and CLI results were: ```text producer: frames=[]; retention={"budget_bytes":5,"observed_bytes":0,"retained_bytes":0,"observed_frames":1,"retained_frames":0,"unknown_size_frames":1} omitted, no trace: terminal=unknown; trace absent; status=unknown; first_divergence=unknown omitted, raw-only trace: terminal=unchanged; trace=equivalent; status=unknown; first_divergence=unknown complete, raw-only trace: terminal=unchanged; trace=equivalent; status=unknown; first_divergence=unknown raw-omitted, raw-only trace: terminal=unknown; trace=unknown; status=unknown; first_divergence=unknown MCP terminal claim: status=pass; claim=pass; observed_status=unchanged MCP overall claim: status=unknown; claim=unknown; observed_status=unknown ``` The omitted-rendered captures have `truncated: true`, complete raw retention, and no terminal-scope residual unknown, matching the producer's deliberate filtering of terminal coverage reasons. Their limitation says: `Cumulative rendered state and visible-line bytes exceeded output_bytes; whole rendered frames were omitted. Raw PTY coverage is reported separately. See truncation_details.rendered_terminal.` The comparison retains that limitation and the first-divergence reason `Incomplete observations prevent locating the first divergence; unaffected dimensions retain their comparisons.` - Expected source: `docs/process-capture.md:80-95,526-537` requires affected incomplete dimensions to remain unknown and allows trace assertions to use only their selected sources' coverage; `src/process/capture/TerminalRenderer.ts:32-37` explicitly retains both representations because neither subsumes the other. - Failure source: The independently executed registered CLI and imported-bundle MCP results above; `src/domain/process/processTraceDimensionProjection.ts:21-46` checks only retained arrays when deciding terminal coverage, and `src/domain/process/processComparison.ts:366-373` uses that answer to substitute the equivalent trace's dimension status. - Evidence provenance: observed; the source explanation and producer reachability below are separately source-derived. - Local verification: reproduced through serialized Evidence at the CLI and MCP boundaries; real target capture and stdio transport were not exercised. - Reproduction completeness: complete for the fixture-backed supported comparison/import/verification route. ### Typed outcome or recovery expected For the omitted-rendered case, retain `trace.verdict: "equivalent"`, terminal `unknown`, overall `unknown`, and unknown first divergence. A terminal reconstruction claim based on that result must remain `unknown`. Keep the complete-coverage control unchanged and raw-truncated trace control unknown. ### Analysis or suspected cause Source-derived: the exact comparison recognizes rendered truncation through `processSourceTruncated`, but trace dimension coverage tests retained `rendered_frames.length` rather than producer-reported omissions. An empty retained array therefore makes a raw-only specification appear to cover the terminal dimension. The passing trace's `unchanged` status then replaces the exact `unknown` result. `src/domain/reconstructionVerification.ts:282-287` consumes the selected dimension directly. Producer reachability is supported by the executed `TerminalRenderer` and bounded normalizer plus the production wiring: `ProcessHarness.ts:240-253` supplies `limits.output_bytes` as the rendered budget, and `ProcessCaptureLifecycle.ts:433-435` excludes terminal coverage reasons from `residual_unknowns` while retaining their accounting and limitations. Five retained raw bytes can exhaust the independent rendered budget because rendered retention counts visible lines plus serialized state. The reproduction uses the actual returned zero-byte/unknown-size omission accounting, rather than inventing an observed rendered byte count. ### Additional context Prepared with AI assistance and independently verified. Last known good version: NOT_ENOUGH_INFO; no regression boundary was established. This report concerns an existing terminal-dimension status, not a request for new timing explanations or separate comparison facets. ## References - Closed [#3931](https://github.com/morluto/rea/issues/3931) and its merged fix [#4178](https://github.com/morluto/rea/pull/4178) addressed a failing trace promoting unrelated unknown terminal coverage to changed. This current case instead uses an equivalent trace to promote affected unknown coverage to unchanged; the existing fix is present in the tested baseline. - Closed [#3180](https://github.com/morluto/rea/issues/3180) addressed complete filesystem traces retaining ignored timestamp differences. Here the missing rendered coverage must block terminal equivalence. - Open [#724](https://github.com/morluto/rea/issues/724) proposes additive content/outcome/timing explanations while preserving statuses. Its discussion does not report this passing raw-only trace erasing rendered-truncation uncertainty. ### Restoration check Replay the same omitted-rendered Evidence and raw-only specification: raw trace equivalence must survive, terminal must stay unknown, and terminal-scoped reconstruction verification must return unknown after serialized bundle import. The no-trace case must keep terminal unknown, the complete rendered control must keep terminal unchanged, and raw-omitted input must keep its trace and terminal unknown. Preserve overall status, first-divergence uncertainty, retained observations, omission metadata and Evidence provenance. ### Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: The executed CLI falsely reports terminal unchanged for omitted rendered observations, and the imported serialized comparison produces a terminal-scoped MCP reconstruction pass. The aggregate comparison and overall verification retain unknown, limiting the observed failure's scope. - Workaround status: available - Workaround basis: The executed no-trace comparison of the same Evidence keeps terminal unknown; retaining it alongside the original trace result preserves both coverage uncertainty and the requested raw conformance evidence.

bsakaguchi closed this issue

2026-10-31 00:48:58 +00:00

bsakaguchi referenced this issue

2026-10-31 00:51:42 +00:00

[GH-ISSUE #7774] [Design] Bound per-binding member-write records in JavaScript semantic analysis #4639

bsakaguchi referenced this issue

2026-10-31 01:12:28 +00:00

[PR #4331] [CLOSED] perf(javascript): select member-mutation origins after coverage #6178