morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3925] [Bug] Repeating a process comparison fails after updating its residual unknown #1532

New issue

Closed

opened 2026-10-16 21:06:46 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:06:46 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3925

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Observed in one stdio session: the initial comparison returned status: "changed"; the repeat before the update returned equal Evidence and left the bundle unchanged. update_unknown succeeded with revision: 2 and status: "investigating". Both subsequent identical comparison attempts returned isError: true and the following diagnostic:

{"code":"revision_conflict","category":"execution_failure","message":"Evidence state changed before the update completed. Refresh the current state and try again.","retryable":true,"remediation":{"action":"Evidence state changed before the update completed. Refresh the current state and try again."},"details":{"reason":"already-exists"}}

The final get_evidence_bundle result equaled the bundle immediately after the update and contained the original comparison Evidence unchanged. list_unknowns still returned the updated head. Retrying the comparison did not restore successful delivery. No data loss was observed. Other derived workflows were not executed.

Reproduction

From the prepared baseline checkout root, run the following shell block. It creates two authenticated capture records that differ only by one interaction event, imports their canonical bundle through MCP, and drives all comparison and update operations through registered tools. The isolated home directory belongs to the disposable reproduction directory. The script asserts the successful pre-update control, successful update, two failed post-update attempts and preservation of the retained state.

export REA_VERIFY_ROOT="$(mktemp -d)"
mkdir -p "$REA_VERIFY_ROOT/home"
node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import { writeFile } from 'node:fs/promises';
import { resolve } from 'node:path';
import { Client } from '@modelcontextprotocol/client';
import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
import { EMPTY_PROCESS_CAPTURE_EXAMPLE as example } from './dist/domain/process/processCaptureExample.js';
import { PROCESS_PROVIDER } from './dist/domain/process/processEvidenceProvider.js';
import { createEvidence } from './dist/domain/evidence.js';
import { createEvidenceBundle } from './dist/domain/evidenceBundle.js';
import { processCaptureSchema } from './dist/domain/process/processCapture.js';
const root = process.env.REA_VERIFY_ROOT;
const capture = result => createEvidence(undefined, PROCESS_PROVIDER, {
  predicateType: 'rea.process-capture', operation: 'capture_process_scenario',
  parameters: {}, result: processCaptureSchema.parse(result),
  confidence: 'observed', authority: 'controlled-replay'
});
const left = capture(example);
const right = capture({ ...example, interaction_events: [{
  sequence: 0, scheduled_at_ms: 0, dispatched_at_ms: 0,
  type: 'input', data: 'fixture', outcome: 'dispatched'
}] });
const bundlePath = resolve(root, 'input-bundle.json');
await writeFile(bundlePath, JSON.stringify(createEvidenceBundle([left, right])));
const client = new Client({ name: 'comparison-repeat-repro', version: '1' });
const transport = new StdioClientTransport({
  command: process.execPath, args: ['scripts/rea.mjs', '--mcp'], cwd: process.cwd(),
  env: { PATH: process.env.PATH, HOME: resolve(root, 'home'), REA_LOG_LEVEL: 'silent' },
  stderr: 'pipe'
});
async function call(name, args = {}) {
  const response = await client.callTool({ name, arguments: args });
  return response;
}
function data(response) {
  assert.notEqual(response.isError, true, JSON.stringify(response));
  return response.structuredContent;
}
try {
  await client.connect(transport);
  data(await call('import_evidence_bundle', { path: bundlePath }));
  const first = data(await call('compare_process_captures', { left, right }));
  console.log(JSON.stringify({ stage: 'first', status: first.normalized_result.status }));
  const beforeRepeat = data(await call('get_evidence_bundle')).result;
  const before = data(await call('compare_process_captures', { left, right }));
  assert.deepEqual(before, first);
  assert.deepEqual(data(await call('get_evidence_bundle')).result, beforeRepeat);
  console.log(JSON.stringify({ stage: 'repeat-before-update', sameEvidence: true, bundleUnchanged: true }));
  const unknown = data(await call('list_unknowns', { domain: 'process-comparison' })).result.items[0];
  const update = {
    unknown_id: unknown.unknown_id, expected_revision: unknown.revision,
    status: 'investigating', severity: unknown.severity,
    supporting_evidence_ids: unknown.supporting_evidence_ids,
    contradicting_evidence_ids: unknown.contradicting_evidence_ids,
    required_authority: unknown.required_authority,
    required_confidence: unknown.required_confidence,
    required_environment: unknown.required_environment,
    recommended_probes: unknown.recommended_probes,
    relationships: unknown.relationships, resolution: null
  };
  const updated = data(await call('update_unknown', update)).result;
  assert.equal(updated.revision, 2);
  assert.equal(updated.status, 'investigating');
  console.log(JSON.stringify({ stage: 'update', revision: updated.revision, status: updated.status }));
  const afterUpdate = data(await call('get_evidence_bundle')).result;
  for (let attempt = 1; attempt <= 2; attempt++) {
    const repeated = await call('compare_process_captures', { left, right });
    assert.equal(repeated.isError, true);
    const error = JSON.parse(repeated.content.find(item => item.type === 'text').text).error;
    assert.equal(error.details.reason, 'already-exists');
    console.log(JSON.stringify({ stage: 'repeat-after-update', attempt, isError: repeated.isError, error }));
  }
  const retained = data(await call('get_evidence_bundle')).result;
  assert.deepEqual(retained, afterUpdate);
  assert.deepEqual(retained.records.find(record => record.evidence_id === first.evidence_id), first);
  assert.deepEqual(data(await call('list_unknowns', { domain: 'process-comparison' })).result.items[0], updated);
  console.log(JSON.stringify({ stage: 'retention', bundleUnchanged: true, comparisonStillRetained: true, updatedHeadPreserved: true }));
} finally {
  await client.close();
  await transport.close();
}
JS
rm -rf "$REA_VERIFY_ROOT"
unset REA_VERIFY_ROOT

Expected behavior

An identical deterministic comparison should continue returning the same derived Evidence after the analyst appends an unknown revision. The comparison's canonical tool effects advertise idempotence, and the existing MCP boundary test requires equal Evidence and no bundle change for an identical repeat. The full-state unknown update changes the residual unknown's status, not either capture or the requested comparison. The repeat should preserve that edit and the revision history.

Impact and scope

A valid repeated comparison fails after a successful residual-unknown status edit, although the immutable inputs are unchanged. Two post-update attempts failed in each of two independent stdio sessions. No data loss was observed: get_evidence_bundle retrieved the original comparison and list_unknowns retained the updated head. Retrieving the retained result is a demonstrated workaround for these immutable inputs. Other derived workflows were not executed.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Synthetic process capture Evidence from the public repository example, imported as a canonical bundle. No target process or analysis provider runs.

Interface

MCP

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

Observed sequence: first comparison status: changed; identical pre-update repeat returned equal Evidence and an unchanged bundle; update_unknown succeeded with revision 2 and status investigating; both post-update repeats returned revision_conflict, category execution_failure, reason already-exists, and retryable: true. The final bundle, retained comparison and updated head were unchanged.

Expected contract: src/contracts/toolEffects.ts:32-43,52,698 declares the comparison idempotent; tests/boundary/mcp/serverUnknownsMcp.test.ts:238-251 requires identical repeats to return equal Evidence without a bundle change; src/contracts/sessionToolContracts.ts:151-154 defines unknown updates as appended full-state immutable revisions.

The cited failing files are unchanged at current upstream main 22ec8bdb2f7d4626587f23b1eed4a207df600386. The intervening launcher change only normalizes global output flags.

Typed outcome or recovery expected

With the same imported capture pair, require the initial comparison and its pre-update repeat to succeed with equal Evidence and no additional retained records. Append the valid unknown status update, then require both post-update identical comparisons to return that same Evidence successfully. The updated unknown head, its revision history and the complete retained bundle must remain unchanged by those repeats. Preserve rejection of stale explicit unknown updates and invalid or conflicting Evidence; successful comparison replay must not bypass those distinct integrity checks.

Analysis or suspected cause

Source-derived: src/server/registerProcessComparisonTool.ts:174-190 calls src/server/recordDerivedEvidence.ts:14-16, then src/application/binary/BinarySessionRecords.ts:329-335 and src/application/investigation/InvestigationRecords.ts:92-105. src/application/investigation/EvidenceLedger.ts:279-285 rejects an existing unknown head whose revision is not 1, even for this identical deterministic comparison.

Additional context

Related closed #719 addressed distinct process comparisons colliding. Open #3333 and PR #3503 address distinct unresolved workflow queries sharing one question. PR #3503 explicitly leaves ledger unknown keying unchanged and does not modify process comparison. This report instead starts with an idempotent identical comparison and fails only after a successful unknown edit. #3005 concerns update_unknown itself failing; that update succeeds here.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3925 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Observed in one stdio session: the initial comparison returned `status: "changed"`; the repeat before the update returned equal Evidence and left the bundle unchanged. `update_unknown` succeeded with `revision: 2` and `status: "investigating"`. Both subsequent identical comparison attempts returned `isError: true` and the following diagnostic: ```json {"code":"revision_conflict","category":"execution_failure","message":"Evidence state changed before the update completed. Refresh the current state and try again.","retryable":true,"remediation":{"action":"Evidence state changed before the update completed. Refresh the current state and try again."},"details":{"reason":"already-exists"}} ``` The final `get_evidence_bundle` result equaled the bundle immediately after the update and contained the original comparison Evidence unchanged. `list_unknowns` still returned the updated head. Retrying the comparison did not restore successful delivery. No data loss was observed. Other derived workflows were not executed. ### Reproduction From the prepared baseline checkout root, run the following shell block. It creates two authenticated capture records that differ only by one interaction event, imports their canonical bundle through MCP, and drives all comparison and update operations through registered tools. The isolated home directory belongs to the disposable reproduction directory. The script asserts the successful pre-update control, successful update, two failed post-update attempts and preservation of the retained state. ```sh export REA_VERIFY_ROOT="$(mktemp -d)" mkdir -p "$REA_VERIFY_ROOT/home" node --input-type=module <<'JS' import assert from 'node:assert/strict'; import { writeFile } from 'node:fs/promises'; import { resolve } from 'node:path'; import { Client } from '@modelcontextprotocol/client'; import { StdioClientTransport } from '@modelcontextprotocol/client/stdio'; import { EMPTY_PROCESS_CAPTURE_EXAMPLE as example } from './dist/domain/process/processCaptureExample.js'; import { PROCESS_PROVIDER } from './dist/domain/process/processEvidenceProvider.js'; import { createEvidence } from './dist/domain/evidence.js'; import { createEvidenceBundle } from './dist/domain/evidenceBundle.js'; import { processCaptureSchema } from './dist/domain/process/processCapture.js'; const root = process.env.REA_VERIFY_ROOT; const capture = result => createEvidence(undefined, PROCESS_PROVIDER, { predicateType: 'rea.process-capture', operation: 'capture_process_scenario', parameters: {}, result: processCaptureSchema.parse(result), confidence: 'observed', authority: 'controlled-replay' }); const left = capture(example); const right = capture({ ...example, interaction_events: [{ sequence: 0, scheduled_at_ms: 0, dispatched_at_ms: 0, type: 'input', data: 'fixture', outcome: 'dispatched' }] }); const bundlePath = resolve(root, 'input-bundle.json'); await writeFile(bundlePath, JSON.stringify(createEvidenceBundle([left, right]))); const client = new Client({ name: 'comparison-repeat-repro', version: '1' }); const transport = new StdioClientTransport({ command: process.execPath, args: ['scripts/rea.mjs', '--mcp'], cwd: process.cwd(), env: { PATH: process.env.PATH, HOME: resolve(root, 'home'), REA_LOG_LEVEL: 'silent' }, stderr: 'pipe' }); async function call(name, args = {}) { const response = await client.callTool({ name, arguments: args }); return response; } function data(response) { assert.notEqual(response.isError, true, JSON.stringify(response)); return response.structuredContent; } try { await client.connect(transport); data(await call('import_evidence_bundle', { path: bundlePath })); const first = data(await call('compare_process_captures', { left, right })); console.log(JSON.stringify({ stage: 'first', status: first.normalized_result.status })); const beforeRepeat = data(await call('get_evidence_bundle')).result; const before = data(await call('compare_process_captures', { left, right })); assert.deepEqual(before, first); assert.deepEqual(data(await call('get_evidence_bundle')).result, beforeRepeat); console.log(JSON.stringify({ stage: 'repeat-before-update', sameEvidence: true, bundleUnchanged: true })); const unknown = data(await call('list_unknowns', { domain: 'process-comparison' })).result.items[0]; const update = { unknown_id: unknown.unknown_id, expected_revision: unknown.revision, status: 'investigating', severity: unknown.severity, supporting_evidence_ids: unknown.supporting_evidence_ids, contradicting_evidence_ids: unknown.contradicting_evidence_ids, required_authority: unknown.required_authority, required_confidence: unknown.required_confidence, required_environment: unknown.required_environment, recommended_probes: unknown.recommended_probes, relationships: unknown.relationships, resolution: null }; const updated = data(await call('update_unknown', update)).result; assert.equal(updated.revision, 2); assert.equal(updated.status, 'investigating'); console.log(JSON.stringify({ stage: 'update', revision: updated.revision, status: updated.status })); const afterUpdate = data(await call('get_evidence_bundle')).result; for (let attempt = 1; attempt <= 2; attempt++) { const repeated = await call('compare_process_captures', { left, right }); assert.equal(repeated.isError, true); const error = JSON.parse(repeated.content.find(item => item.type === 'text').text).error; assert.equal(error.details.reason, 'already-exists'); console.log(JSON.stringify({ stage: 'repeat-after-update', attempt, isError: repeated.isError, error })); } const retained = data(await call('get_evidence_bundle')).result; assert.deepEqual(retained, afterUpdate); assert.deepEqual(retained.records.find(record => record.evidence_id === first.evidence_id), first); assert.deepEqual(data(await call('list_unknowns', { domain: 'process-comparison' })).result.items[0], updated); console.log(JSON.stringify({ stage: 'retention', bundleUnchanged: true, comparisonStillRetained: true, updatedHeadPreserved: true })); } finally { await client.close(); await transport.close(); } JS rm -rf "$REA_VERIFY_ROOT" unset REA_VERIFY_ROOT ``` ### Expected behavior An identical deterministic comparison should continue returning the same derived Evidence after the analyst appends an unknown revision. The comparison's canonical tool effects advertise idempotence, and the existing MCP boundary test requires equal Evidence and no bundle change for an identical repeat. The full-state unknown update changes the residual unknown's status, not either capture or the requested comparison. The repeat should preserve that edit and the revision history. ### Impact and scope A valid repeated comparison fails after a successful residual-unknown status edit, although the immutable inputs are unchanged. Two post-update attempts failed in each of two independent stdio sessions. No data loss was observed: `get_evidence_bundle` retrieved the original comparison and `list_unknowns` retained the updated head. Retrieving the retained result is a demonstrated workaround for these immutable inputs. Other derived workflows were not executed. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Synthetic process capture Evidence from the public repository example, imported as a canonical bundle. No target process or analysis provider runs. ### Interface MCP ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence Observed sequence: first comparison `status: changed`; identical pre-update repeat returned equal Evidence and an unchanged bundle; `update_unknown` succeeded with revision 2 and status `investigating`; both post-update repeats returned `revision_conflict`, category `execution_failure`, reason `already-exists`, and `retryable: true`. The final bundle, retained comparison and updated head were unchanged. Expected contract: `src/contracts/toolEffects.ts:32-43,52,698` declares the comparison idempotent; `tests/boundary/mcp/serverUnknownsMcp.test.ts:238-251` requires identical repeats to return equal Evidence without a bundle change; `src/contracts/sessionToolContracts.ts:151-154` defines unknown updates as appended full-state immutable revisions. The cited failing files are unchanged at current upstream main `22ec8bdb2f7d4626587f23b1eed4a207df600386`. The intervening launcher change only normalizes global output flags. ### Typed outcome or recovery expected With the same imported capture pair, require the initial comparison and its pre-update repeat to succeed with equal Evidence and no additional retained records. Append the valid unknown status update, then require both post-update identical comparisons to return that same Evidence successfully. The updated unknown head, its revision history and the complete retained bundle must remain unchanged by those repeats. Preserve rejection of stale explicit unknown updates and invalid or conflicting Evidence; successful comparison replay must not bypass those distinct integrity checks. ### Analysis or suspected cause Source-derived: `src/server/registerProcessComparisonTool.ts:174-190` calls `src/server/recordDerivedEvidence.ts:14-16`, then `src/application/binary/BinarySessionRecords.ts:329-335` and `src/application/investigation/InvestigationRecords.ts:92-105`. `src/application/investigation/EvidenceLedger.ts:279-285` rejects an existing unknown head whose revision is not 1, even for this identical deterministic comparison. ### Additional context Related closed #719 addressed distinct process comparisons colliding. Open #3333 and PR #3503 address distinct unresolved workflow queries sharing one question. PR #3503 explicitly leaves ledger unknown keying unchanged and does not modify process comparison. This report instead starts with an idempotent identical comparison and fails only after a successful unknown edit. #3005 concerns `update_unknown` itself failing; that update succeeds here.

bsakaguchi commented

2026-10-16 21:06:46 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Confirmed: after update_unknown commits revision 2, subsequent comparisons in the same session should not keep failing with revision_conflict already-exists — the update is done, so the next comparison must either read the new head or refresh cleanly. A retryable conflict that never resolves on retry strands callers; the conflict check should be tied to the actual in-flight update, not to the committed revision.

@XinwangTao commented on GitHub (Oct 15, 2026): Confirmed: after update_unknown commits revision 2, subsequent comparisons in the same session should not keep failing with revision_conflict already-exists — the update is done, so the next comparison must either read the new head or refresh cleanly. A retryable conflict that never resolves on retry strands callers; the conflict check should be tied to the actual in-flight update, not to the committed revision.

bsakaguchi referenced this issue

2026-10-17 21:12:09 +00:00

[PR #1532] [MERGED] docs(website): expand the Touhou essay with oracle lessons #2949

bsakaguchi closed this issue

2026-10-31 00:31:18 +00:00

bsakaguchi commented

2026-10-31 00:31:19 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 18, 2026):

Addressed on main in github.com/morluto/rea@10e9448d89. Deterministic comparison replay now checks the residual unknown against its immutable creation revision, so later analyst status updates no longer cause an already-exists conflict or get overwritten. The committed regression covers repeated process comparisons after unknown updates, including after handoff. Closing as fixed based on source and regression-test review; I have not rerun the original runtime reproduction.

@coygeek commented on GitHub (Oct 18, 2026): Addressed on main in https://github.com/morluto/rea/commit/10e9448d89899d740d96b4f69915149e77d73965. Deterministic comparison replay now checks the residual unknown against its immutable creation revision, so later analyst status updates no longer cause an `already-exists` conflict or get overwritten. The committed regression covers repeated process comparisons after unknown updates, including after handoff. Closing as fixed based on source and regression-test review; I have not rerun the original runtime reproduction.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1532

No description provided.

Rows
Columns