morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4000] [Bug] Process comparison Evidence omits the selected freshness requirement #1590

New issue

Closed

opened 2026-10-16 21:07:32 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-16 21:07:32 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4000

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

compare_process_captures enforces an explicit max_capture_age_ms requirement but omits it from returned and retained comparison Evidence. Comparing the same captures with and without a successful freshness requirement returns the same Evidence ID. The retained record cannot identify which requirement the call checked.

Two focused MCP runs reproduced the omission. The second also checked a zero-age rejection control, which returned invalid_request. This report concerns the provenance of successful comparisons; the selected freshness limit still rejects stale input.

Reproduction

With the source baseline and locked dependencies installed, add the following temporary test as tests/boundary/mcp/processFreshnessPolicy.test.ts:

import { expect, it } from "vitest";
import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { PROCESS_PROVIDER } from "../../../src/domain/process/processEvidenceProvider.js";
import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "../../../src/domain/process/processCaptureExample.js";
import { jsonValueSchema } from "../../../src/domain/jsonValue.js";

it("retains a selected process comparison freshness requirement", async () => {
  const harness = await createApplicationMcpHarness({
    serverOptions: { environment: {} },
  });
  try {
    const capture = createEvidence(undefined, PROCESS_PROVIDER, {
      predicateType: "rea.process-capture",
      operation: "capture_process_scenario",
      parameters: {},
      result: jsonValueSchema.parse(EMPTY_PROCESS_CAPTURE_EXAMPLE),
      confidence: "observed",
      authority: "controlled-replay",
    });
    const call = (max?: number) => harness.client.callTool({
      name: "compare_process_captures",
      arguments: {
        left: capture,
        right: capture,
        ...(max === undefined ? {} : { max_capture_age_ms: max }),
      },
    });
    const rejected = await call(0);
    expect(rejected.isError).toBe(true);
    const unconstrained = await call();
    const selected = await call(Number.MAX_SAFE_INTEGER);
    expect(unconstrained.isError).not.toBe(true);
    expect(selected.isError).not.toBe(true);
    const before = parseEvidence(unconstrained.structuredContent);
    const after = parseEvidence(selected.structuredContent);
    console.log(JSON.stringify({
      selected_policy_retained: Object.hasOwn(after.parameters, "max_capture_age_ms"),
      same_evidence_id: before.evidence_id === after.evidence_id,
      retained_policy: harness.session.evidenceById(after.evidence_id)?.parameters.max_capture_age_ms ?? null,
    }));
    expect(after.parameters).toHaveProperty("max_capture_age_ms", Number.MAX_SAFE_INTEGER);
  } finally {
    await harness.close();
  }
});

Run npx vitest run tests/boundary/mcp/processFreshnessPolicy.test.ts. The final preservation assertion fails with exit status 1. Remove the temporary test afterward. This uses production MCP registration, comparison and session retention over the SDK in-memory transport with the canonical synthetic capture. It does not launch a target process.

Expected behavior

Successful comparison Evidence should retain the caller's explicit freshness requirement. Omitting the option must remain distinguishable from selecting it, including in derived Evidence identity.

AGENTS.md:13 requires preserving caller-selected inputs and metadata. src/contracts/sessionToolSchemas.ts:40 accepts the freshness input; src/contracts/promptContracts.ts:393 describes comparison under recorded normalization and freshness requirements.

Impact and scope

A later consumer of returned or retained comparison Evidence cannot determine which freshness requirement was enforced. Both successful calls share the same semantic Evidence identity because the selected requirement is absent. The reproduction establishes this for synthetic capture Evidence through MCP; real process capture and other adapters were not tested. No freshness-enforcement bypass was observed.

Saving the original MCP request separately may retain the requirement outside Evidence, but that workaround was not verified. No REA-native recovery of the omitted selection was verified.

REA version

rea 6.5.0 at source commit 4692bb3330

Runtime and platform

Node 24.18.0; macOS arm64

Target and artifact context

Canonical synthetic process capture Evidence from EMPTY_PROCESS_CAPTURE_EXAMPLE; production MCP registration and session ledger over SDK in-memory transports. No target process, native provider or external network was used in the reproduction.

Interface

MCP

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

The focused MCP test printed:

{"selected_policy_retained":false,"same_evidence_id":true,"retained_policy":null}
AssertionError: expected { ... } to have property "max_capture_age_ms" with value 9007199254740991
Expected: 9007199254740991
Received: undefined

The zero-age control returned isError: true, error.code: "invalid_request", and an issue at ["max_capture_age_ms"] with reason "out_of_range". This establishes that the request's freshness enforcement runs while successful comparison Evidence omits the selected policy.

Source-derived corroboration: src/server/registerProcessComparisonTool.ts:130 passes the freshness limit into comparison. The parameters constructed at line 150 omit it before recording and delivery. The cited implementation, contracts and reproduction dependencies are unchanged between the tested baseline and current upstream main 35e17a717229c1905828c0a0664a1825a985cf28. The baseline reproduction was not rerun on that revision.

Typed outcome or recovery expected

The zero-age control must continue rejecting the historical captures. A successful explicit freshness selection must appear with its exact value in returned and retained Evidence and distinguish its identity from an otherwise identical call without the option. Preserve existing comparison results and source Evidence links.

Analysis or suspected cause

Source-derived: the MCP tool projects trace_spec and capture normalization into Evidence parameters, but does not project max_capture_age_ms, even though it passes that input to the comparison operation. The matching Evidence IDs reflect identical retained semantic content, not a cryptographic hash collision.

Additional context

The existing tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts passed all 15 tests in the baseline verification. It checks stale-input rejection but does not assert preservation of a successful call's selected freshness requirement.

Related closed #61 introduced process comparison and freshness requirements. Open #3925 concerns repeating a comparison after an unknown-state update, and open #3980 concerns reconstruction verification of unresolved capture evidence. Those are different contracts and failure signals.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4000 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `compare_process_captures` enforces an explicit `max_capture_age_ms` requirement but omits it from returned and retained comparison Evidence. Comparing the same captures with and without a successful freshness requirement returns the same Evidence ID. The retained record cannot identify which requirement the call checked. Two focused MCP runs reproduced the omission. The second also checked a zero-age rejection control, which returned `invalid_request`. This report concerns the provenance of successful comparisons; the selected freshness limit still rejects stale input. ### Reproduction With the source baseline and locked dependencies installed, add the following temporary test as `tests/boundary/mcp/processFreshnessPolicy.test.ts`: ```ts import { expect, it } from "vitest"; import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; import { PROCESS_PROVIDER } from "../../../src/domain/process/processEvidenceProvider.js"; import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "../../../src/domain/process/processCaptureExample.js"; import { jsonValueSchema } from "../../../src/domain/jsonValue.js"; it("retains a selected process comparison freshness requirement", async () => { const harness = await createApplicationMcpHarness({ serverOptions: { environment: {} }, }); try { const capture = createEvidence(undefined, PROCESS_PROVIDER, { predicateType: "rea.process-capture", operation: "capture_process_scenario", parameters: {}, result: jsonValueSchema.parse(EMPTY_PROCESS_CAPTURE_EXAMPLE), confidence: "observed", authority: "controlled-replay", }); const call = (max?: number) => harness.client.callTool({ name: "compare_process_captures", arguments: { left: capture, right: capture, ...(max === undefined ? {} : { max_capture_age_ms: max }), }, }); const rejected = await call(0); expect(rejected.isError).toBe(true); const unconstrained = await call(); const selected = await call(Number.MAX_SAFE_INTEGER); expect(unconstrained.isError).not.toBe(true); expect(selected.isError).not.toBe(true); const before = parseEvidence(unconstrained.structuredContent); const after = parseEvidence(selected.structuredContent); console.log(JSON.stringify({ selected_policy_retained: Object.hasOwn(after.parameters, "max_capture_age_ms"), same_evidence_id: before.evidence_id === after.evidence_id, retained_policy: harness.session.evidenceById(after.evidence_id)?.parameters.max_capture_age_ms ?? null, })); expect(after.parameters).toHaveProperty("max_capture_age_ms", Number.MAX_SAFE_INTEGER); } finally { await harness.close(); } }); ``` Run `npx vitest run tests/boundary/mcp/processFreshnessPolicy.test.ts`. The final preservation assertion fails with exit status 1. Remove the temporary test afterward. This uses production MCP registration, comparison and session retention over the SDK in-memory transport with the canonical synthetic capture. It does not launch a target process. ### Expected behavior Successful comparison Evidence should retain the caller's explicit freshness requirement. Omitting the option must remain distinguishable from selecting it, including in derived Evidence identity. `AGENTS.md:13` requires preserving caller-selected inputs and metadata. `src/contracts/sessionToolSchemas.ts:40` accepts the freshness input; `src/contracts/promptContracts.ts:393` describes comparison under recorded normalization and freshness requirements. ### Impact and scope A later consumer of returned or retained comparison Evidence cannot determine which freshness requirement was enforced. Both successful calls share the same semantic Evidence identity because the selected requirement is absent. The reproduction establishes this for synthetic capture Evidence through MCP; real process capture and other adapters were not tested. No freshness-enforcement bypass was observed. Saving the original MCP request separately may retain the requirement outside Evidence, but that workaround was not verified. No REA-native recovery of the omitted selection was verified. ### REA version rea 6.5.0 at source commit 4692bb33300c7cb77b095b6cc0c33da861c1b860 ### Runtime and platform Node 24.18.0; macOS arm64 ### Target and artifact context Canonical synthetic process capture Evidence from `EMPTY_PROCESS_CAPTURE_EXAMPLE`; production MCP registration and session ledger over SDK in-memory transports. No target process, native provider or external network was used in the reproduction. ### Interface MCP ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence The focused MCP test printed: ```text {"selected_policy_retained":false,"same_evidence_id":true,"retained_policy":null} AssertionError: expected { ... } to have property "max_capture_age_ms" with value 9007199254740991 Expected: 9007199254740991 Received: undefined ``` The zero-age control returned `isError: true`, `error.code: "invalid_request"`, and an issue at `["max_capture_age_ms"]` with reason `"out_of_range"`. This establishes that the request's freshness enforcement runs while successful comparison Evidence omits the selected policy. Source-derived corroboration: `src/server/registerProcessComparisonTool.ts:130` passes the freshness limit into comparison. The parameters constructed at line 150 omit it before recording and delivery. The cited implementation, contracts and reproduction dependencies are unchanged between the tested baseline and current upstream main `35e17a717229c1905828c0a0664a1825a985cf28`. The baseline reproduction was not rerun on that revision. ### Typed outcome or recovery expected The zero-age control must continue rejecting the historical captures. A successful explicit freshness selection must appear with its exact value in returned and retained Evidence and distinguish its identity from an otherwise identical call without the option. Preserve existing comparison results and source Evidence links. ### Analysis or suspected cause Source-derived: the MCP tool projects `trace_spec` and capture normalization into Evidence parameters, but does not project `max_capture_age_ms`, even though it passes that input to the comparison operation. The matching Evidence IDs reflect identical retained semantic content, not a cryptographic hash collision. ### Additional context The existing `tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts` passed all 15 tests in the baseline verification. It checks stale-input rejection but does not assert preservation of a successful call's selected freshness requirement. Related closed #61 introduced process comparison and freshness requirements. Open #3925 concerns repeating a comparison after an unknown-state update, and open #3980 concerns reconstruction verification of unresolved capture evidence. Those are different contracts and failure signals.

bsakaguchi commented

2026-10-16 21:07:33 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 15, 2026):

Agreed: the enforcement works, but provenance is incomplete — two successful calls with different freshness requirements collapsing to the same Evidence ID makes the retained record unable to answer "which policy was checked". Persisting the selected max_capture_age_ms in the comparison Evidence (even when the check passed) closes that audit gap without weakening the stale-input rejection.

@XinwangTao commented on GitHub (Oct 15, 2026): Agreed: the enforcement works, but provenance is incomplete — two successful calls with different freshness requirements collapsing to the same Evidence ID makes the retained record unable to answer "which policy was checked". Persisting the selected max_capture_age_ms in the comparison Evidence (even when the check passed) closes that audit gap without weakening the stale-input rejection.

bsakaguchi closed this issue

2026-10-17 21:02:22 +00:00

bsakaguchi referenced this issue

2026-10-17 21:12:39 +00:00

[PR #1590] [MERGED] fix(browser): keep declared scenario secrets out of redacted evidence #2991

bsakaguchi referenced this issue

2026-10-17 21:12:47 +00:00

[PR #1611] [CLOSED] fix(browser): redact escaped scenario artifact secrets #3004

bsakaguchi referenced this issue

2026-10-31 00:44:13 +00:00

[GH-ISSUE #6289] [Bug] Distinct localStorage code units receive the same digest and scenarios compare unchanged #3803