morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3216] [Bug] Bundle comparison reports changed without an inline delta for display-only subject differences #1153

New issue

Closed

opened 2026-10-15 18:31:34 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:34 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3216

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Comparing two valid authenticated Evidence bundles that differ only in subject.local_path returns status: "changed", changes: [], limitations: [], records_unchanged: 1, and zero delta and unresolved counters. Both records retain the same authenticated Evidence ID. The production CLI exits 0 with no stderr. The production stdio MCP compare_bundles tool returns the same comparison in its Evidence normalized_result, without isError.

Separate CLI checks reproduced the same result for a name-only difference and for combined path and name differences. Identical-input CLI and MCP controls returned unchanged, equal digests and an empty change list.

Reproduction

From a clean checkout at 58300885207133422af235e0f2b10658cf958b03, install the lockfile dependencies and build the runtime using the repository's pinned development toolchain. These are setup prerequisites; verification used already-installed dependencies and a prepared build.
```
npm ci
npm run build:cached
```
From the checkout root, create the complete synthetic inputs inline. No artifact file or provider engine is needed.

node --input-type=module <<'JS'
import { mkdirSync, writeFileSync } from "node:fs";
const directory = "repro-bundle-comparison";
mkdirSync(directory, { recursive: true });
const left = {
  "artifacts": [
    {
      "architecture": "x86_64",
      "digest": {
        "sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
      },
      "format": "elf"
    }
  ],
  "captures": [],
  "environments": [],
  "providers": [
    {
      "id": "fixture",
      "name": "Fixture",
      "version": "1"
    }
  ],
  "records": [
    {
      "analysis_profile": null,
      "authority": "shipped-artifact",
      "confidence": "observed",
      "environment": null,
      "evidence_id": "ev_5c4779adb465316ebd3669723d7cf661e1166a6c11ddc2ab71645d2ab3116fe7",
      "evidence_links": [],
      "limitations": [],
      "locations": [],
      "normalized_result": {
        "ok": true
      },
      "operation": "observe",
      "parameters": {},
      "predicate_type": "rea.analysis",
      "provider": {
        "id": "fixture",
        "name": "Fixture",
        "version": "1"
      },
      "raw_result": null,
      "relationship_coverage": null,
      "subject": {
        "architecture": "x86_64",
        "digest": {
          "sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
        },
        "format": "elf",
        "local_path": "left/artifact.bin",
        "name": "artifact.bin"
      }
    }
  ],
  "scenarios": [],
  "unknowns": []
};
const right = structuredClone(left);
right.records[0].subject.local_path = "right/artifact.bin";
writeFileSync(`${directory}/left.json`, JSON.stringify(left));
writeFileSync(`${directory}/right.json`, JSON.stringify(right));
JS

Compare the bundles and observe the unexplained changed result.

node scripts/rea.mjs compare repro-bundle-comparison/left.json repro-bundle-comparison/right.json --json

Run the identical-input control. It returns unchanged.

node scripts/rea.mjs compare repro-bundle-comparison/left.json repro-bundle-comparison/left.json --json

The verifier also started the production stdio server with node scripts/rea.mjs mcp and called compare_bundles with these two bundle paths as left_bundle_path and right_bundle_path, and record_pairs: []. No target was opened.

Expected behavior

src/contracts/sessionToolContracts.ts:116 documents that compare_bundles compares exact record membership, explicit one-to-one observation pairs and complete residual-unknown revision histories, with output that is "digest-anchored and returns every change inline." The production MCP catalog advertised the same contract. src/domain/evidence.ts:247 excludes subject display path and name from authenticated Evidence identity; src/domain/evidence.ts:293 supports rebinding display metadata while retaining that identity.

For these inputs, comparison should either return unchanged for a difference outside the compared semantic content, or return changed with an inline delta explaining the display-metadata difference. A changed result with no inline explanation violates the complete-inline-change contract. Preserve valid canonical bundle digests for provenance; Evidence identity exclusion does not require the two bundle digests to match.

Impact and scope

The supported impact is P2. Status-based callers and automation receive a change signal with no explanation for valid inputs. The failure is bounded to the tested bundle differences; no data loss or general service outage was observed. No security boundary was crossed.

For the tested case, callers can inspect changes, unresolved and the detailed counters, which retain the correct record-membership result, and read the input display metadata if its difference matters. Retain the digests for provenance. An unresolved comparison must not be treated as equivalent.

REA version

6.4.0 (main at 5830088520)

Runtime and platform

Node 26.10.0; macOS arm64. No deep-analysis provider engine was involved.

Target and artifact context

Two valid synthetic one-record Evidence bundles with the same authenticated Evidence ID, artifact manifest and provider, and no residual unknowns. The synthetic subject declares ELF/x86_64 with a SHA-256 digest of 64 lowercase a characters. The inputs differ only in subject display path; no actual ELF file is analyzed. These observations do not establish that independently analyzing two real artifact copies always produces otherwise identical records.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The path-only CLI comparison exited 0 with no stderr and returned:

{"status":"changed","left_bundle_sha256":"3fbc51db499e1e862be9f21ee146b3775ae5bd681b839938388aa977abd41ec2","right_bundle_sha256":"763f1cbb4c41ed3659b634268d48a870427d3600f175ae59ac0a346c15dbc237","summary":{"records_unchanged":1,"records_added":0,"records_removed":0,"records_changed":0,"unknowns_unchanged":0,"unknowns_added":0,"unknowns_removed":0,"unknowns_advanced":0,"unknowns_diverged":0,"unresolved":0},"changes":[],"limitations":[]}

The production MCP comparison returned the same value in normalized_result, without isError.

Typed outcome or recovery expected

Return a consistent semantic unchanged result or an explained changed result for path-only, name-only and combined display differences, while preserving digest provenance. Keep identical-bundle controls unchanged.

Preserve the verified semantic-result controls. Changing the authenticated normalized_result from { "ok": true } to { "ok": false } without an explicit observation pair returned unknown with two unresolved membership entries through CLI and MCP. With an explicit one-to-one MCP pair, it returned changed, records_changed: 1 and one observed_change entry. The display-metadata repair must preserve both distinctions.

Analysis or suspected cause

Source-derived at baseline 58300885207133422af235e0f2b10658cf958b03: src/domain/bundleComparison.ts:104 selects changed when canonical bundle digests differ and there are no unresolved changes, even when changes is empty. Record membership uses Evidence IDs at src/domain/bundleComparison.ts:131. Bundle digests hash full canonical serialization at src/domain/bundleComparison.ts:299, and src/domain/evidenceBundle.ts:560 includes display metadata in the serialized record.

Source-derived adapter paths: src/cliEvidenceCommands.ts:50 and src/application/EvidenceBundleCommands.ts:70 route the CLI through bundle validation to comparison. src/server/registerBundleComparisonTool.ts:53 calls the same comparison for MCP.

Additional context

Related reports #724 and #2616 concern explanations and attribution within process-capture comparisons; this report concerns authenticated Evidence bundle comparisons that return a top-level change with no inline delta for display-only subject metadata.

Related: #3215 reports a separate failure in the same area; the two have independent fixes.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3216 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Comparing two valid authenticated Evidence bundles that differ only in `subject.local_path` returns `status: "changed"`, `changes: []`, `limitations: []`, `records_unchanged: 1`, and zero delta and unresolved counters. Both records retain the same authenticated Evidence ID. The production CLI exits 0 with no stderr. The production stdio MCP `compare_bundles` tool returns the same comparison in its Evidence `normalized_result`, without `isError`. Separate CLI checks reproduced the same result for a name-only difference and for combined path and name differences. Identical-input CLI and MCP controls returned `unchanged`, equal digests and an empty change list. ### Reproduction 1. From a clean checkout at `58300885207133422af235e0f2b10658cf958b03`, install the lockfile dependencies and build the runtime using the repository's pinned development toolchain. These are setup prerequisites; verification used already-installed dependencies and a prepared build. ```sh npm ci npm run build:cached ``` 2. From the checkout root, create the complete synthetic inputs inline. No artifact file or provider engine is needed. ```sh node --input-type=module <<'JS' import { mkdirSync, writeFileSync } from "node:fs"; const directory = "repro-bundle-comparison"; mkdirSync(directory, { recursive: true }); const left = { "artifacts": [ { "architecture": "x86_64", "digest": { "sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" }, "format": "elf" } ], "captures": [], "environments": [], "providers": [ { "id": "fixture", "name": "Fixture", "version": "1" } ], "records": [ { "analysis_profile": null, "authority": "shipped-artifact", "confidence": "observed", "environment": null, "evidence_id": "ev_5c4779adb465316ebd3669723d7cf661e1166a6c11ddc2ab71645d2ab3116fe7", "evidence_links": [], "limitations": [], "locations": [], "normalized_result": { "ok": true }, "operation": "observe", "parameters": {}, "predicate_type": "rea.analysis", "provider": { "id": "fixture", "name": "Fixture", "version": "1" }, "raw_result": null, "relationship_coverage": null, "subject": { "architecture": "x86_64", "digest": { "sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" }, "format": "elf", "local_path": "left/artifact.bin", "name": "artifact.bin" } } ], "scenarios": [], "unknowns": [] }; const right = structuredClone(left); right.records[0].subject.local_path = "right/artifact.bin"; writeFileSync(`${directory}/left.json`, JSON.stringify(left)); writeFileSync(`${directory}/right.json`, JSON.stringify(right)); JS ``` 3. Compare the bundles and observe the unexplained `changed` result. ```sh node scripts/rea.mjs compare repro-bundle-comparison/left.json repro-bundle-comparison/right.json --json ``` 4. Run the identical-input control. It returns `unchanged`. ```sh node scripts/rea.mjs compare repro-bundle-comparison/left.json repro-bundle-comparison/left.json --json ``` The verifier also started the production stdio server with `node scripts/rea.mjs mcp` and called `compare_bundles` with these two bundle paths as `left_bundle_path` and `right_bundle_path`, and `record_pairs: []`. No target was opened. ### Expected behavior `src/contracts/sessionToolContracts.ts:116` documents that `compare_bundles` compares exact record membership, explicit one-to-one observation pairs and complete residual-unknown revision histories, with output that is "digest-anchored and returns every change inline." The production MCP catalog advertised the same contract. `src/domain/evidence.ts:247` excludes subject display path and name from authenticated Evidence identity; `src/domain/evidence.ts:293` supports rebinding display metadata while retaining that identity. For these inputs, comparison should either return `unchanged` for a difference outside the compared semantic content, or return `changed` with an inline delta explaining the display-metadata difference. A `changed` result with no inline explanation violates the complete-inline-change contract. Preserve valid canonical bundle digests for provenance; Evidence identity exclusion does not require the two bundle digests to match. ### Impact and scope The supported impact is P2. Status-based callers and automation receive a change signal with no explanation for valid inputs. The failure is bounded to the tested bundle differences; no data loss or general service outage was observed. No security boundary was crossed. For the tested case, callers can inspect `changes`, `unresolved` and the detailed counters, which retain the correct record-membership result, and read the input display metadata if its difference matters. Retain the digests for provenance. An unresolved comparison must not be treated as equivalent. ### REA version 6.4.0 (main at 58300885207133422af235e0f2b10658cf958b03) ### Runtime and platform Node 26.10.0; macOS arm64. No deep-analysis provider engine was involved. ### Target and artifact context Two valid synthetic one-record Evidence bundles with the same authenticated Evidence ID, artifact manifest and provider, and no residual unknowns. The synthetic subject declares ELF/x86_64 with a SHA-256 digest of 64 lowercase `a` characters. The inputs differ only in subject display path; no actual ELF file is analyzed. These observations do not establish that independently analyzing two real artifact copies always produces otherwise identical records. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The path-only CLI comparison exited 0 with no stderr and returned: ```text {"status":"changed","left_bundle_sha256":"3fbc51db499e1e862be9f21ee146b3775ae5bd681b839938388aa977abd41ec2","right_bundle_sha256":"763f1cbb4c41ed3659b634268d48a870427d3600f175ae59ac0a346c15dbc237","summary":{"records_unchanged":1,"records_added":0,"records_removed":0,"records_changed":0,"unknowns_unchanged":0,"unknowns_added":0,"unknowns_removed":0,"unknowns_advanced":0,"unknowns_diverged":0,"unresolved":0},"changes":[],"limitations":[]} ``` The production MCP comparison returned the same value in `normalized_result`, without `isError`. ### Typed outcome or recovery expected Return a consistent semantic `unchanged` result or an explained `changed` result for path-only, name-only and combined display differences, while preserving digest provenance. Keep identical-bundle controls `unchanged`. Preserve the verified semantic-result controls. Changing the authenticated `normalized_result` from `{ "ok": true }` to `{ "ok": false }` without an explicit observation pair returned `unknown` with two unresolved membership entries through CLI and MCP. With an explicit one-to-one MCP pair, it returned `changed`, `records_changed: 1` and one `observed_change` entry. The display-metadata repair must preserve both distinctions. ### Analysis or suspected cause Source-derived at baseline `58300885207133422af235e0f2b10658cf958b03`: `src/domain/bundleComparison.ts:104` selects `changed` when canonical bundle digests differ and there are no unresolved changes, even when `changes` is empty. Record membership uses Evidence IDs at `src/domain/bundleComparison.ts:131`. Bundle digests hash full canonical serialization at `src/domain/bundleComparison.ts:299`, and `src/domain/evidenceBundle.ts:560` includes display metadata in the serialized record. Source-derived adapter paths: `src/cliEvidenceCommands.ts:50` and `src/application/EvidenceBundleCommands.ts:70` route the CLI through bundle validation to comparison. `src/server/registerBundleComparisonTool.ts:53` calls the same comparison for MCP. ### Additional context Related reports #724 and #2616 concern explanations and attribution within process-capture comparisons; this report concerns authenticated Evidence bundle comparisons that return a top-level change with no inline delta for display-only subject metadata. Related: #3215 reports a separate failure in the same area; the two have independent fixes.

bsakaguchi

2026-10-15 18:31:34 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:35 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Resolved on main by 030c47732.

Comparison status is derived from supported deltas and unresolved changes; display-only digest differences no longer produce changed with an empty inline delta.

Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: src/domain/bundleComparison.test.ts, tests/boundary/cli/bundleComparisonCli.test.ts and tests/boundary/mcp/bundleComparisonMcp.test.ts.

@morluto commented on GitHub (Oct 14, 2026): Resolved on main by 030c47732. Comparison status is derived from supported deltas and unresolved changes; display-only digest differences no longer produce changed with an empty inline delta. Rechecked on main b8510ee00 in the current focused run: 20 test files passed, 225 tests passed and one platform-dependent test skipped. Relevant regression coverage: src/domain/bundleComparison.test.ts, tests/boundary/cli/bundleComparisonCli.test.ts and tests/boundary/mcp/bundleComparisonMcp.test.ts.

bsakaguchi referenced this issue

2026-10-16 21:02:34 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:09:04 +00:00

[PR #1153] [CLOSED] fix(javascript): omit self-referential bundler edges with disclosure #2687