morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4436] [Bug] Unmodified 6.5.0 Evidence bundles are rejected on main as an integrity mismatch: records with `relationship_coverage: null` changed identity #1761

New issue

Closed

opened 2026-10-16 21:09:57 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-16 21:09:57 +00:00

Owner

Originally created by @Rinkia on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4436

Before submitting

I searched the open and closed issues for a related report (evidence_integrity_mismatch, "semantic identifier does not match", historical evidence import, relationship_coverage identity). Related but different: #2050 (a single record passed as a bundle) and #4398 (auditing bundles before sharing). The change is announced as a breaking change in the open release PR #3476, but its wording covers a smaller set of records than the ones that fail; see below.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

A bundle exported by released rea-agents@6.5.0, then imported unchanged with main @ d72d856f5:

{"code":"evidence_integrity_mismatch","category":"integrity_mismatch",
 "message":"Evidence bundle does not match the bundle schema at records.0.evidence_id: Evidence semantic identifier does not match its record. Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export.",
 "retryable":false}

The bundle is unmodified and was written by export_evidence_bundle. 6.5.0 imports it and re-exports it byte for byte.

Cause: db8128dc0 (refactor(evidence)!: require current coverage commitments) changed semanticProjection in src/domain/evidence.ts:

-  ...(evidence.relationship_coverage === null
-    ? {}
-    : { relationship_coverage: evidence.relationship_coverage }),
+  relationship_coverage: evidence.relationship_coverage,

So a record whose relationship_coverage is null now hashes "relationship_coverage":null where 6.5.0 hashed nothing, and its evidence_id changes. 6.5.0 already writes the field explicitly ("relationship_coverage": null) in every record I exported. The commit and the 6.6.0 release notes in #3476 say:

Historical records without the field are rejected; saved files are not migrated.

In practice, records that have the field are rejected too, as long as it is null. In my sample that is every record: WASM and EVM inspections. Records with non-null coverage keep their ID, because the field was hashed in both versions.

The same analysis now gets two IDs. For WETH9's runtime code (inspect_evm_interface, same input file):

Build	`evidence_id`
6.5.0	`ev_b76a66404fdef6a7998c624f808ec293eec4f966452bd8bc230eecc812d45fa2`
`main` @ `d72d856f5`, fresh run	`ev_940d15f49440c820a6bcb635495782a20744bf5311d08f54a3f401f02342fed1`

I recomputed both independently in Python (an RFC 8785 serializer, SHA-256 over the projection): the 6.5.0 rule reproduces all 3 IDs in a 6.5.0 bundle, and the main rule reproduces the fresh main ID. The same serializer reproduces the exported bundle file byte for byte, so the difference is only the projection.

Reproduction

In a rea-agents@6.5.0 MCP session, run inspect_evm_interface on any flat runtime bytecode, then export_evidence_bundle to small.json. I used WETH9 (0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2) from eth_getCode; the hex file has SHA-256 a9817472fdcc178e9cb0a7b8488edee744b0b65be99ae90d515b227fd27a3e5b.
rea evidence-import small.json with 6.5.0: {"imported":2,…}.
node scripts/rea.mjs evidence-import small.json with a build of main: the error above.

The same failure applies to import_evidence_bundle, compare and anything else that parses saved Evidence (I tested the CLI import only).

Expected behavior

One of the following, with the maintainers choosing:

Accept the 6.5.0 identity for records whose relationship_coverage is explicitly null. For example, if the current-rule ID doesn't match, verify against the previous projection and keep the record's original ID. Records missing the field could still be rejected, which matches the announced breaking change.
Or keep rejecting, but say why. Use a distinct reason/code, such as an Evidence identity produced by an earlier REA version, instead of integrity_mismatch with "Supply an unmodified bundle". Also correct the release note to "all records whose relationship coverage is null, including records that state it explicitly", and point to a way to recreate the Evidence.

Impact and scope

Everyone with saved 6.5.0 bundles (the current release) after 6.6.0 ships. The message suggests the file was tampered with, which is the opposite of what happened.
Tested: 6.5.0 (published image) and main @ d72d856f5 built from source in node:24-bookworm. Not tested: other operations' records, MCP import, inspect-evidence-bundle.

Other checks from the same session. No bugs found; listed here for context. On 6.5.0, bundle round trips were byte-identical through both CLI and MCP export. Edits to results or subject digests, manifest changes, record reordering, duplicate records and unknown fields were rejected. Whitespace and trailing-newline changes were accepted and re-exported in canonical form. Edits to the display-only subject.local_path/name were accepted, as the projection comment says.

Originally created by @Rinkia on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4436 ### Before submitting - [x] I searched the open and closed issues for a related report (evidence_integrity_mismatch, "semantic identifier does not match", historical evidence import, relationship_coverage identity). Related but different: #2050 (a single record passed as a bundle) and #4398 (auditing bundles before sharing). The change is announced as a breaking change in the open release PR #3476, but its wording covers a smaller set of records than the ones that fail; see below. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior A bundle exported by released `rea-agents@6.5.0`, then imported unchanged with `main` @ `d72d856f5`: ```json {"code":"evidence_integrity_mismatch","category":"integrity_mismatch", "message":"Evidence bundle does not match the bundle schema at records.0.evidence_id: Evidence semantic identifier does not match its record. Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export.", "retryable":false} ``` The bundle is unmodified and was written by `export_evidence_bundle`. 6.5.0 imports it and re-exports it byte for byte. Cause: db8128dc0 (`refactor(evidence)!: require current coverage commitments`) changed `semanticProjection` in `src/domain/evidence.ts`: ```diff - ...(evidence.relationship_coverage === null - ? {} - : { relationship_coverage: evidence.relationship_coverage }), + relationship_coverage: evidence.relationship_coverage, ``` So a record whose `relationship_coverage` is `null` now hashes `"relationship_coverage":null` where 6.5.0 hashed nothing, and its `evidence_id` changes. 6.5.0 already writes the field explicitly (`"relationship_coverage": null`) in every record I exported. The commit and the 6.6.0 release notes in #3476 say: > Historical records without the field are rejected; saved files are not migrated. In practice, records that *have* the field are rejected too, as long as it is `null`. In my sample that is every record: WASM and EVM inspections. Records with non-null coverage keep their ID, because the field was hashed in both versions. The same analysis now gets two IDs. For WETH9's runtime code (`inspect_evm_interface`, same input file): | Build | `evidence_id` | |---|---| | 6.5.0 | `ev_b76a66404fdef6a7998c624f808ec293eec4f966452bd8bc230eecc812d45fa2` | | `main` @ `d72d856f5`, fresh run | `ev_940d15f49440c820a6bcb635495782a20744bf5311d08f54a3f401f02342fed1` | I recomputed both independently in Python (an RFC 8785 serializer, SHA-256 over the projection): the 6.5.0 rule reproduces all 3 IDs in a 6.5.0 bundle, and the main rule reproduces the fresh main ID. The same serializer reproduces the exported bundle file byte for byte, so the difference is only the projection. ### Reproduction 1. In a `rea-agents@6.5.0` MCP session, run `inspect_evm_interface` on any flat runtime bytecode, then `export_evidence_bundle` to `small.json`. I used WETH9 (`0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2`) from `eth_getCode`; the hex file has SHA-256 `a9817472fdcc178e9cb0a7b8488edee744b0b65be99ae90d515b227fd27a3e5b`. 2. `rea evidence-import small.json` with 6.5.0: `{"imported":2,…}`. 3. `node scripts/rea.mjs evidence-import small.json` with a build of `main`: the error above. The same failure applies to `import_evidence_bundle`, `compare` and anything else that parses saved Evidence (I tested the CLI import only). ### Expected behavior One of the following, with the maintainers choosing: - Accept the 6.5.0 identity for records whose `relationship_coverage` is explicitly `null`. For example, if the current-rule ID doesn't match, verify against the previous projection and keep the record's original ID. Records missing the field could still be rejected, which matches the announced breaking change. - Or keep rejecting, but say why. Use a distinct reason/code, such as an Evidence identity produced by an earlier REA version, instead of `integrity_mismatch` with "Supply an unmodified bundle". Also correct the release note to "all records whose relationship coverage is null, including records that state it explicitly", and point to a way to recreate the Evidence. ### Impact and scope - Everyone with saved 6.5.0 bundles (the current release) after 6.6.0 ships. The message suggests the file was tampered with, which is the opposite of what happened. - Tested: 6.5.0 (published image) and `main` @ `d72d856f5` built from source in `node:24-bookworm`. Not tested: other operations' records, MCP import, `inspect-evidence-bundle`. **Other checks from the same session.** No bugs found; listed here for context. On 6.5.0, bundle round trips were byte-identical through both CLI and MCP export. Edits to results or subject digests, manifest changes, record reordering, duplicate records and unknown fields were rejected. Whitespace and trailing-newline changes were accepted and re-exported in canonical form. Edits to the display-only `subject.local_path`/`name` were accepted, as the projection comment says.

bsakaguchi commented

2026-10-16 21:09:58 +00:00

Author

Owner

@pablodiazjorge commented on GitHub (Oct 16, 2026):

A data point for the other side of this: records that lack the field, which is the announced case. They hit the same misleading message, which supports your second option even for records the release note does cover.

rea-lab publishes Evidence bundles exported by rea-agents@6.4.0 with its writeups: native Ghidra operations on PE targets, such as binary_overview, search_strings and procedure_pseudo_code. I imported the five original, unsanitised bundles on Windows 11 with evidence-import --format json:

Bundle	Records	6.4.0	`main` @ `781d5751a`
winhellocpp	18	imported 18	`evidence_integrity_mismatch`
own-xor -O0	7	imported 7	`evidence_integrity_mismatch`
own-xor -O2	7	imported 7	`evidence_integrity_mismatch`
wallace src	11	imported 11	`evidence_integrity_mismatch`
wallace version2	21	imported 21	`evidence_integrity_mismatch`

The message is the same each time: Evidence bundle does not match the bundle schema at records.0.relationship_coverage: Invalid input: expected object, received undefined (N more issues). Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export. N is always the record count minus one, so the missing field is the only issue. The published, sanitised copies (only subject.name and local_path replaced) fail the same way.

So rejecting these matches db8128dc0, but the bundles are unmodified and were written by export_evidence_bundle, which is exactly what the message tells the user to supply. A reason naming the REA version that wrote the record would help here too.

main was built with npm run compile, because build:cached now stops on the missing 6.5.0 Windows native artifact, which this import path does not use.

@pablodiazjorge commented on GitHub (Oct 16, 2026): A data point for the other side of this: records that lack the field, which is the announced case. They hit the same misleading message, which supports your second option even for records the release note does cover. [rea-lab](https://github.com/pablodiazjorge/rea-lab) publishes Evidence bundles exported by `rea-agents@6.4.0` with its writeups: native Ghidra operations on PE targets, such as `binary_overview`, `search_strings` and `procedure_pseudo_code`. I imported the five original, unsanitised bundles on Windows 11 with `evidence-import --format json`: | Bundle | Records | 6.4.0 | `main` @ `781d5751a` | |---|---|---|---| | winhellocpp | 18 | imported 18 | `evidence_integrity_mismatch` | | own-xor -O0 | 7 | imported 7 | `evidence_integrity_mismatch` | | own-xor -O2 | 7 | imported 7 | `evidence_integrity_mismatch` | | wallace src | 11 | imported 11 | `evidence_integrity_mismatch` | | wallace version2 | 21 | imported 21 | `evidence_integrity_mismatch` | The message is the same each time: `Evidence bundle does not match the bundle schema at records.0.relationship_coverage: Invalid input: expected object, received undefined (N more issues). Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export.` N is always the record count minus one, so the missing field is the only issue. The published, sanitised copies (only `subject.name` and `local_path` replaced) fail the same way. So rejecting these matches db8128dc0, but the bundles are unmodified and were written by `export_evidence_bundle`, which is exactly what the message tells the user to supply. A reason naming the REA version that wrote the record would help here too. `main` was built with `npm run compile`, because `build:cached` now stops on the missing 6.5.0 Windows native artifact, which this import path does not use.

bsakaguchi referenced this issue

2026-10-17 21:01:31 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi closed this issue

2026-10-31 00:32:37 +00:00

bsakaguchi commented

2026-10-31 00:32:38 +00:00

Author

Owner

@Rinkia commented on GitHub (Oct 18, 2026):

Status on the published release: rea-agents@6.7.0 rejects the same unmodified 6.5.0 bundle (small-6.5.0.json, WETH9 inspect_evm_interface Evidence) with:

code: evidence_integrity_mismatch
message: "Evidence bundle does not match the bundle schema at records.0.evidence_id: Evidence semantic identifier does not match its record. Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export."

The 6.6.0 release note now says to regenerate old Evidence and that saved files are not migrated, which covers the documentation half of this issue. The error itself still says to "Supply an unmodified bundle", and that's wrong for an unmodified 6.5.0 bundle. PR #4442 (tested earlier on e9a2bce33) is still open.

@Rinkia commented on GitHub (Oct 18, 2026): Status on the published release: `rea-agents@6.7.0` rejects the same unmodified 6.5.0 bundle (`small-6.5.0.json`, WETH9 `inspect_evm_interface` Evidence) with: ``` code: evidence_integrity_mismatch message: "Evidence bundle does not match the bundle schema at records.0.evidence_id: Evidence semantic identifier does not match its record. Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export." ``` The 6.6.0 release note now says to regenerate old Evidence and that saved files are not migrated, which covers the documentation half of this issue. The error itself still says to "Supply an unmodified bundle", and that's wrong for an unmodified 6.5.0 bundle. PR #4442 (tested earlier on e9a2bce33) is still open.

bsakaguchi commented

2026-10-31 00:32:38 +00:00

Author

Owner

@pablodiazjorge commented on GitHub (Oct 18, 2026):

Re-checked the other half on the published release: rea-agents@6.7.0 (built from the rea-agents-6.7.0 tag with npm run compile; Windows 11, Node 24.18.0) still rejects all five rea-lab bundles from my earlier comment, exported by 6.4.0 without relationship_coverage (64 records), with the same message as on main:

Evidence bundle does not match the bundle schema at records.0.relationship_coverage: Invalid input: expected object, received undefined (N more issues). Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export.

Two details in the JSON output matter for agents: remediation.action repeats that same sentence, so an agent that follows the remediation is told to resubmit the file it already has, and details.reason is the generic Evidence bundle validation failed.

#4442 fixes the explicit-null case reported here, but it doesn't touch this message: its diff changes semanticProjection, the identity check, a test and the migration guide. Records without the field, which the 6.6.0 release note rejects by design, would keep getting "Supply an unmodified bundle" after it merges. A distinct reason, with a remediation that says what the release note already says ("Regenerate old Evidence/snapshots … with this release"), would cover both cases.

@pablodiazjorge commented on GitHub (Oct 18, 2026): Re-checked the other half on the published release: `rea-agents@6.7.0` (built from the `rea-agents-6.7.0` tag with `npm run compile`; Windows 11, Node 24.18.0) still rejects all five rea-lab bundles from my earlier comment, exported by 6.4.0 without `relationship_coverage` (64 records), with the same message as on main: `Evidence bundle does not match the bundle schema at records.0.relationship_coverage: Invalid input: expected object, received undefined (N more issues). Supply an unmodified bundle written by export_evidence_bundle or rea evidence-export.` Two details in the JSON output matter for agents: `remediation.action` repeats that same sentence, so an agent that follows the remediation is told to resubmit the file it already has, and `details.reason` is the generic `Evidence bundle validation failed`. #4442 fixes the explicit-null case reported here, but it doesn't touch this message: its diff changes `semanticProjection`, the identity check, a test and the migration guide. Records without the field, which the 6.6.0 release note rejects by design, would keep getting "Supply an unmodified bundle" after it merges. A distinct reason, with a remediation that says what the release note already says ("Regenerate old Evidence/snapshots … with this release"), would cover both cases.

bsakaguchi commented

2026-10-31 00:32:38 +00:00

Author

Owner

@morluto commented on GitHub (Oct 23, 2026):

Resolved on main in 546b2a6cf. The permitted documented-breaking-change option is implemented: exact pre-6.6 explicit-null commitments are diagnosed as earlier identities, remain rejected without rewriting IDs, and recovery says recreate observations. CHANGELOG and MCP guide explicitly include serialized null. Validation: src/domain/evidence.test.ts — diagnoses a previous explicit-null coverage commitment without admitting or rewriting its ID; recorded execution at e1d9ebd3c. One selected Evidence control passed with28 skipped; local source/docs correction verified, not hosted GitHub release-note editing or a new installed6.5 bundle run.

@morluto commented on GitHub (Oct 23, 2026): Resolved on main in [546b2a6cf](https://github.com/morluto/rea/commit/546b2a6cf6ffbf4ca693799ec6ae587c23157501). The permitted documented-breaking-change option is implemented: exact pre-6.6 explicit-null commitments are diagnosed as earlier identities, remain rejected without rewriting IDs, and recovery says recreate observations. CHANGELOG and MCP guide explicitly include serialized null. Validation: `src/domain/evidence.test.ts` — diagnoses a previous explicit-null coverage commitment without admitting or rewriting its ID; recorded execution at `e1d9ebd3c`. One selected Evidence control passed with28 skipped; local source/docs correction verified, not hosted GitHub release-note editing or a new installed6.5 bundle run.

bsakaguchi referenced this issue

2026-10-31 00:55:14 +00:00

[PR #1762] [MERGED] fix(javascript): honor package exports before sibling files #4802

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1761

No description provided.

Rows
Columns

[GH-ISSUE #4436] [Bug] Unmodified 6.5.0 Evidence bundles are rejected on main as an integrity mismatch: records with relationship_coverage: null changed identity #1761

Before submitting

Area

Observed behavior

Reproduction

Expected behavior

Impact and scope

[GH-ISSUE #4436] [Bug] Unmodified 6.5.0 Evidence bundles are rejected on main as an integrity mismatch: records with `relationship_coverage: null` changed identity #1761