[PR #7308] fix(reconstruction): reject inconsistent artifact comparisons #7812

Open
opened 2026-11-05 01:47:52 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7308
Author: @ducanhnguyen223
Created: 10/26/2026
Status: 🔄 Open

Base: main ← Head: codex/fix-7304-artifact-consistency


📝 Commits (3)

  • f0cd6f8 fix(reconstruction): reject inconsistent artifact comparisons
  • e7163b9 fix(reconstruction): corroborate artifact equality with sources
  • 4942599 fix(reconstruction): corroborate definite artifact outcomes

📊 Changes

6 files changed (+714 additions, -24 deletions)

View changed files

📝 src/domain/artifactComparison.ts (+36 -17)
📝 src/domain/comparisonEvidenceResult.ts (+4 -4)
📝 src/domain/reconstructionVerification.test.ts (+258 -2)
📝 src/domain/reconstructionVerification.ts (+90 -1)
➕ tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts (+255 -0)
📝 tests/boundary/mcp/artifactMcp.test.ts (+71 -0)

📄 Description

Summary

Fixes #7304. Reject imported artifact-comparison Evidence that declares unchanged while retained rows or aggregate counts record changes, before it can support a passing structural-artifact claim.

Problem and expected behavior

verify_reconstruction can trust an aggregate status of unchanged even when the same comparison retains change rows or non-zero added, removed, changed, unknown, or contradiction counts. That inconsistency should fail closed with the existing typed evidence_integrity_mismatch outcome. Consistent unchanged and changed comparisons keep their existing behavior.

Change and scope

Validate the unchanged-status invariant in the artifact comparison result owner before reconstruction verification classifies the claim. Validate integrity-mismatch citations and bind both manifest IDs to the cited source inventories. For definite artifact outcomes, replay the shared comparison owner and withhold a verdict when its result disagrees; repeat claims for the same ordered source pair reuse the replay within one verification. This is an ordinary fix inside the existing Evidence admission path; it does not change schemas, tool signatures, provider behavior, or generated catalogs. The original comparison Evidence and citations remain retained.

Contract and boundary impact

  • Semantic owner: src/domain/artifactComparison.ts owns result consistency and comparison semantics; comparisonEvidenceResult.ts validates admitted citation closure. Reconstruction binds manifest identities to sources and corroborates definite outcomes using that same comparator.
  • Scope: Existing Evidence integrity boundary; no target-execution, injection, mutation, or protection-recovery capability is added.
  • CLI/MCP contract: Inconsistent imported comparisons now return the existing typed integrity-mismatch error from verify_reconstruction; no schema or catalog change.
  • Provider/platform compatibility: None.
  • Evidence/provenance: The inconsistent aggregate is rejected; the MCP regression confirms the original Evidence is unchanged.
  • Target effects: No target or provider is launched. Boundary coverage uses synthetic canonical Evidence and an in-memory MCP harness.
  • Generated files/package: No impact.

Evidence and regression coverage

  • Tests: Domain regression covers a retained changed row and the same non-zero counts with rows removed. MCP boundary regression checks the typed error and Evidence preservation.

  • Base reproduction: Before the fix, the domain regression failed because the inconsistent comparison did not throw; the other 12 tests in that file passed.

  • Remaining gaps: No real provider or target-runtime execution; neither is needed to check consistency of imported comparison Evidence.

  • Boundary regression: tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts; no lifecycle change.

  • Additional coverage: Authenticated MCP and domain regressions exercise unknown source coverage, unhashed and mismatched content, mismatched manifest identities, forged changed and unchanged outcomes, valid changed and unknown controls, and PyInstaller declaration-only equality. The SDK JSON Schema cannot express the cross-field refinement; the named handler enforces it at runtime.

  • Measured cost: A single-sample in-memory MCP run over two CLI-produced 3,349-occurrence ZIP inventories took 164.5 ms before and 311.3 ms after for the first single claim; warm calls were 144.8 ms and 298.9 ms. A warm 20-claim batch over one source pair took 2,814.5 ms and 3,113.1 ms. Whole-process peak RSS was 785 and 769 MiB; no independent process-family sampling, token counts, schema discovery, or artifact creation costs were measured. The replay sample used commit 6ab256bfe, before the later extension to definite changed outcomes; its unchanged-claim path is equivalent. These are single samples with warm OS caches, not a statistical comparison.

Validation performed

  • npm run test:local -- src/domain/reconstructionVerification.test.ts src/domain/reconstructionVerification.functionSources.test.ts src/domain/artifactComparison.test.ts — 103 passed.

  • npm run test:focused -- tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts tests/boundary/mcp/artifactMcp.test.ts — 18 passed, including the declaration-only control.

  • npm run check:fast and git diff --check — passed on the current diff.

  • fnm exec --using=24.22.1 npm run test:local -- src/domain/reconstructionVerification.test.ts src/domain/reconstructionVerification.functionSources.test.ts src/domain/staticRuntimeCorrelation.test.ts — 3 files, 96 tests passed.

  • fnm exec --using=24.22.1 npm run test:focused -- tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts — 1 test passed.

  • fnm exec --using=24.22.1 npm run check:fast — typecheck and lint passed.

  • fnm exec --using=24.22.1 npm run format:check and git diff --check — passed.

Compatibility, safety, and release

  • Breaking changes/migration: Imported comparisons whose definite status is not corroborated by their cited source inventories now yield unknown; mismatched manifest identities are rejected. Recompute the comparison from its source Evidence when needed.
  • Real Hopper/Ghidra, browser, or OS coverage: Not applicable.
  • Package/release metadata: None.
  • Security/privacy/process review: No new process, network, or target behavior.

Author readiness

Human owner: @ducanhnguyen223

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

This is a draft for early feedback only; maintainer review is not being requested until the human owner completes the declarations above.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7308 **Author:** [@ducanhnguyen223](https://github.com/ducanhnguyen223) **Created:** 10/26/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `codex/fix-7304-artifact-consistency` --- ### 📝 Commits (3) - [`f0cd6f8`](https://github.com/morluto/rea/commit/f0cd6f8d015bff1e4fb18751a6e211defb8d494a) fix(reconstruction): reject inconsistent artifact comparisons - [`e7163b9`](https://github.com/morluto/rea/commit/e7163b925f80bd392a3e44bc14b195659ad76fba) fix(reconstruction): corroborate artifact equality with sources - [`4942599`](https://github.com/morluto/rea/commit/4942599e739ce672ec656a5a54d060da31e2841f) fix(reconstruction): corroborate definite artifact outcomes ### 📊 Changes **6 files changed** (+714 additions, -24 deletions) <details> <summary>View changed files</summary> 📝 `src/domain/artifactComparison.ts` (+36 -17) 📝 `src/domain/comparisonEvidenceResult.ts` (+4 -4) 📝 `src/domain/reconstructionVerification.test.ts` (+258 -2) 📝 `src/domain/reconstructionVerification.ts` (+90 -1) ➕ `tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts` (+255 -0) 📝 `tests/boundary/mcp/artifactMcp.test.ts` (+71 -0) </details> ### 📄 Description ## Summary Fixes #7304. Reject imported artifact-comparison Evidence that declares `unchanged` while retained rows or aggregate counts record changes, before it can support a passing structural-artifact claim. ## Problem and expected behavior `verify_reconstruction` can trust an aggregate status of `unchanged` even when the same comparison retains change rows or non-zero added, removed, changed, unknown, or contradiction counts. That inconsistency should fail closed with the existing typed `evidence_integrity_mismatch` outcome. Consistent unchanged and changed comparisons keep their existing behavior. ## Change and scope Validate the unchanged-status invariant in the artifact comparison result owner before reconstruction verification classifies the claim. Validate integrity-mismatch citations and bind both manifest IDs to the cited source inventories. For definite artifact outcomes, replay the shared comparison owner and withhold a verdict when its result disagrees; repeat claims for the same ordered source pair reuse the replay within one verification. This is an ordinary fix inside the existing Evidence admission path; it does not change schemas, tool signatures, provider behavior, or generated catalogs. The original comparison Evidence and citations remain retained. ## Contract and boundary impact - **Semantic owner:** `src/domain/artifactComparison.ts` owns result consistency and comparison semantics; `comparisonEvidenceResult.ts` validates admitted citation closure. Reconstruction binds manifest identities to sources and corroborates definite outcomes using that same comparator. - **Scope:** Existing Evidence integrity boundary; no target-execution, injection, mutation, or protection-recovery capability is added. - **CLI/MCP contract:** Inconsistent imported comparisons now return the existing typed integrity-mismatch error from `verify_reconstruction`; no schema or catalog change. - **Provider/platform compatibility:** None. - **Evidence/provenance:** The inconsistent aggregate is rejected; the MCP regression confirms the original Evidence is unchanged. - **Target effects:** No target or provider is launched. Boundary coverage uses synthetic canonical Evidence and an in-memory MCP harness. - **Generated files/package:** No impact. ## Evidence and regression coverage - **Tests:** Domain regression covers a retained changed row and the same non-zero counts with rows removed. MCP boundary regression checks the typed error and Evidence preservation. - **Base reproduction:** Before the fix, the domain regression failed because the inconsistent comparison did not throw; the other 12 tests in that file passed. - **Remaining gaps:** No real provider or target-runtime execution; neither is needed to check consistency of imported comparison Evidence. - **Boundary regression:** `tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts`; no lifecycle change. - **Additional coverage:** Authenticated MCP and domain regressions exercise unknown source coverage, unhashed and mismatched content, mismatched manifest identities, forged changed and unchanged outcomes, valid changed and unknown controls, and PyInstaller declaration-only equality. The SDK JSON Schema cannot express the cross-field refinement; the named handler enforces it at runtime. - **Measured cost:** A single-sample in-memory MCP run over two CLI-produced 3,349-occurrence ZIP inventories took 164.5 ms before and 311.3 ms after for the first single claim; warm calls were 144.8 ms and 298.9 ms. A warm 20-claim batch over one source pair took 2,814.5 ms and 3,113.1 ms. Whole-process peak RSS was 785 and 769 MiB; no independent process-family sampling, token counts, schema discovery, or artifact creation costs were measured. The replay sample used commit `6ab256bfe`, before the later extension to definite changed outcomes; its unchanged-claim path is equivalent. These are single samples with warm OS caches, not a statistical comparison. ## Validation performed - `npm run test:local -- src/domain/reconstructionVerification.test.ts src/domain/reconstructionVerification.functionSources.test.ts src/domain/artifactComparison.test.ts` — 103 passed. - `npm run test:focused -- tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts tests/boundary/mcp/artifactMcp.test.ts` — 18 passed, including the declaration-only control. - `npm run check:fast` and `git diff --check` — passed on the current diff. - `fnm exec --using=24.22.1 npm run test:local -- src/domain/reconstructionVerification.test.ts src/domain/reconstructionVerification.functionSources.test.ts src/domain/staticRuntimeCorrelation.test.ts` — 3 files, 96 tests passed. - `fnm exec --using=24.22.1 npm run test:focused -- tests/boundary/mcp/artifactComparisonIntegrityMcp.test.ts` — 1 test passed. - `fnm exec --using=24.22.1 npm run check:fast` — typecheck and lint passed. - `fnm exec --using=24.22.1 npm run format:check` and `git diff --check` — passed. ## Compatibility, safety, and release - Breaking changes/migration: Imported comparisons whose definite status is not corroborated by their cited source inventories now yield unknown; mismatched manifest identities are rejected. Recompute the comparison from its source Evidence when needed. - Real Hopper/Ghidra, browser, or OS coverage: Not applicable. - Package/release metadata: None. - Security/privacy/process review: No new process, network, or target behavior. ## Author readiness Human owner: @ducanhnguyen223 - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. This is a draft for early feedback only; maintainer review is not being requested until the human owner completes the declarations above. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7812
No description provided.