morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #4075] [MERGED] fix(reconstruction): require comparable cited Evidence for coverage #6039

New issue

Closed

opened 2026-10-31 01:11:22 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:11:22 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4075
Author: @N0zoM1z0
Created: 10/15/2026
Status: ✅ Merged
Merged: 10/15/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/reconstruction-coverage-evidence-authority-3357

📝 Commits (1)

a31456b fix(reconstruction): require comparable cited Evidence for coverage

📊 Changes

9 files changed (+569 additions, -81 deletions)

View changed files

➕ docs/reconstruction-coverage.md (+43 -0)
📝 docs/reconstruction-obligation-ledgers.md (+3 -0)
📝 src/domain/reconstructionCoverage.fixture.ts (+5 -0)
📝 src/domain/reconstructionCoverage.test.ts (+23 -0)
📝 src/domain/reconstructionCoverage.ts (+70 -27)
📝 src/domain/reconstructionCoverageEvaluation.ts (+57 -54)
➕ src/domain/reconstructionCoverageEvidence.test.ts (+244 -0)
➕ src/domain/reconstructionCoverageEvidence.ts (+64 -0)
📝 tests/boundary/mcp/reconstructionCoverageMcp.test.ts (+60 -0)

📄 Description

Summary

Fixes #3357. A passing controlled-replay result supported only by inferred analyst Evidence now remains unknown, with the cited ID and a specific authority/confidence explanation. Qualifying observed replay still permits ready; missing/tampered Evidence remains a typed input error.

Problem and expected behavior

The evaluator authenticated bundles but discarded the record metadata for subsequent proof checks. Contract labels and a caller's pass summary could therefore promote an inference into replay proof, even when its record explicitly disclaimed a replay. A separate observed package proof in the bundle must not repair that result.

Change and scope

Retain parsed Evidence and unique artifact/surface/claim/owner indexes at the validation boundary. Authentication consumes the already parsed bundle through its existing owner; all phases reuse the facts. Removes duplicate index construction and the per-surface full owner-table scan. Duplicate ownership stays visible and returns the existing partial outcome.
Qualify only each observation's cited records. Equivalent latest assessments need at least one qualifying latest observation; conflicting ties stay unknown and older observed proof cannot replace a newer weak result. Runtime proof requires observed comparable provenance; deterministic shipped-artifact derivations remain valid for static contracts.
Apply the same qualification owner to package/integration proofs. Weak proof cannot establish either a successful or failed workflow. Preserve the relevant Evidence IDs and actionable unknowns.
Correct the synthetic complete fixture's verifier/clean-install provenance, which previously used default shipped-artifact tags. Document the exact qualification rules.

Per-request indexes are keyed by the validated snapshot's declaration/Evidence IDs and discarded after evaluation. No persistent/live-state cache, new provider, execution capability, tool call or JSON contract is introduced. Production behavior affected: [x] read supplied files/captured data; existing contribution scope.

Evidence and regression coverage

Public Linux CLI on initial main d7651b6: base=ready, weak=ready (bug), declared authority mismatch=unknown, missing ID=invalid_request/exit 1. Fixed: base=ready, weak=unknown, declared mismatch=unknown, missing ID still typed input failure. Record IDs/manifests authenticate; only the result's Evidence reference changes for the weak case.
First new metadata suite on unfixed production code: 17 failures and 21 passing cases/controls. Final 47 cases across three files pass, covering provenance/confidence, static and runtime contracts, cited contextual inference, latest/tied observations, package proof classes, observed versus inferred failure, duplicate owners, and MCP tamper rejection.
Actual Linux Process Capture: /usr/bin/printf and an owned Node candidate each ran twice. Raw marker output and zero exit agree; actual executable/candidate-source hashes match the commitments. Public CLI and stdio MCP both return ready for these observed references and unknown when the verifier instead cites inference. Replaying that same actual weak input on parent 47b4628 still returns the incorrect ready. Input bytes remain unchanged; process settlement and MCP transport cleanup complete.
The actual target claim is limited to the selected output and exit. Captures retain process-sampling/image-binding, inherited-environment, network and unselected-filesystem limitations. This is not full printf parity or proof of a complete reconstruction. The coverage metadata is assembled by the probe over unchanged production capture/analysis Evidence.

Evidence assertions: [x] observed/derived/inferred remain distinct; [x] source identity/provenance retained; [x] insufficient proof remains visible.

Validation performed

npm run test:focused -- src/domain/reconstructionCoverageEvidence.test.ts src/domain/reconstructionCoverage.test.ts tests/boundary/mcp/reconstructionCoverageMcp.test.ts — 47 passed on final runtime source.
npm run check — passed: typecheck, lint, formatting, unused-code, metadata, module boundaries and architecture guards. Zero lint findings in touched coverage files.
npm run docs:check — passed, including five documentation consistency tests.
Actual capture/CLI/stdio-MCP probes and four original CLI controls — passed as scoped above. Local verification is Linux; no real Hopper/Ghidra or Windows/macOS runtime claim for this domain change.

Completed-task cost comparison

Parent 47b4628 versus a31456b, pinned Node 24.18.0/npm 11.16.0, identical inputs and completed CLI operations. Two alternating samples per cache mode; baseline / fixed:

Input / Node compile cache	Wall (s)	Reaped child CPU (s)	Sampled peak family RSS (MiB)
actual process Evidence, cold	1.82 / 1.66	2.21 / 2.05	328.9 / 316.6
actual process Evidence, warm	1.60 / 1.61	1.95 / 1.97	326.2 / 326.0
synthetic 5,001-surface model, cold	2.21 / 1.86	2.64 / 2.27	339.9 / 338.8
synthetic 5,001-surface model, warm	1.98 / 1.61	2.39 / 2.04	341.8 / 332.8

Every successful result is identical. Full response size stays 194 cl100k_base tokens / 442 bytes for actual observations and 123 tokens / 310 bytes for the capacity model. The corrected actual weak response is 260 tokens versus 193 previously (+67 necessary tokens for the reason/probe).

The capacity workload is explicitly synthetic metadata: 5,001 declared surfaces/owners and one claim. It measures evaluation scale and establishes no real-provider/reconstruction coverage. The real small task's warm timing is effectively unchanged; the model shows lower work after removing repeated scans. Shared host, two samples, uncontrolled OS caches and 200 ms RSS sampling limit conclusions; no general speedup guarantee. Cold disables Node compile caching; warm primes per-version caches. Wall uses child completion, CPU covers reaped children, and sampled RSS may double-count shared pages/miss short processes. These are retained-response proxy token counts: no model consumed the payloads; complete agent-task tokens, input/schema negotiation and billing remain unmeasured.

Heavy local tasks ran serially on two CPU cores, at most two test workers, with Node heap defaults bounded and a task-local memory guard within the user-approved 8 GiB overall budget. Measured validation family peak was about 4.55 GiB; no memory guard triggered.

Compatibility and release

Call/configuration/schema shapes remain unchanged. Corrections withhold readiness for weak or unrelated proof, and retain unknown/failure provenance. Complete Evidence stays in the caller's authenticated bundle. Existing ownership/freshness/contradiction rules remain in force; no package/version change.

Author readiness

Human owner: @N0zoM1z0

The human owner explicitly authorized delegated review, proxy completion of these declarations, ready-for-review publication and normal merge for this batch. The agent reviewed the final diff and evidence; this records that delegation and does not assert a separate manual review occurred.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4075 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/15/2026 **Status:** ✅ Merged **Merged:** 10/15/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/reconstruction-coverage-evidence-authority-3357` --- ### 📝 Commits (1) - [`a31456b`](https://github.com/morluto/rea/commit/a31456b265a5ea39e062b67d8fb3632ee27e8cac) fix(reconstruction): require comparable cited Evidence for coverage ### 📊 Changes **9 files changed** (+569 additions, -81 deletions) <details> <summary>View changed files</summary> ➕ `docs/reconstruction-coverage.md` (+43 -0) 📝 `docs/reconstruction-obligation-ledgers.md` (+3 -0) 📝 `src/domain/reconstructionCoverage.fixture.ts` (+5 -0) 📝 `src/domain/reconstructionCoverage.test.ts` (+23 -0) 📝 `src/domain/reconstructionCoverage.ts` (+70 -27) 📝 `src/domain/reconstructionCoverageEvaluation.ts` (+57 -54) ➕ `src/domain/reconstructionCoverageEvidence.test.ts` (+244 -0) ➕ `src/domain/reconstructionCoverageEvidence.ts` (+64 -0) 📝 `tests/boundary/mcp/reconstructionCoverageMcp.test.ts` (+60 -0) </details> ### 📄 Description ## Summary Fixes #3357. A passing controlled-replay result supported only by inferred analyst Evidence now remains `unknown`, with the cited ID and a specific authority/confidence explanation. Qualifying observed replay still permits `ready`; missing/tampered Evidence remains a typed input error. ## Problem and expected behavior The evaluator authenticated bundles but discarded the record metadata for subsequent proof checks. Contract labels and a caller's `pass` summary could therefore promote an inference into replay proof, even when its record explicitly disclaimed a replay. A separate observed package proof in the bundle must not repair that result. ## Change and scope - Retain parsed Evidence and unique artifact/surface/claim/owner indexes at the validation boundary. Authentication consumes the already parsed bundle through its existing owner; all phases reuse the facts. Removes duplicate index construction and the per-surface full owner-table scan. Duplicate ownership stays visible and returns the existing `partial` outcome. - Qualify only each observation's cited records. Equivalent latest assessments need at least one qualifying latest observation; conflicting ties stay unknown and older observed proof cannot replace a newer weak result. Runtime proof requires observed comparable provenance; deterministic shipped-artifact derivations remain valid for static contracts. - Apply the same qualification owner to package/integration proofs. Weak proof cannot establish either a successful or failed workflow. Preserve the relevant Evidence IDs and actionable unknowns. - Correct the synthetic complete fixture's verifier/clean-install provenance, which previously used default shipped-artifact tags. Document the exact [qualification rules](https://github.com/morluto/rea/blob/fix/reconstruction-coverage-evidence-authority-3357/docs/reconstruction-coverage.md). Per-request indexes are keyed by the validated snapshot's declaration/Evidence IDs and discarded after evaluation. No persistent/live-state cache, new provider, execution capability, tool call or JSON contract is introduced. Production behavior affected: [x] read supplied files/captured data; existing [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). ## Evidence and regression coverage - Public Linux CLI on initial main `d7651b6`: base=`ready`, weak=`ready` (bug), declared authority mismatch=`unknown`, missing ID=`invalid_request`/exit 1. Fixed: base=`ready`, weak=`unknown`, declared mismatch=`unknown`, missing ID still typed input failure. Record IDs/manifests authenticate; only the result's Evidence reference changes for the weak case. - First new metadata suite on unfixed production code: **17 failures and 21 passing cases/controls**. Final **47 cases across three files pass**, covering provenance/confidence, static and runtime contracts, cited contextual inference, latest/tied observations, package proof classes, observed versus inferred failure, duplicate owners, and MCP tamper rejection. - **Actual Linux Process Capture:** `/usr/bin/printf` and an owned Node candidate each ran twice. Raw marker output and zero exit agree; actual executable/candidate-source hashes match the commitments. Public CLI and **stdio MCP** both return `ready` for these observed references and `unknown` when the verifier instead cites inference. Replaying that same actual weak input on parent `47b4628` still returns the incorrect `ready`. Input bytes remain unchanged; process settlement and MCP transport cleanup complete. - The actual target claim is limited to the selected output and exit. Captures retain process-sampling/image-binding, inherited-environment, network and unselected-filesystem limitations. This is not full printf parity or proof of a complete reconstruction. The coverage metadata is assembled by the probe over unchanged production capture/analysis Evidence. Evidence assertions: [x] observed/derived/inferred remain distinct; [x] source identity/provenance retained; [x] insufficient proof remains visible. ## Validation performed - `npm run test:focused -- src/domain/reconstructionCoverageEvidence.test.ts src/domain/reconstructionCoverage.test.ts tests/boundary/mcp/reconstructionCoverageMcp.test.ts` — **47 passed** on final runtime source. - `npm run check` — passed: typecheck, lint, formatting, unused-code, metadata, module boundaries and architecture guards. **Zero lint findings in touched coverage files.** - `npm run docs:check` — passed, including five documentation consistency tests. - Actual capture/CLI/stdio-MCP probes and four original CLI controls — passed as scoped above. Local verification is Linux; no real Hopper/Ghidra or Windows/macOS runtime claim for this domain change. ### Completed-task cost comparison Parent `47b4628` versus `a31456b`, pinned Node 24.18.0/npm 11.16.0, identical inputs and completed CLI operations. Two alternating samples per cache mode; **baseline / fixed**: | Input / Node compile cache | Wall (s) | Reaped child CPU (s) | Sampled peak family RSS (MiB) | | --- | --- | --- | --- | | actual process Evidence, cold | 1.82 / 1.66 | 2.21 / 2.05 | 328.9 / 316.6 | | actual process Evidence, warm | 1.60 / 1.61 | 1.95 / 1.97 | 326.2 / 326.0 | | synthetic 5,001-surface model, cold | 2.21 / 1.86 | 2.64 / 2.27 | 339.9 / 338.8 | | synthetic 5,001-surface model, warm | 1.98 / 1.61 | 2.39 / 2.04 | 341.8 / 332.8 | Every successful result is identical. Full response size stays **194 cl100k_base tokens / 442 bytes** for actual observations and **123 tokens / 310 bytes** for the capacity model. The corrected actual weak response is 260 tokens versus 193 previously (+67 necessary tokens for the reason/probe). The capacity workload is explicitly **synthetic metadata: 5,001 declared surfaces/owners and one claim**. It measures evaluation scale and establishes no real-provider/reconstruction coverage. The real small task's warm timing is effectively unchanged; the model shows lower work after removing repeated scans. Shared host, two samples, uncontrolled OS caches and 200 ms RSS sampling limit conclusions; no general speedup guarantee. Cold disables Node compile caching; warm primes per-version caches. Wall uses child completion, CPU covers reaped children, and sampled RSS may double-count shared pages/miss short processes. These are retained-response proxy token counts: no model consumed the payloads; complete agent-task tokens, input/schema negotiation and billing remain unmeasured. Heavy local tasks ran serially on two CPU cores, at most two test workers, with Node heap defaults bounded and a task-local memory guard within the user-approved 8 GiB overall budget. Measured validation family peak was about **4.55 GiB**; no memory guard triggered. ## Compatibility and release Call/configuration/schema shapes remain unchanged. Corrections withhold readiness for weak or unrelated proof, and retain unknown/failure provenance. Complete Evidence stays in the caller's authenticated bundle. Existing ownership/freshness/contradiction rules remain in force; no package/version change. ## Author readiness Human owner: @N0zoM1z0 The human owner explicitly authorized delegated review, proxy completion of these declarations, ready-for-review publication and normal merge for this batch. The agent reviewed the final diff and evidence; this records that delegation and does not assert a separate manual review occurred. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:11:22 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:46:28 +00:00

[PR #6271] [MERGED] fix(il2cpp): refuse a seed path under a non-directory as caller input #7245

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#6039

No description provided.

Rows
Columns