[PR #6141] [CLOSED] fix(snapshot): keep mismatch, output-loss, and declaration require diagnostics precise #7157

Closed
opened 2026-11-01 21:45:48 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6141
Author: @agorevski
Created: 10/21/2026
Status: ❌ Closed

Base: main ← Head: fix/misc-snapshot-output-ts


📝 Commits (2)

  • ba46733 fix(snapshot): keep target mismatch, output loss, and declaration require exact
  • dead84a fix(snapshot): separate omitted-output and completion limitations

📊 Changes

6 files changed (+113 additions, -3 deletions)

View changed files

📝 src/application/DirectAnalysis.ts (+2 -2)
📝 src/application/javascript/JavaScriptArtifactPathResolution.test.ts (+54 -0)
📝 src/application/javascript/JavaScriptArtifactPathResolution.ts (+6 -0)
📝 src/application/reverse/ReverseEngineeringService.ts (+6 -1)
📝 tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts (+33 -0)
📝 tests/support/ownedRizinCommandEvidenceTests.ts (+12 -0)

📄 Description

Summary

Fixes #5945
Fixes #5967
Fixes #6105

Three independent evidence-correctness defects, each fixed at its producing boundary:

  • #5945 Replaying a snapshot saved for different target bytes returned the generic "Evidence is invalid or has changed" message and remediation. The producer constructed the base EvidenceIntegrityError, so the mismatch-specific presentation never applied. It now constructs AnalysisSnapshotMismatchError, which keeps the same evidence_integrity_mismatch code and retryable=false while selecting the target-mismatch message and snapshot recovery advice.
  • #5967 The objdump/Rizin owned-capture success path (exit 0) dropped the diagnosticTruncated marker, so Evidence reported output_truncated: false for known omitted output. The owned runner now returns the marker on success, and the service assigns it, matching the existing non-zero-exit path.
  • #6105 In require mode, an explicitly written require("./dep.d.ts") was downgraded to unavailable by the generic declaration check, while import "./dep.d.ts" resolved. Only the exact requested path is now exempt in module-specifier resolution. The emitted-extension fallback (require("./only.js") finding only.d.ts) stays unavailable.

Problem and expected behavior

  • #5945: smallest trigger is two same-size thin Mach-O headers differing in one flag, a snapshot saved for A, then analyze B --snapshot A.json. Expected: refuse the snapshot (unchanged) with a target-specific message and the existing snapshot recovery action. Observed: generic integrity text in message and remediation.action; details.reason was already correct.
  • #5967: a process exiting 0 with outputIncomplete produced a snapshot with diagnosticTruncated: true, but Evidence had output_truncated: false. Expected: the same partial-output disclosure as the non-zero exit path.
  • #6105: require("./dep.d.ts") with the file present returned unavailable with candidate_path: "dep.d.ts" instead of the resolved inventoried declaration. Expected: explicit exact declaration resolves in both loader modes; emitted-extension declaration fallback stays unavailable.

Change and scope

  • DirectAnalysis.ts: prepareSnapshot constructs AnalysisSnapshotMismatchError (existing subclass of EvidenceIntegrityError); the now-unused base import is removed.
  • ReverseEngineeringService.ts: the owned-command runner returns outputTruncated from snapshot.diagnosticTruncated on success; the service assigns it instead of leaving false.
  • JavaScriptArtifactPathResolution.ts: resolveFileCandidates takes an optional explicit path. resolveCandidate passes the declared candidate only for module-specifier context, so that exact path is not downgraded. Directory/package-main and fallback candidates are unchanged.

Not included: no change to exit status, completion-status semantics, provider output capture, snapshot file format, or package-main declaration handling (still downgraded). No lifecycle redesign.

Contract and boundary impact

  • Semantic owner and earliest changed stage: the snapshot-admission producer (#5945), the owned-command result projection (#5967), and JavaScript artifact path resolution (#6105). Each change is at that owner; no downstream repair.
  • Contribution scope fit and linked maintainer decision: not applicable (bug fixes inside existing boundaries; no new provider or capability).
  • CLI and MCP/tool-catalog contract: no schema change. Error code/category/retryable are unchanged for #5945; message and remediation text change to the existing mismatch wording. #5967 changes output_truncated from false to true only for exit-0 captures with omitted output. #6105 changes resolution_status from unavailable to resolved only for explicit exact declaration requests.
  • Provider, bridge, target-format, or platform compatibility: none changed. The #5967 reproduction used simulated provider streams; no real objdump or Rizin ran.
  • Evidence, artifact, provenance, or reconstruction contract: #6105 changes a static relationship's resolution for explicit declaration requests; the existing declaration limitation text still applies to consumers that call it compile-time evidence. Unknown or unavailable outcomes for fallback candidates are preserved.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): none changed. Snapshot is still not modified on mismatch.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact (CI generates and validates ignored catalogs and skills/; edit authored instructions in .agents/skills/reverse-engineer-anything/ and never commit build outputs): none. Build outputs were produced by the focused test runner and not committed.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated:
    • src/application/javascript/JavaScriptArtifactPathResolution.test.ts: new describe "explicit TypeScript declaration require ownership" (./dep.d.ts, ./cfg.d.mts, ./legacy.d.cts resolve to their exact declaration in both loader modes). Existing emitted-fallback test unchanged.
    • tests/support/ownedRizinCommandEvidenceTests.ts: new row "normal exit with omitted output" (exit 0, diagnosticTruncated: true → output_truncated: true); runs through tests/composition/reverseEngineeringEdgeCases.test.ts.
    • tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts: new case "names the target mismatch when a snapshot is replayed against other bytes" (asserts code, message, and remediation action).
  • Base reproduction or other evidence: all five new cases fail on the base source and pass with the fix (see Validation). The issue reproductions (Python fixture for #5945, CLI for #6105, the simulated objdump/Rizin test for #5967) were not re-executed against the built CLI in this change.
  • User-visible CLI/MCP output (if applicable): #5945 expected output (from the new test's assertions) is code: "evidence_integrity_mismatch", message and remediation.action as in the issue's "Expected behavior"; the CLI/MCP output was not captured from a built binary.
  • Remaining proof gaps: no real Ghidra, IDA, Hopper, Frida, browser, or Windows objdump/Rizin execution; no darwin CLI run (the existing tests/acceptance/nativeDispatchSnapshot.test.ts is darwin-only and was not run); no built dist/ CLI run of the #6105 synthetic directory; the full suite (npm run test:run, check:pr) was not run.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Review follow-up at 7eb8b5002 (the bullets below describe the earlier commit):

  • npm run test:focused -- src/application/javascript/JavaScriptArtifactPathResolution.test.ts tests/composition/reverseEngineeringEdgeCases.test.ts tests/boundary/filesystem/ciChangeScope.test.ts — exit 0, 3 files, 635 tests passed. Covers the new bare require("dep.d.ts") case and the omitted-output limitation wording.
  • node scripts/run-typescript.mjs --noEmit — exit 0.
  • npx oxfmt on the changed files — formatted.
  • Not run: the full suite.

Run from the worktree of fix/misc-snapshot-output-ts (base origin/main 01788bac8):

  • npm run test:focused -- src/application/javascript/JavaScriptArtifactPathResolution.test.ts tests/composition/reverseEngineeringEdgeCases.test.ts tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts — exit 0; 3 files passed; 385 tests passed, 2 skipped (pre-existing platform-gated skips).
  • Fail-before: same command with the three source files reverted to base (test files unchanged) — exit 1; 5 failed, 380 passed, 2 skipped (the three #6105 cases, the #5967 row, and the #5945 case).
  • npm run test:local -- <same three paths> — exit 0 but ran no tests: the runner reports that built-project files need test:focused. Not counted as validation.
  • npm run typecheck — exit 0.
  • npx oxlint <six changed files> — exit 0, no errors. Warnings are complexity/size warnings in JavaScriptArtifactPathResolution.ts that also appear on the base file; none new.
  • npx oxfmt --check <six changed files> — exit 0.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Error codes and categories unchanged; #5945 message/remediation wording changes only for target-mismatch snapshot refusals.
  • Real Hopper/Ghidra, browser, or OS coverage: none run; see proof gaps above.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no new process, filesystem, or network behavior; snapshot is not written on mismatch (unchanged).

Author readiness

Human owner: @agorevski

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6141 **Author:** [@agorevski](https://github.com/agorevski) **Created:** 10/21/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/misc-snapshot-output-ts` --- ### 📝 Commits (2) - [`ba46733`](https://github.com/morluto/rea/commit/ba467330b5b00fda8b0a51a45114eae4111d9fc5) fix(snapshot): keep target mismatch, output loss, and declaration require exact - [`dead84a`](https://github.com/morluto/rea/commit/dead84a738e687ad6a03d39f7f6b73857a38a502) fix(snapshot): separate omitted-output and completion limitations ### 📊 Changes **6 files changed** (+113 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `src/application/DirectAnalysis.ts` (+2 -2) 📝 `src/application/javascript/JavaScriptArtifactPathResolution.test.ts` (+54 -0) 📝 `src/application/javascript/JavaScriptArtifactPathResolution.ts` (+6 -0) 📝 `src/application/reverse/ReverseEngineeringService.ts` (+6 -1) 📝 `tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts` (+33 -0) 📝 `tests/support/ownedRizinCommandEvidenceTests.ts` (+12 -0) </details> ### 📄 Description <!-- PR title: type(optional-scope): imperative outcome The title becomes the squash-merge commit subject and generated changelog entry. --> ## Summary Fixes #5945 Fixes #5967 Fixes #6105 Three independent evidence-correctness defects, each fixed at its producing boundary: - **#5945** Replaying a snapshot saved for different target bytes returned the generic "Evidence is invalid or has changed" message and remediation. The producer constructed the base `EvidenceIntegrityError`, so the mismatch-specific presentation never applied. It now constructs `AnalysisSnapshotMismatchError`, which keeps the same `evidence_integrity_mismatch` code and retryable=false while selecting the target-mismatch message and snapshot recovery advice. - **#5967** The objdump/Rizin owned-capture success path (exit 0) dropped the `diagnosticTruncated` marker, so Evidence reported `output_truncated: false` for known omitted output. The owned runner now returns the marker on success, and the service assigns it, matching the existing non-zero-exit path. - **#6105** In require mode, an explicitly written `require("./dep.d.ts")` was downgraded to `unavailable` by the generic declaration check, while `import "./dep.d.ts"` resolved. Only the exact requested path is now exempt in module-specifier resolution. The emitted-extension fallback (`require("./only.js")` finding `only.d.ts`) stays unavailable. ## Problem and expected behavior - #5945: smallest trigger is two same-size thin Mach-O headers differing in one flag, a snapshot saved for A, then `analyze B --snapshot A.json`. Expected: refuse the snapshot (unchanged) with a target-specific message and the existing snapshot recovery action. Observed: generic integrity text in `message` and `remediation.action`; `details.reason` was already correct. - #5967: a process exiting 0 with `outputIncomplete` produced a snapshot with `diagnosticTruncated: true`, but Evidence had `output_truncated: false`. Expected: the same partial-output disclosure as the non-zero exit path. - #6105: `require("./dep.d.ts")` with the file present returned `unavailable` with `candidate_path: "dep.d.ts"` instead of the resolved inventoried declaration. Expected: explicit exact declaration resolves in both loader modes; emitted-extension declaration fallback stays unavailable. ## Change and scope - `DirectAnalysis.ts`: `prepareSnapshot` constructs `AnalysisSnapshotMismatchError` (existing subclass of `EvidenceIntegrityError`); the now-unused base import is removed. - `ReverseEngineeringService.ts`: the owned-command runner returns `outputTruncated` from `snapshot.diagnosticTruncated` on success; the service assigns it instead of leaving `false`. - `JavaScriptArtifactPathResolution.ts`: `resolveFileCandidates` takes an optional explicit path. `resolveCandidate` passes the declared candidate only for `module-specifier` context, so that exact path is not downgraded. Directory/package-main and fallback candidates are unchanged. Not included: no change to exit status, completion-status semantics, provider output capture, snapshot file format, or package-main declaration handling (still downgraded). No lifecycle redesign. ## Contract and boundary impact - Semantic owner and earliest changed stage: the snapshot-admission producer (#5945), the owned-command result projection (#5967), and JavaScript artifact path resolution (#6105). Each change is at that owner; no downstream repair. - Contribution scope fit and linked maintainer decision: not applicable (bug fixes inside existing boundaries; no new provider or capability). - CLI and MCP/tool-catalog contract: no schema change. Error `code`/`category`/`retryable` are unchanged for #5945; message and remediation text change to the existing mismatch wording. #5967 changes `output_truncated` from false to true only for exit-0 captures with omitted output. #6105 changes `resolution_status` from `unavailable` to `resolved` only for explicit exact declaration requests. - Provider, bridge, target-format, or platform compatibility: none changed. The #5967 reproduction used simulated provider streams; no real objdump or Rizin ran. - Evidence, artifact, provenance, or reconstruction contract: #6105 changes a static relationship's resolution for explicit declaration requests; the existing declaration limitation text still applies to consumers that call it compile-time evidence. Unknown or unavailable outcomes for fallback candidates are preserved. - Target selection, authority, upstream behavior, and cleanup details (if applicable): none changed. Snapshot is still not modified on mismatch. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs): none. Build outputs were produced by the focused test runner and not committed. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [ ] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - `src/application/javascript/JavaScriptArtifactPathResolution.test.ts`: new describe "explicit TypeScript declaration require ownership" (`./dep.d.ts`, `./cfg.d.mts`, `./legacy.d.cts` resolve to their exact declaration in both loader modes). Existing emitted-fallback test unchanged. - `tests/support/ownedRizinCommandEvidenceTests.ts`: new row "normal exit with omitted output" (exit 0, `diagnosticTruncated: true` → `output_truncated: true`); runs through `tests/composition/reverseEngineeringEdgeCases.test.ts`. - `tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts`: new case "names the target mismatch when a snapshot is replayed against other bytes" (asserts code, message, and remediation action). - Base reproduction or other evidence: all five new cases fail on the base source and pass with the fix (see Validation). The issue reproductions (Python fixture for #5945, CLI for #6105, the simulated objdump/Rizin test for #5967) were not re-executed against the built CLI in this change. - User-visible CLI/MCP output (if applicable): #5945 expected output (from the new test's assertions) is `code: "evidence_integrity_mismatch"`, `message` and `remediation.action` as in the issue's "Expected behavior"; the CLI/MCP output was not captured from a built binary. - Remaining proof gaps: no real Ghidra, IDA, Hopper, Frida, browser, or Windows objdump/Rizin execution; no darwin CLI run (the existing `tests/acceptance/nativeDispatchSnapshot.test.ts` is darwin-only and was not run); no built `dist/` CLI run of the #6105 synthetic directory; the full suite (`npm run test:run`, `check:pr`) was not run. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Review follow-up at `7eb8b5002` (the bullets below describe the earlier commit): - `npm run test:focused -- src/application/javascript/JavaScriptArtifactPathResolution.test.ts tests/composition/reverseEngineeringEdgeCases.test.ts tests/boundary/filesystem/ciChangeScope.test.ts` — exit 0, 3 files, 635 tests passed. Covers the new bare `require("dep.d.ts")` case and the omitted-output limitation wording. - `node scripts/run-typescript.mjs --noEmit` — exit 0. - `npx oxfmt` on the changed files — formatted. - Not run: the full suite. Run from the worktree of `fix/misc-snapshot-output-ts` (base `origin/main` 01788bac8): - `npm run test:focused -- src/application/javascript/JavaScriptArtifactPathResolution.test.ts tests/composition/reverseEngineeringEdgeCases.test.ts tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts` — exit 0; 3 files passed; 385 tests passed, 2 skipped (pre-existing platform-gated skips). - Fail-before: same command with the three source files reverted to base (test files unchanged) — exit 1; 5 failed, 380 passed, 2 skipped (the three #6105 cases, the #5967 row, and the #5945 case). - `npm run test:local -- <same three paths>` — exit 0 but ran no tests: the runner reports that built-project files need `test:focused`. Not counted as validation. - `npm run typecheck` — exit 0. - `npx oxlint <six changed files>` — exit 0, no errors. Warnings are complexity/size warnings in `JavaScriptArtifactPathResolution.ts` that also appear on the base file; none new. - `npx oxfmt --check <six changed files>` — exit 0. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Error codes and categories unchanged; #5945 message/remediation wording changes only for target-mismatch snapshot refusals. - Real Hopper/Ghidra, browser, or OS coverage: none run; see proof gaps above. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no new process, filesystem, or network behavior; snapshot is not written on mismatch (unchanged). ## Author readiness <!-- The human owner completes these declarations before requesting review. The PR author is the owner by default. A different owner must explicitly accept ownership on this PR. AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA. Draft PRs may leave them unchecked. These declarations record initial readiness when opening the PR for review; later pushes do not require a new human sign-off. Keep the description and validation evidence current as the work changes. See CONTRIBUTING.md#pull-request-ownership-and-readiness. --> Human owner: @agorevski <!-- @GitHub-handle --> - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge <!-- Draft PRs may leave pending items unchecked. Resolve material scope and authority questions before integration. --> - [x] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-01 21:45:48 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7157
No description provided.