morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5945] [Bug] CLI snapshot target mismatch uses generic Evidence recovery text #3615

New issue

Closed

opened 2026-10-31 00:42:24 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:42:24 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5945

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

CLI

Observed behavior

rea analyze B --snapshot A.json correctly refuses a valid snapshot saved for different target bytes, but returns the generic Evidence-integrity sentence in both message and remediation.action:

“Evidence is invalid or has changed. Recreate or re-import it, then try again.”

The precise explanation survives in details.reason: “Analysis snapshot target does not match the requested binary”. The defect concerns the main message and recovery guidance, rather than acceptance of a mismatched snapshot.

This was reproduced through the built CLI using a snapshot created by a successful inspect-native-dispatch-metadata call. Replaying the snapshot against its original target succeeds.

Reproduction

From the built source checkout with its locked dependencies installed, save the following as reproduce.py, then run REA_REPRO_DIR=./snapshot-target-repro python3 reproduce.py.

The script creates two minimal 32-byte Mach-O arm64 files, differing only in the MH_PIE flag. It explicitly disables deep-provider installations and uses REA's actual native metadata reader to create the snapshot. It neither executes the targets nor supplies simulated provider results.

import hashlib
import json
import os
from pathlib import Path
import struct
import subprocess

root = Path(os.environ['REA_REPRO_DIR']).resolve()
root.mkdir(parents=True, exist_ok=True)
for name, flags in [('a.macho', 0), ('b.macho', 0x200000)]:
    (root / name).write_bytes(
        struct.pack('<8I', 0xfeedfacf, 0x100000c, 0, 2, 0, 0, flags, 0)
    )
env = os.environ.copy()
env.update(
    REA_ANALYSIS_PROVIDER='auto',
    HOPPER_LAUNCHER_PATH='/rea-unconfigured-deep-provider/hopper',
    GHIDRA_INSTALL_DIR='/rea-unconfigured-deep-provider/ghidra',
    JAVA_HOME='/rea-unconfigured-deep-provider/java',
    REA_IDA_MCP_CONFIG='/rea-unconfigured-deep-provider/ida-mcp.json',
    REA_KUNA_COMMAND='/rea-unconfigured-deep-provider/kuna',
    REA_KUNA_CACHE_DIR='/rea-unconfigured-deep-provider/kuna-cache',
)
snapshot = root / 'analysis.json'
def run(command, target):
    result = subprocess.run(
        ['node', 'scripts/rea.mjs', command, str(root / target),
         '--snapshot', str(snapshot), '--format', 'json'],
        env=env, text=True, capture_output=True, check=False,
    )
    assert result.stderr == '', result.stderr
    return result.returncode, json.loads(result.stdout)

created_code, created = run('inspect-native-dispatch-metadata', 'a.macho')
replay_code, replay = run('inspect-native-dispatch-metadata', 'a.macho')
assert created_code == replay_code == 0
assert created['evidence_id'] == replay['evidence_id']
before = hashlib.sha256(snapshot.read_bytes()).hexdigest()
code, failure = run('analyze', 'b.macho')
assert code == 1
assert before == hashlib.sha256(snapshot.read_bytes()).hexdigest()
assert failure['details']['reason'] == 'Analysis snapshot target does not match the requested binary'
assert failure['message'] == failure['remediation']['action'] == 'Evidence is invalid or has changed. Recreate or re-import it, then try again.'
print('Creation and same-target replay: exit 0, identical Evidence ID')
print('Different-target analyze: exit 1; snapshot unchanged')
print(json.dumps(failure, indent=2))

The final script completed once with all assertions passing. Earlier direct CLI runs independently returned the same mismatch output.

Expected behavior

Keep rejecting the snapshot, while exposing its target mismatch in message and selecting the existing snapshot recovery advice:

“Run analysis without this snapshot, then save a fresh snapshot using the intended artifact, provider, and analysis profile.”

The dedicated domain error explicitly covers a valid snapshot belonging to another target. Its message selection preserves the mismatch reason, and its recovery selection supplies the advice above. The snapshot documentation describes exact target-bound replay.

Impact and scope

P3: the observed defect is diagnostic and recovery wording. Target isolation still works, the snapshot remains unchanged, and the detailed reason remains available to callers. No invalid replay or data corruption was observed.

Observed for CLI analyze with different target bytes. Other commands that reach the shared prepareSnapshot path have the same source-derived exposure; they were not individually executed. Callers can inspect details.reason and select the matching target or omit the unrelated snapshot.

REA version

6.7.0, source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0; Python 3 for fixture generation and CLI invocation. Actual built CLI and native metadata reader; deep providers disabled by explicit configuration.

Target and artifact context

Two authored minimal thin Mach-O arm64 executable headers, each 32 bytes long, with zero load commands. A has flags 0; B has flags 0x200000 (MH_PIE). The CLI-created snapshot for A contains one native-dispatch workflow entry and its Evidence record. Same-target replay succeeds before the different-target refusal.

Interface

CLI

Earliest failure stage

Setup or provider admission

Relevant output or evidence

Output from the executed script:

Creation and same-target replay: exit 0, identical Evidence ID
Different-target analyze: exit 1; snapshot unchanged
{
  "error": "Analysis failed",
  "code": "evidence_integrity_mismatch",
  "category": "integrity_mismatch",
  "message": "Evidence is invalid or has changed. Recreate or re-import it, then try again.",
  "retryable": false,
  "remediation": {
    "action": "Evidence is invalid or has changed. Recreate or re-import it, then try again."
  },
  "details": {
    "reason": "Analysis snapshot target does not match the requested binary"
  }
}

The production snapshot creation and same-target replay establish that the snapshot is usable. The byte comparison establishes that the failed invocation does not modify it.

Typed outcome or recovery expected

Retain code: "evidence_integrity_mismatch", category: "integrity_mismatch", and retryable: false. Use the target-specific message and snapshot recovery action already supplied for AnalysisSnapshotMismatchError.

Analysis or suspected cause

Source-derived: direct-analysis admission validates the snapshot and selected target, then constructs the base EvidenceIntegrityError when their identities differ. The caller projects that failure before provider preview.

The base instance does not match the AnalysisSnapshotMismatchError checks. Generic message selection replaces the main message, and the remediation fallback repeats it. Using the dedicated mismatch class at the producing boundary would preserve the existing typed code while selecting the intended diagnostic and recovery text.

Additional context

The relevant direct-analysis, snapshot-contract, error-class, and presentation sources are unchanged on fetched upstream main revision 705058f3e2df82503f1c601fc464aea7cbb626b8. Execution used the baseline above. No regression range was established.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5945 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CLI ### Observed behavior `rea analyze B --snapshot A.json` correctly refuses a valid snapshot saved for different target bytes, but returns the generic Evidence-integrity sentence in both `message` and `remediation.action`: “Evidence is invalid or has changed. Recreate or re-import it, then try again.” The precise explanation survives in `details.reason`: “Analysis snapshot target does not match the requested binary”. The defect concerns the main message and recovery guidance, rather than acceptance of a mismatched snapshot. This was reproduced through the built CLI using a snapshot created by a successful `inspect-native-dispatch-metadata` call. Replaying the snapshot against its original target succeeds. ### Reproduction From the built source checkout with its locked dependencies installed, save the following as `reproduce.py`, then run `REA_REPRO_DIR=./snapshot-target-repro python3 reproduce.py`. The script creates two minimal 32-byte Mach-O arm64 files, differing only in the `MH_PIE` flag. It explicitly disables deep-provider installations and uses REA's actual native metadata reader to create the snapshot. It neither executes the targets nor supplies simulated provider results. ```python import hashlib import json import os from pathlib import Path import struct import subprocess root = Path(os.environ['REA_REPRO_DIR']).resolve() root.mkdir(parents=True, exist_ok=True) for name, flags in [('a.macho', 0), ('b.macho', 0x200000)]: (root / name).write_bytes( struct.pack('<8I', 0xfeedfacf, 0x100000c, 0, 2, 0, 0, flags, 0) ) env = os.environ.copy() env.update( REA_ANALYSIS_PROVIDER='auto', HOPPER_LAUNCHER_PATH='/rea-unconfigured-deep-provider/hopper', GHIDRA_INSTALL_DIR='/rea-unconfigured-deep-provider/ghidra', JAVA_HOME='/rea-unconfigured-deep-provider/java', REA_IDA_MCP_CONFIG='/rea-unconfigured-deep-provider/ida-mcp.json', REA_KUNA_COMMAND='/rea-unconfigured-deep-provider/kuna', REA_KUNA_CACHE_DIR='/rea-unconfigured-deep-provider/kuna-cache', ) snapshot = root / 'analysis.json' def run(command, target): result = subprocess.run( ['node', 'scripts/rea.mjs', command, str(root / target), '--snapshot', str(snapshot), '--format', 'json'], env=env, text=True, capture_output=True, check=False, ) assert result.stderr == '', result.stderr return result.returncode, json.loads(result.stdout) created_code, created = run('inspect-native-dispatch-metadata', 'a.macho') replay_code, replay = run('inspect-native-dispatch-metadata', 'a.macho') assert created_code == replay_code == 0 assert created['evidence_id'] == replay['evidence_id'] before = hashlib.sha256(snapshot.read_bytes()).hexdigest() code, failure = run('analyze', 'b.macho') assert code == 1 assert before == hashlib.sha256(snapshot.read_bytes()).hexdigest() assert failure['details']['reason'] == 'Analysis snapshot target does not match the requested binary' assert failure['message'] == failure['remediation']['action'] == 'Evidence is invalid or has changed. Recreate or re-import it, then try again.' print('Creation and same-target replay: exit 0, identical Evidence ID') print('Different-target analyze: exit 1; snapshot unchanged') print(json.dumps(failure, indent=2)) ``` The final script completed once with all assertions passing. Earlier direct CLI runs independently returned the same mismatch output. ### Expected behavior Keep rejecting the snapshot, while exposing its target mismatch in `message` and selecting the existing snapshot recovery advice: “Run analysis without this snapshot, then save a fresh snapshot using the intended artifact, provider, and analysis profile.” The [dedicated domain error](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/evidenceErrors.ts#L27-L28) explicitly covers a valid snapshot belonging to another target. Its [message selection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L245-L247) preserves the mismatch reason, and its [recovery selection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L60-L61) supplies the advice above. The [snapshot documentation](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/cli.md#L227-L265) describes exact target-bound replay. ### Impact and scope P3: the observed defect is diagnostic and recovery wording. Target isolation still works, the snapshot remains unchanged, and the detailed reason remains available to callers. No invalid replay or data corruption was observed. Observed for CLI `analyze` with different target bytes. Other commands that reach the shared `prepareSnapshot` path have the same source-derived exposure; they were not individually executed. Callers can inspect `details.reason` and select the matching target or omit the unrelated snapshot. ### REA version `6.7.0`, source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0; Python 3 for fixture generation and CLI invocation. Actual built CLI and native metadata reader; deep providers disabled by explicit configuration. ### Target and artifact context Two authored minimal thin Mach-O arm64 executable headers, each 32 bytes long, with zero load commands. A has flags `0`; B has flags `0x200000` (`MH_PIE`). The CLI-created snapshot for A contains one native-dispatch workflow entry and its Evidence record. Same-target replay succeeds before the different-target refusal. ### Interface CLI ### Earliest failure stage Setup or provider admission ### Relevant output or evidence Output from the executed script: ```text Creation and same-target replay: exit 0, identical Evidence ID Different-target analyze: exit 1; snapshot unchanged { "error": "Analysis failed", "code": "evidence_integrity_mismatch", "category": "integrity_mismatch", "message": "Evidence is invalid or has changed. Recreate or re-import it, then try again.", "retryable": false, "remediation": { "action": "Evidence is invalid or has changed. Recreate or re-import it, then try again." }, "details": { "reason": "Analysis snapshot target does not match the requested binary" } } ``` The production snapshot creation and same-target replay establish that the snapshot is usable. The byte comparison establishes that the failed invocation does not modify it. ### Typed outcome or recovery expected Retain `code: "evidence_integrity_mismatch"`, `category: "integrity_mismatch"`, and `retryable: false`. Use the target-specific message and snapshot recovery action already supplied for `AnalysisSnapshotMismatchError`. ### Analysis or suspected cause Source-derived: [direct-analysis admission](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/DirectAnalysis.ts#L565-L603) validates the snapshot and selected target, then constructs the base `EvidenceIntegrityError` when their identities differ. [The caller](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/DirectAnalysis.ts#L275-L296) projects that failure before provider preview. The base instance does not match the `AnalysisSnapshotMismatchError` checks. [Generic message selection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L269-L274) replaces the main message, and [the remediation fallback](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisErrorPresentation.ts#L135-L142) repeats it. Using the dedicated mismatch class at the producing boundary would preserve the existing typed code while selecting the intended diagnostic and recovery text. ### Additional context The relevant direct-analysis, snapshot-contract, error-class, and presentation sources are unchanged on fetched upstream `main` revision `705058f3e2df82503f1c601fc464aea7cbb626b8`. Execution used the baseline above. No regression range was established.

bsakaguchi closed this issue

2026-10-31 00:42:24 +00:00

bsakaguchi commented

2026-10-31 00:42:24 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

DirectAnalysis now constructs AnalysisSnapshotMismatchError at target mismatch, selecting the existing snapshot-specific presentation and recovery. The composition regression builds a valid snapshot and compares replay against different target bytes, asserting evidence_integrity_mismatch and the exact fresh-snapshot action. Refusal remains in place.

Implementation: DirectAnalysis.ts, directAnalysisFactories.test.ts.

The relevant regression checks in directAnalysisFactories.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. DirectAnalysis now constructs AnalysisSnapshotMismatchError at target mismatch, selecting the existing snapshot-specific presentation and recovery. The composition regression builds a valid snapshot and compares replay against different target bytes, asserting evidence_integrity_mismatch and the exact fresh-snapshot action. Refusal remains in place. Implementation: [DirectAnalysis.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/DirectAnalysis.ts#L707), [directAnalysisFactories.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/composition/directAnalysisFactories.test.ts#L285). The relevant regression checks in [directAnalysisFactories.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/composition/directAnalysisFactories.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 00:48:27 +00:00

[GH-ISSUE #7129] [Feature] Static readers for firmware containers: Intel HEX/S-record regions, U-Boot uImage headers, and FIT images #4280

bsakaguchi referenced this issue

2026-10-31 00:52:01 +00:00

[GH-ISSUE #7955] [Roadmap] Embedded firmware: make REA work on the devices around us #4654

bsakaguchi referenced this issue

2026-11-01 21:10:54 +00:00

[GH-ISSUE #3641] [Feature] USB protocol reverse engineering: descriptors, HID report schemas, and recorded transaction reconstruction #1363

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471