[GH-ISSUE #48] Epic: Evidence v2 and deterministic process fidelity #35

Open
opened 2026-10-15 18:20:16 +00:00 by bsakaguchi · 4 comments
Owner

Originally created by @morluto on GitHub (Jul 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/48

Outcome

Provide trustworthy process observations and comparisons alongside Evidence v2 and real Hopper function-analysis proof. Agents compose experiments with commands and scripts; REA captures the requested runtime activity and returns evidence.

This epic remains open for event-complete process observation and the broader Hopper conformance claims. It no longer requires a custom reactive experiment framework or built-in replay engine.

Scope after #555 and #572

#555 removed permission grants, filesystem scope ceilings, and repeated approval fields. #572 removed loopback replay servers, command shims, reactive state machines, custom checkpoint triggers, and the controlled JavaScript replay/characterization flows.

Direct process capture retains caller-selected commands, PATH resolution, working directories, environment overrides, timed input/resize/signal events, terminal observations, before/after filesystem snapshots, process sampling, settlement, cancellation, and owned-resource cleanup. Network activity is not observed by this capture, and process sampling must not claim event-complete lifecycle coverage.

Agents can run fixture servers and repeat authority/candidate commands themselves, then use comparison tools on the resulting evidence. Needing an agent-authored script is not an incomplete REA capability. Generic conformance and evidence-integrity checks remain useful without an executable replay subsystem.

Earlier comments requiring the #60 → reactive #57 → paired experiment → replay export pipeline describe the retired direction. Historical closure of those implementation issues is not a current replay capability claim.

Evidence requirements

Results distinguish observation, inference, and unknowns. Missing, truncated, unsupported, or unobserved activity cannot be represented as equivalence or an empty fact. Explicitly sensitive inputs and transport credentials are redacted; ordinary selected local inputs and diagnostics are preserved.

Use actual OS permissions and truthful provider prerequisites. Validate target identity and input/protocol constraints, and prove cancellation and cleanup. Portable scenario parsing alone does not establish Windows process ownership or executable support.

Remaining child work

Hopper dossiers and semantic proof remain under #54/#55, with broader trackers #31/#25 and focused real-provider issues #339–#344. Complete function bodies, references, calls, comments, and CFG coverage must be reported from actual provider evidence; unavailable coverage cannot satisfy a positive conformance claim.

Owned process trees and filesystem effects remain under #59. Event-complete process lifecycle integration is tracked by #333. Missing PTY modules, ABI mismatches, and actual probe failures need actionable diagnostics under #568.

The Evidence v2 foundations in #49–#53 and Hopper ownership work in #56 remain historical delivered work. Browser/Electron capabilities are tracked separately by #39/#64; environment evidence remains under #69/#382. None requires restoring REA grants or replay orchestration.

Exit gate

Public CLI and MCP behavior must agree. Process tests must prove actual capture, failure classification, cancellation, settlement, and owned-resource cleanup. Comparisons must retain unknown coverage rather than claiming determinism or equality from missing observations.

Run the applicable current gates: npm run check:pr, npm run verify:package, source-built fixtures, and the matching real-Hopper/provider verification lanes. The broader semantic and event-completeness claims require their own evidence before the remaining child issues can close.

Originally created by @morluto on GitHub (Jul 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/48 ## Outcome Provide trustworthy process observations and comparisons alongside Evidence v2 and real Hopper function-analysis proof. Agents compose experiments with commands and scripts; REA captures the requested runtime activity and returns evidence. This epic remains open for event-complete process observation and the broader Hopper conformance claims. It no longer requires a custom reactive experiment framework or built-in replay engine. ## Scope after #555 and #572 #555 removed permission grants, filesystem scope ceilings, and repeated approval fields. #572 removed loopback replay servers, command shims, reactive state machines, custom checkpoint triggers, and the controlled JavaScript replay/characterization flows. Direct process capture retains caller-selected commands, PATH resolution, working directories, environment overrides, timed input/resize/signal events, terminal observations, before/after filesystem snapshots, process sampling, settlement, cancellation, and owned-resource cleanup. Network activity is not observed by this capture, and process sampling must not claim event-complete lifecycle coverage. Agents can run fixture servers and repeat authority/candidate commands themselves, then use comparison tools on the resulting evidence. Needing an agent-authored script is not an incomplete REA capability. Generic conformance and evidence-integrity checks remain useful without an executable replay subsystem. Earlier comments requiring the #60 → reactive #57 → paired experiment → replay export pipeline describe the retired direction. Historical closure of those implementation issues is not a current replay capability claim. ## Evidence requirements Results distinguish observation, inference, and unknowns. Missing, truncated, unsupported, or unobserved activity cannot be represented as equivalence or an empty fact. Explicitly sensitive inputs and transport credentials are redacted; ordinary selected local inputs and diagnostics are preserved. Use actual OS permissions and truthful provider prerequisites. Validate target identity and input/protocol constraints, and prove cancellation and cleanup. Portable scenario parsing alone does not establish Windows process ownership or executable support. ## Remaining child work Hopper dossiers and semantic proof remain under #54/#55, with broader trackers #31/#25 and focused real-provider issues #339–#344. Complete function bodies, references, calls, comments, and CFG coverage must be reported from actual provider evidence; unavailable coverage cannot satisfy a positive conformance claim. Owned process trees and filesystem effects remain under #59. Event-complete process lifecycle integration is tracked by #333. Missing PTY modules, ABI mismatches, and actual probe failures need actionable diagnostics under #568. The Evidence v2 foundations in #49–#53 and Hopper ownership work in #56 remain historical delivered work. Browser/Electron capabilities are tracked separately by #39/#64; environment evidence remains under #69/#382. None requires restoring REA grants or replay orchestration. ## Exit gate Public CLI and MCP behavior must agree. Process tests must prove actual capture, failure classification, cancellation, settlement, and owned-resource cleanup. Comparisons must retain unknown coverage rather than claiming determinism or equality from missing observations. Run the applicable current gates: `npm run check:pr`, `npm run verify:package`, source-built fixtures, and the matching real-Hopper/provider verification lanes. The broader semantic and event-completeness claims require their own evidence before the remaining child issues can close.
Author
Owner

@morluto commented on GitHub (Jul 17, 2026):

Most original Evidence v2 and low-level process-capture foundations are delivered. The remaining epic exit gate is the expanded reactive paired experiment: stateful replay, repeatability classification, causal first divergence, and portable generated conformance packages. Current dependency order is #60, expanded #57, #61, then #62.

<!-- gh-comment-id:5001703103 --> @morluto commented on GitHub (Jul 17, 2026): Most original Evidence v2 and low-level process-capture foundations are delivered. The remaining epic exit gate is the expanded reactive paired experiment: stateful replay, repeatability classification, causal first divergence, and portable generated conformance packages. Current dependency order is #60, expanded #57, #61, then #62.
Author
Owner

@morluto commented on GitHub (Jul 17, 2026):

Current-main audit: the Evidence v2 and base Process Capture v4 stack are delivered, but the epic is not complete. The remaining exit path is the reactive paired-experiment pipeline: #60 finite-state replay, expanded #57 reactive trigger algebra, #61 paired repeatability/instability experiments, #62 portable conformance export, plus completion of real-Hopper conformance. Keep this epic open as the dependency/exit-gate tracker.

<!-- gh-comment-id:5002759152 --> @morluto commented on GitHub (Jul 17, 2026): Current-main audit: the Evidence v2 and base Process Capture v4 stack are delivered, but the epic is not complete. The remaining exit path is the reactive paired-experiment pipeline: #60 finite-state replay, expanded #57 reactive trigger algebra, #61 paired repeatability/instability experiments, #62 portable conformance export, plus completion of real-Hopper conformance. Keep this epic open as the dependency/exit-gate tracker.
Author
Owner

@morluto commented on GitHub (Jul 20, 2026):

The Evidence v2 and deterministic process-fidelity work is now split into focused children:

Process capture and replay

  • #57 Define reusable deterministic process-capture scenario contracts
    • #330 Add reactive scenario algebra for process capture
    • #331 Build reactive process-capture end-to-end fixture
  • #59 Capture owned process trees and filesystem effects
    • #332 Capture event-complete process-tree lifecycle
    • #333 Integrate event-backed process-tree capture into Process Capture v4
  • #60 Add deterministic loopback HTTP and WebSocket replay
    • #334 Integrate finite-state replay machine into process capture
    • #335 Add CLI/MCP command for finite-state replay machines
  • #62 Build process-fidelity conformance fixtures
    • #336 Design portable process-fidelity conformance export package
    • #337 Add semantic-diff trust gates for conformance fixtures
    • #338 Run conformance fixtures in clean-checkout CI replay

Hopper conformance

  • #25 Build a multi-binary Hopper conformance and fidelity suite
    • #339 Record real-Hopper semantic proof for source-built fixtures
    • #340 Verify Hopper cancellation, timeout, mutation readback, and shutdown
  • #31 Complete truthful analyze_function dossiers
    • #341 Run real-Hopper analyze_function dossier conformance
  • #54 Complete Hopper function dossiers with real references and CFG edges
    • #342 Real-Hopper verifier for caller/callee chains
    • #343 Real-Hopper verifier for CFG successors and assembly output
  • #55 Strengthen semantic Hopper conformance across source-built fixtures
    • #344 Run real-Hopper two-binary semantic conformance workflow

This epic can close when the process-capture and Hopper children are closed.

<!-- gh-comment-id:5023452525 --> @morluto commented on GitHub (Jul 20, 2026): The Evidence v2 and deterministic process-fidelity work is now split into focused children: Process capture and replay - [ ] #57 Define reusable deterministic process-capture scenario contracts - [ ] #330 Add reactive scenario algebra for process capture - [ ] #331 Build reactive process-capture end-to-end fixture - [ ] #59 Capture owned process trees and filesystem effects - [ ] #332 Capture event-complete process-tree lifecycle - [ ] #333 Integrate event-backed process-tree capture into Process Capture v4 - [ ] #60 Add deterministic loopback HTTP and WebSocket replay - [ ] #334 Integrate finite-state replay machine into process capture - [ ] #335 Add CLI/MCP command for finite-state replay machines - [ ] #62 Build process-fidelity conformance fixtures - [ ] #336 Design portable process-fidelity conformance export package - [ ] #337 Add semantic-diff trust gates for conformance fixtures - [ ] #338 Run conformance fixtures in clean-checkout CI replay Hopper conformance - [ ] #25 Build a multi-binary Hopper conformance and fidelity suite - [ ] #339 Record real-Hopper semantic proof for source-built fixtures - [ ] #340 Verify Hopper cancellation, timeout, mutation readback, and shutdown - [ ] #31 Complete truthful analyze_function dossiers - [ ] #341 Run real-Hopper analyze_function dossier conformance - [ ] #54 Complete Hopper function dossiers with real references and CFG edges - [ ] #342 Real-Hopper verifier for caller/callee chains - [ ] #343 Real-Hopper verifier for CFG successors and assembly output - [ ] #55 Strengthen semantic Hopper conformance across source-built fixtures - [ ] #344 Run real-Hopper two-binary semantic conformance workflow This epic can close when the process-capture and Hopper children are closed.
Author
Owner

@morluto commented on GitHub (Jul 22, 2026):

End-to-end follow-through is tracked in #406. It uses the process-fidelity work here as one stage in a broader reconstruction journey and verifies integration with capability discovery, grants, JavaScript/Electron workflows, partial-order comparison, contradiction preservation, and coverage closure.

<!-- gh-comment-id:5041149559 --> @morluto commented on GitHub (Jul 22, 2026): End-to-end follow-through is tracked in #406. It uses the process-fidelity work here as one stage in a broader reconstruction journey and verifies integration with capability discovery, grants, JavaScript/Electron workflows, partial-order comparison, contradiction preservation, and coverage closure.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#35
No description provided.