morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #62] Build process-fidelity conformance fixtures #49

New issue

Closed

opened 2026-10-15 18:20:30 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-15 18:20:30 +00:00

Owner

Originally created by @morluto on GitHub (Jul 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/62

Problem

The process harness needs source-owned behavioral oracles that exercise interaction and cleanup rather than only happy-path output.

Outcome

Add portable source fixtures and an integration matrix for deterministic process capture and comparison.

Fixture behaviors

interactive prompts and terminal modes
Unicode and terminal resizing
signal handling and supervisor re-exec
child/grandchild workers
temporary credentials represented only by redacted fixtures
filesystem create/modify/delete/symlink behavior
HTTP callbacks and ordered WebSocket reconnects
timeouts, hangs, crashes, partial writes, and cleanup
normalized clock/PID/port/path differences
intentional behavioral differences for comparison

Acceptance criteria

Every positive assertion has a source-owned oracle. Tests cover approval denial, bounds, malformed scenarios, cancellation, redaction, loopback policy, descendant cleanup, deterministic replay, and unknown classification. No committed secrets or generated runtime artifacts.

Dependencies

Parent: #48. Requires all process-harness child capabilities.

Non-goals

External accounts, platform GUI behavior, or privileged macOS state.

Root capability expansion: capture-to-conformance generation

A successful investigation must be exportable as a reviewable checked-in verifier, not remain only a REA session record.

Fixture package

Generate a portable package containing canonical scenario, policy prerequisites, authority/candidate bindings, compatibility ranges, replay/shim plans, normalization with per-rule rationale, expected predicates or finite specification, Evidence v2 bundle digests, secret aliases, deterministic runner, and machine/human failure reports.

Trust and updates

Generated fixtures cite source evidence and are labeled derived.
Regeneration produces a semantic diff.
Snapshot acceptance is never automatic.
Dropping dimensions, weakening predicates, or expanding normalization requires review.
Stale evidence, incompatible schemas, missing providers, and policy denial fail closed.
Paths become stable bindings instead of deleted identity.
CI requires no production credentials or external services.

Acceptance

Capture a real interactive authority/candidate pair, export the package, run it from a clean checkout with different temp roots and ports, detect an injected behavior change, and prove that weakening normalization or expected dimensions changes the fixture digest and requires approval.

Originally created by @morluto on GitHub (Jul 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/62 ## Problem The process harness needs source-owned behavioral oracles that exercise interaction and cleanup rather than only happy-path output. ## Outcome Add portable source fixtures and an integration matrix for deterministic process capture and comparison. ## Fixture behaviors - interactive prompts and terminal modes - Unicode and terminal resizing - signal handling and supervisor re-exec - child/grandchild workers - temporary credentials represented only by redacted fixtures - filesystem create/modify/delete/symlink behavior - HTTP callbacks and ordered WebSocket reconnects - timeouts, hangs, crashes, partial writes, and cleanup - normalized clock/PID/port/path differences - intentional behavioral differences for comparison ## Acceptance criteria Every positive assertion has a source-owned oracle. Tests cover approval denial, bounds, malformed scenarios, cancellation, redaction, loopback policy, descendant cleanup, deterministic replay, and unknown classification. No committed secrets or generated runtime artifacts. ## Dependencies Parent: #48. Requires all process-harness child capabilities. ## Non-goals External accounts, platform GUI behavior, or privileged macOS state. ## Root capability expansion: capture-to-conformance generation A successful investigation must be exportable as a reviewable checked-in verifier, not remain only a REA session record. ### Fixture package Generate a portable package containing canonical scenario, policy prerequisites, authority/candidate bindings, compatibility ranges, replay/shim plans, normalization with per-rule rationale, expected predicates or finite specification, Evidence v2 bundle digests, secret aliases, deterministic runner, and machine/human failure reports. ### Trust and updates - Generated fixtures cite source evidence and are labeled derived. - Regeneration produces a semantic diff. - Snapshot acceptance is never automatic. - Dropping dimensions, weakening predicates, or expanding normalization requires review. - Stale evidence, incompatible schemas, missing providers, and policy denial fail closed. - Paths become stable bindings instead of deleted identity. - CI requires no production credentials or external services. ### Acceptance Capture a real interactive authority/candidate pair, export the package, run it from a clean checkout with different temp roots and ports, detect an injected behavior change, and prove that weakening normalization or expected dimensions changes the fixture digest and requires approval.

bsakaguchi

2026-10-15 18:20:30 +00:00

closed this issue
added the
area: build

P1: high

roadmap

enhancement

area: tests
labels

bsakaguchi commented

2026-10-15 18:20:32 +00:00

Author

Owner

@morluto commented on GitHub (Jul 17, 2026):

Current main has a source-owned process fidelity fixture and broad denial, bounds, redaction, cancellation, and replay coverage. The remaining scope is portable capture-to-conformance export: scenario/replay/shim/evidence packages, semantic-diff and trust gates, clean-checkout authority/candidate export, and CI replay. It depends on the expanded #57/#60/#61 work.

@morluto commented on GitHub (Jul 17, 2026): Current main has a source-owned process fidelity fixture and broad denial, bounds, redaction, cancellation, and replay coverage. The remaining scope is portable capture-to-conformance export: scenario/replay/shim/evidence packages, semantic-diff and trust gates, clean-checkout authority/candidate export, and CI replay. It depends on the expanded #57/#60/#61 work.

bsakaguchi commented

2026-10-15 18:20:32 +00:00

Author

Owner

@morluto commented on GitHub (Jul 17, 2026):

Current-main audit: source-owned interactive, silent, partial, process-tree, crash, and hang fixtures cover bounds, redaction, cancellation, cleanup, filesystem effects, and replay. Remaining scope is portable capture-to-conformance export: a committed scenario/replay/shim/evidence package, semantic-diff trust gates, and clean-checkout CI replay. Keep open with that reduced scope.

@morluto commented on GitHub (Jul 17, 2026): Current-main audit: source-owned interactive, silent, partial, process-tree, crash, and hang fixtures cover bounds, redaction, cancellation, cleanup, filesystem effects, and replay. Remaining scope is portable capture-to-conformance export: a committed scenario/replay/shim/evidence package, semantic-diff trust gates, and clean-checkout CI replay. Keep open with that reduced scope.

bsakaguchi commented

2026-10-15 18:20:32 +00:00

Author

Owner

@morluto commented on GitHub (Jul 20, 2026):

Split into focused child issues:

These replace the broad remaining scope in this issue. Closing this parent can happen once all children are closed.

@morluto commented on GitHub (Jul 20, 2026): Split into focused child issues: - [ ] #336 - [ ] #337 - [ ] #338 These replace the broad remaining scope in this issue. Closing this parent can happen once all children are closed.

bsakaguchi referenced this issue

2026-10-15 18:21:47 +00:00

[GH-ISSUE #268] Build an evidence-backed reconstruction workspace and coverage closure workflow #157

bsakaguchi referenced this issue

2026-10-15 18:28:29 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-10-31 00:28:40 +00:00