mirror of
https://github.com/morluto/rea.git
synced 2026-11-05 06:45:22 +00:00
[GH-ISSUE #4933] [Design] Project captures into an environment manifest, serve it over streamable HTTP, export agent-environment bundles #3060
Closed
opened 2026-10-31 00:35:41 +00:00 by bsakaguchi
·
4 comments
No Branch/Tag specified
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
Mirrored from GitHub Pull Request
question
roadmap
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
morluto/rea#3060
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @earakely-scale on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4933
Before submitting
Decision needed
Should REA (1) project retained web network capture Evidence into a canonical environment manifest analysis view, (2) optionally serve its MCP catalog over streamable HTTP behind an explicit
rea mcp --httpflag, (3) serve a manifest as an agent environment (rea environment-serve) that replays recorded exchanges with provenance, and (4) export a manifest as a bundle folder (rea environment-export) that the open-source agentenv framework can run withagent-env run? Each of the four can be decided independently; (1) is a pure analysis view, (2) and (3) add an inbound loopback listener, (4) adds a CI lane that needs Docker and a pinned Python package.Context and evidence
Observed. REA's
api-surfaceview already recovers an application's request-line inventory from a HAR capture. We ran the publishedrea-agents@6.5.0against a Playwright 1.58.2 capture of one browsing session on a self-hosted Grafana 13.2.4 (open source, run locally, admin credentials only, login exchange dropped and secret body fields scrubbed before inspection): 305 records → 99 api-surface groups. A small external generator turned those groups plus the recorded exchanges into an MCP environment with 73 tools that the agentenv framework deployed through its gateway and verified end to end (four tool calls returned the recorded answers). Checked against the OpenAPI document Grafana serves itself, 16 of 16 templated path segments agreed with the spec's parameter positions; 0 were over-templated, 0 missed. The full 305-record capture exceeded theapi-surface96 MiB projection budget, so inspection ran in eight 40-record chunks.Source-derived.
createServer()andEvidenceMcpServer.connect(transport)are transport-agnostic; onlysrc/main.tsbindsserveStdio. The already-pinned@modelcontextprotocol/server@2.3.1root export shipsWebStandardStreamableHTTPServerTransport, so streamable HTTP needs a ~40-linenode:http↔ webRequest/Responsebridge and no new dependency.BinarySessionowns one active target per process, so an HTTP mode is a one-client, one-session transport. REA's four template rules (numeric, uuid, hex, iso-date) leave Grafana's 14-character opaque UIDs and plugin identifiers literal, which the external generator covered with three extra rules.Proposed. Everything REA does stays local and deterministic and produces ordinary files; running the result is the user's choice. Replay serving is not a replay engine in the orchestration sense: it answers each tool call with one retained record, cites the Evidence id and record ordinal, keeps
_meta.provenance.untrusted_content, and executes nothing. The manifest carries explicitunknowns(authorization model, response semantics beyond recorded exchanges, state transitions) instead of inventing behavior.Unknown. Whether the maintainer considers any inbound listener in scope given "REA is a local-only reverse-engineering tool"; whether a hosted lane that installs a pinned third-party Python package and builds a Docker image is acceptable; whether JavaScript application, EVM, WASM and SQLite Evidence should feed the same manifest view later (the view envelope is single-parent today).
Expected interaction with targets
0.0.0.0must be requested explicitly)Boundary details (optional)
Non-goals: no outbound network calls; no process launch or attach by the serve runtime; no target execution; no change to
server.json,rea setupregistrations or installation docs, which stay stdio. The HTTP listener serves one session; host-header validation uses the SDK's localhost allow-list for loopback binds and is relaxed only when the caller binds a non-loopback host, which is logged. The manifest view inheritsapi-surface's sensitive-value refusal and byte budgets. Export writes the parent capture Evidence beside the manifest so the served environment can prove the manifest matches the Evidence it was projected from (ids must agree at startup).Options and tradeoffs
inspect_analysis_viewenvelope, budgets and Evidence wrapping, and adds no catalog bytes beyond the request schema. A new tool would allow multi-parent fusion now but costs catalog tokens; we propose the view first and multi-parent fusion as a later decision.after-uid-literal,opaque-token,sibling-variance) are off unless requested in the view and are labelled per parameter, so default output stays as conservative asapi-surface.@modelcontextprotocol/node. The bridge is small and adds no dependency; the node package pulls hono and lags the server package version.environment-serveandenvironment-exportfollowevidence-export/evidence-import; no nested command groups.docs/specs/agent-environments/<slice>/{spec,plan,tasks,verification}.mdwith anIssue:line, following the multi-increment precedent, so one issue covers four small PRs.Verification available so far
I have a prototype or observed results
Success criteria and open questions
rea inspect-analysis-viewwithview.kind = "environment-manifest"on the existing HAR fixtures yields a manifest whose templates matchapi-surfaceexactly when no extra rule is requested, and whose extra-rule parameters are each labelled with their rule.rea mcp --httppasses initialize, tools/list, one tool call and DELETE from the Pythonmcpstreamable HTTP client, and refuses non-loopback hosts unless asked.rea environment-serveserves the env card and replays the Grafana-shaped fixture;rea environment-exportplusagent-env run <dir>passes the exported smoke task on a hosted Ubuntu runner.References
echo-mcpexample is the bundle shape we mirror)@XinwangTao commented on GitHub (Oct 18, 2026):
Nice direction — the separation into four independently decidable items is the right framing. (1) as a pure analysis view is clearly safe and useful: it turns an already-recovered request-line inventory into a reusable contract artifact without any new runtime surface. I'd also like (4): exporting a bundle that agentenv can run is the strongest end-to-end validation of the manifest format itself, and making the verification reproducible (pinned Python package + Docker CI lane) keeps that signal trustworthy. For (2)/(3), since both add a loopback listener, one concern is lifetime and safety: binding only on 127.0.0.1, refusing cross-origin requests, and tying the server's lifecycle to the explicit flag (or to a short-lived, automatically-terminated process) would keep this from becoming a footgun for users who run
rea mcpunattended. Also worth clarifying how secrets in recorded exchanges are handled when exporting a bundle — the scrubbing you did for Grafana should probably be an enforced step, not a manual one, beforeenvironment-exportwrites anything to disk.@XinwangTao commented on GitHub (Oct 18, 2026):
Two cents on the open questions: (1) keep the serve runtime stateless for v1 — a replay environment's value is deterministic, idempotent answers; a reset operation on the data plane implies mutable session state that agentenv would have to discover and use consistently, which complicates the bundle contract. Stateless replay keeps the exported bundle self-contained (state lives in the parent Evidence plus the manifest). (2) On templating opaque identifiers by default: keep it opt-in via the extra rules as proposed. Default templating of unknown-shaped identifiers would silently over-generalize paths in captures where the literal is actually part of the resource model — the Grafana 14-char UID is a good example of the former, but plenty of APIs have semantically meaningful path segments that don't match the four built-in rules. Label-and-opt-in keeps api-surface conservative and lets the caller decide per view.
@earakely-scale commented on GitHub (Oct 18, 2026):
@morluto, requesting a scope decision on the four items above, now that each has a PR to read:
environment-serve: #4975. Serves a manifest as a replay MCP server; reads two Evidence files once, binds one port, executes nothing from the captured application.environment-exportplus the hostedreal-agentenvlane: #4999. Writes a bundle folder at the path the caller names; the lane installs a pinned Python package and builds a Docker image on the runner, which is the second question asked above.Items 1 and 2 stand alone; 3 stacks on both; 4 stacks on 3, so a decision on 1 and 2 alone would already let the first two merge.
What the four items produce end to end, built with the prerelease from these branches and published so it can be inspected: five replay environments generated from one recorded browser session each against self-hosted Grafana, Gitea, NocoDB, Mattermost and Ghost, with synthetic data.
Happy to narrow any item or split the lane into its own PR if that helps the decision.
@morluto commented on GitHub (Oct 26, 2026):
Thank you for separating the scope decisions and supplying the end-to-end examples. The four proposed slices have now landed: the manifest view in #4954, streamable HTTP in #4955, replay serving in #4975, and bundle export plus agentenv verification in #4999. The serving implementation records its limits, including historical replay without application-state reconstruction. Closing this design request as completed; future extensions or different replay semantics can be discussed separately.