kvcache-ai/AgentENV

Fork 0

mirror of https://github.com/kvcache-ai/AgentENV.git synced 2026-11-05 01:17:36 +00:00

[PR #325] [CLOSED] feat(compose): add Docker Compose sandboxes #350

New issue

Closed

opened 2026-10-29 14:22:30 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:22:30 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/325
Author: @Primary33
Created: 9/29/2026
Status: ❌ Closed

Base: main ← Head: feat/sandbox-compose

📝 Commits (1)

ac97300 feat(compose): add Docker Compose sandboxes

📊 Changes

60 files changed (+6544 additions, -261 deletions)

View changed files

📝 .github/workflows/release.yml (+10 -0)
➕ compose-image/Dockerfile (+118 -0)
➕ compose-image/test_e2e.py (+425 -0)
📝 crates/aenv/src/client/sandboxes.rs (+106 -1)
📝 crates/aenv/src/commands/build.rs (+246 -83)
➕ crates/aenv/src/commands/compose.rs (+486 -0)
➕ crates/aenv/src/commands/compose/runtime.rs (+520 -0)
📝 crates/aenv/src/commands/mod.rs (+11 -0)
📝 crates/aenv/src/commands/start.rs (+2 -10)
📝 crates/aenv/src/main.rs (+3 -0)
➕ crates/aenv/tests/compose_runtime.rs (+62 -0)
📝 deploy/docker/Dockerfile.agentenv (+7 -0)
📝 docs/src/SUMMARY.md (+1 -0)
➕ docs/src/concepts/sandboxes/compose.md (+302 -0)
📝 scripts/install.sh (+3 -0)
📝 services/Makefile (+5 -0)
📝 services/api/proto/scheduler.pb.go (+30 -9)
📝 services/api/proto/scheduler.proto (+4 -0)
➕ services/compose/cmd/main.go (+61 -0)
➕ services/compose/cmd/main_test.go (+52 -0)

...and 40 more files

📄 Description

What

Run multi-service Docker Compose projects inside one Firecracker sandbox, including remote Dockerfile builds that import service images directly into the node cache:

aenv build --compose compose.yaml --output compose.built.yaml
aenv compose up -f compose.built.yaml

Terminal-Bench 4.0: all 11 Compose task environments passed, covering 38 services. Validation includes original environment builds, startup, configured health checks, and execution inside every task's main container. This is environment validation; agent solving and task scoring were not run.

Why

Multi-service tasks need to build and start without a client-side Docker daemon or destination registry, while preserving independent writable service filesystems and AgentENV's existing pause, resume, fork, and snapshot behavior.

Builds on the plain snapshotter introduced in #318.

Scope and non-goals

Supports Linux x86-64/KVM, 1–24 services, local Dockerfile contexts, build arguments/targets/no-cache, profiles, explicit environment interpolation, dependencies, health checks, named volumes, bridge networking, fixed TCP ports, shared service network namespaces, and SYS_PTRACE.

Runtime requests remain image-only. Scaling, host bind mounts, external files/resources, privileged containers, host networking/devices, arbitrary added capabilities, and UDP publishing remain unsupported. --harbor supplies task build/keepalive defaults; a Harbor backend and benchmark agents/verifiers are outside this change.

Design and behavior changes

The CLI requests an authenticated build plan from POST /sandboxes-compose/plan. Planning runs on the server, has a 30-second budget, reads no client files, and allocates no builders or sandboxes. The Go planner is packaged with the server; build contexts are uploaded separately through the existing BuildKit tunnel.
aenv build --compose builds active services and atomically publishes the normalized Compose file after all builds succeed. Existing output files are rejected. Default builds import manifest digests into one node's cache; --image-repository optionally distributes images through a registry.
Clients use build IDs and image digests without node identifiers or placement headers. The gateway resolves active YAML/JSON services and passes image dependencies to the scheduler, which selects healthy, eligible capacity holding every required digest. Node heartbeats replace cache inventories, and successful image imports publish availability before reporting ready. Durable image-build results keep their build-ID routing until explicit deletion, independently of worker cleanup; snapshot storage formats are unchanged.
Each service receives an independent writable drive. The Linux CLI's hidden guest commands register the mounted filesystems with the plain snapshotter and run Compose without guest-side builds or pulls. Guest runtime assets and the optional compatible-kernel build target are in compose-image/Dockerfile.
Compose startup, rollback, pause/resume, fork, capture, and deletion use the existing sandbox lifecycle. Cached image resolution metadata is refreshed when reusing entries, and gateway root-path proxying reaches the service's / route.

Compatibility and operations

Public API / generated protocol: additive Compose creation/planning endpoints and optional image-build/status fields; OpenAPI source and regenerated Rust bindings are included. Existing template builds retain their behavior.
Configuration / defaults: creation is enabled with compose.base_image; planning requires compose.planner_binary. Only the server needs the planner. Default Compose builds require no destination registry; base-image pulls still use their configured sources.
Snapshot / storage: existing snapshot manifests and artifact formats are unchanged. Imported images are node-local cache entries subject to eviction; durable build metadata records the image-only result.
Upgrade / rollback: upgrade the server/planner, gateway, scheduler, CLI, and guest runtime together. Complete active image-only builds and retire Compose sandboxes before rolling back runtime support. Snapshot restore requires a compatible guest kernel.
Host / dependencies: dedicated Docker/Compose guest image, container-capable guest kernel (including raw-table support), KVM, ublk, and the documented non-root runtime capabilities. The host kernel is not replaced. Optional registry mode uses client credentials for push and runtime-node credentials for pull.

Validation

All validation ran on a dedicated Ubuntu 24.04 Linux x86-64/KVM test server, with non-root execution and isolated state/dependency directories. Normalized SHA-256 fingerprints for all 819 source files were checked against the final working tree.

Terminal-Bench 4.0 — 11/11 environments passed

Tested the unmodified v4.0.0 task checkout, commit 452bf305c6daa62fc59061d22133a7cbc7c1572e, using the repository's existing environment harness:

python3 -u compose-image/test_e2e.py terminal-bench \
  --tasks /path/to/terminal-bench/tasks \
  --results /path/to/results

AENV_API_URL, AENV_API_KEY, and AENV_CLI point to the isolated test deployment. The default node-local image-import path was used, with no destination registry supplied to task builds.

Task	Services	Final environment result
`ctr-optimization`	2	Passed
`cumulative-layout-shift`	2	Passed
`freight-dispatch-shift`	2	Passed
`heat-pump-warranty`	7	Passed
`intrastat-meldung`	6	Passed
`kv-live-surgery`	2	Passed
`legacy-utility-triage`	3	Passed
`live-database-cutover`	5	Passed
`medical-claims-processing`	3	Passed
`nextjs-performance`	2	Passed
`payments-pipeline-fix`	4	Passed
Total	38	11/11 passed

For every task, the harness built the original Compose environment, started its sandbox, checked that each service was running or had exited successfully, verified every configured health check, and executed sh -c true in main. The source checkout remained clean. All task sandboxes were deleted; final running, starting, and paused sandbox counts were zero, and the isolated server and ublk daemon were stopped.

These results do not represent agent success rates or task verifier scores.

Project checks

cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets --all-features -- -D warnings
Rust library/CLI tests: 1,096 passed — 968 library tests, 124 CLI unit tests, and 4 CLI process integration tests. Capability tests were also validated separately in isolated namespaces.
make -C services fmt-check vet and go test ./... from services/
Capability-runner, service-installation, and buildctl-installation shell checks
Server, CLI, planner, ublk daemon, and complete Compose guest-image builds
Compose real-VM E2E: 6/6 passed (python3 -u compose-image/test_e2e.py)
BuildKit real-VM E2E: 9/9 passed (python3 -u scripts/buildkit/test.py)
OpenAPI source and bindings regenerated with OpenAPI Generator 7.22.0; documentation updated

Compose E2E covers planning without allocations, build args/target/context/.dockerignore, literal dollars, local image dependency scheduling, successful and failed build cleanup, CLI startup, service DNS/ports/health, original image defaults, pause/resume, fork isolation, snapshot restore, and rejection/failure cleanup. BuildKit E2E covers concurrency, instruction/cache mounts, deadlines, external Dockerfiles/secrets, cancellation/isolation, numeric users, publication after client exit, startup overrides, and terminal progress.

Validation fixes include an over-capacity recovery-heartbeat fixture, the guest image builder's missing Protobuf development package, and a redundant Result wrapper. Full unrelated hardware/storage suites, optional registry-push E2E, and benchmark solving/scoring were not run in this validation round; the aggregate make test-unit target was not used.

Image placement validation

Gateway and scheduler race checks pass with go test -race ./gateway/internal ./scheduler/internal.
Coverage includes YAML/JSON image references, explicit interpolation and profiles, multiple-image placement, cache eviction, unhealthy/full/draining nodes, scheduler inventory recovery, retained build-result routing, serialized heartbeat publication, and the JSON error contract for gateway validation.
All 4 capability tests pass with the existing non-root runner in isolated namespaces.
Compose and BuildKit E2E were rerun through an isolated gateway, scheduler, and runtime, using the generated files without node markers or routing headers.
A real-VM recovery probe passed after worker cleanup, binding expiry, runtime restart, and scheduler restart: build-ID routing and local-image inventories recovered through heartbeats, the image started successfully, and explicit deletion left zero running/starting/paused sandboxes. Image scheduling returns a retryable 503 until a fresh inventory arrives after scheduler restart.

Risks and reviewer notes

Node-local images require eligible capacity holding all required digests; cache eviction can require rebuilding, and creation fails when no such capacity is available. Registry mode provides distribution across nodes. Partial failures may leave imported or pushed images for normal cache/registry retention to reclaim.

Review the server planning endpoint, image-only publication/cleanup, build-result routing, cache inventory replacement, scheduler eligibility checks, and writable service-drive allocation. The plain snapshotter retains active metadata in memory, so independent Docker/containerd/snapshotter restarts are unsupported. Recreating a container reuses its writable rootfs; creating a new sandbox resets it.

Checklist

One coherent Compose feature, including its image-build, lifecycle, and routing support.
New behavior and regressions are covered by tests.
Logs/examples contain no credentials, tokens, private registry references, or test-server settings.
Generated bindings were updated from their source schema.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/325 **Author:** [@Primary33](https://github.com/Primary33) **Created:** 9/29/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/sandbox-compose` --- ### 📝 Commits (1) - [`ac97300`](https://github.com/kvcache-ai/AgentENV/commit/ac97300194786389b5d69f6269922376d2335c98) feat(compose): add Docker Compose sandboxes ### 📊 Changes **60 files changed** (+6544 additions, -261 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/release.yml` (+10 -0) ➕ `compose-image/Dockerfile` (+118 -0) ➕ `compose-image/test_e2e.py` (+425 -0) 📝 `crates/aenv/src/client/sandboxes.rs` (+106 -1) 📝 `crates/aenv/src/commands/build.rs` (+246 -83) ➕ `crates/aenv/src/commands/compose.rs` (+486 -0) ➕ `crates/aenv/src/commands/compose/runtime.rs` (+520 -0) 📝 `crates/aenv/src/commands/mod.rs` (+11 -0) 📝 `crates/aenv/src/commands/start.rs` (+2 -10) 📝 `crates/aenv/src/main.rs` (+3 -0) ➕ `crates/aenv/tests/compose_runtime.rs` (+62 -0) 📝 `deploy/docker/Dockerfile.agentenv` (+7 -0) 📝 `docs/src/SUMMARY.md` (+1 -0) ➕ `docs/src/concepts/sandboxes/compose.md` (+302 -0) 📝 `scripts/install.sh` (+3 -0) 📝 `services/Makefile` (+5 -0) 📝 `services/api/proto/scheduler.pb.go` (+30 -9) 📝 `services/api/proto/scheduler.proto` (+4 -0) ➕ `services/compose/cmd/main.go` (+61 -0) ➕ `services/compose/cmd/main_test.go` (+52 -0) _...and 40 more files_ </details> ### 📄 Description ## What Run multi-service Docker Compose projects inside one Firecracker sandbox, including remote Dockerfile builds that import service images directly into the node cache: ```sh aenv build --compose compose.yaml --output compose.built.yaml aenv compose up -f compose.built.yaml ``` **Terminal-Bench 4.0: all 11 Compose task environments passed, covering 38 services.** Validation includes original environment builds, startup, configured health checks, and execution inside every task's `main` container. This is environment validation; agent solving and task scoring were not run. ## Why Multi-service tasks need to build and start without a client-side Docker daemon or destination registry, while preserving independent writable service filesystems and AgentENV's existing pause, resume, fork, and snapshot behavior. ## Related issue Builds on the plain snapshotter introduced in #318. ## Scope and non-goals Supports Linux x86-64/KVM, 1–24 services, local Dockerfile contexts, build arguments/targets/no-cache, profiles, explicit environment interpolation, dependencies, health checks, named volumes, bridge networking, fixed TCP ports, shared service network namespaces, and `SYS_PTRACE`. Runtime requests remain image-only. Scaling, host bind mounts, external files/resources, privileged containers, host networking/devices, arbitrary added capabilities, and UDP publishing remain unsupported. `--harbor` supplies task build/keepalive defaults; a Harbor backend and benchmark agents/verifiers are outside this change. ## Design and behavior changes - The CLI requests an authenticated build plan from `POST /sandboxes-compose/plan`. Planning runs on the server, has a 30-second budget, reads no client files, and allocates no builders or sandboxes. The Go planner is packaged with the server; build contexts are uploaded separately through the existing BuildKit tunnel. - `aenv build --compose` builds active services and atomically publishes the normalized Compose file after all builds succeed. Existing output files are rejected. Default builds import manifest digests into one node's cache; `--image-repository` optionally distributes images through a registry. - Clients use build IDs and image digests without node identifiers or placement headers. The gateway resolves active YAML/JSON services and passes image dependencies to the scheduler, which selects healthy, eligible capacity holding every required digest. Node heartbeats replace cache inventories, and successful image imports publish availability before reporting ready. Durable image-build results keep their build-ID routing until explicit deletion, independently of worker cleanup; snapshot storage formats are unchanged. - Each service receives an independent writable drive. The Linux CLI's hidden guest commands register the mounted filesystems with the plain snapshotter and run Compose without guest-side builds or pulls. Guest runtime assets and the optional compatible-kernel build target are in `compose-image/Dockerfile`. - Compose startup, rollback, pause/resume, fork, capture, and deletion use the existing sandbox lifecycle. Cached image resolution metadata is refreshed when reusing entries, and gateway root-path proxying reaches the service's `/` route. ## Compatibility and operations - Public API / generated protocol: additive Compose creation/planning endpoints and optional image-build/status fields; OpenAPI source and regenerated Rust bindings are included. Existing template builds retain their behavior. - Configuration / defaults: creation is enabled with `compose.base_image`; planning requires `compose.planner_binary`. Only the server needs the planner. Default Compose builds require no destination registry; base-image pulls still use their configured sources. - Snapshot / storage: existing snapshot manifests and artifact formats are unchanged. Imported images are node-local cache entries subject to eviction; durable build metadata records the image-only result. - Upgrade / rollback: upgrade the server/planner, gateway, scheduler, CLI, and guest runtime together. Complete active image-only builds and retire Compose sandboxes before rolling back runtime support. Snapshot restore requires a compatible guest kernel. - Host / dependencies: dedicated Docker/Compose guest image, container-capable guest kernel (including raw-table support), KVM, ublk, and the documented non-root runtime capabilities. The host kernel is not replaced. Optional registry mode uses client credentials for push and runtime-node credentials for pull. ## Validation All validation ran on a dedicated Ubuntu 24.04 Linux x86-64/KVM test server, with non-root execution and isolated state/dependency directories. Normalized SHA-256 fingerprints for all 819 source files were checked against the final working tree. ### Terminal-Bench 4.0 — 11/11 environments passed Tested the unmodified `v4.0.0` task checkout, commit `452bf305c6daa62fc59061d22133a7cbc7c1572e`, using the repository's existing environment harness: ```sh python3 -u compose-image/test_e2e.py terminal-bench \ --tasks /path/to/terminal-bench/tasks \ --results /path/to/results ``` `AENV_API_URL`, `AENV_API_KEY`, and `AENV_CLI` point to the isolated test deployment. The default node-local image-import path was used, with no destination registry supplied to task builds. | Task | Services | Final environment result | | --- | ---: | --- | | `ctr-optimization` | 2 | Passed | | `cumulative-layout-shift` | 2 | Passed | | `freight-dispatch-shift` | 2 | Passed | | `heat-pump-warranty` | 7 | Passed | | `intrastat-meldung` | 6 | Passed | | `kv-live-surgery` | 2 | Passed | | `legacy-utility-triage` | 3 | Passed | | `live-database-cutover` | 5 | Passed | | `medical-claims-processing` | 3 | Passed | | `nextjs-performance` | 2 | Passed | | `payments-pipeline-fix` | 4 | Passed | | **Total** | **38** | **11/11 passed** | For every task, the harness built the original Compose environment, started its sandbox, checked that each service was running or had exited successfully, verified every configured health check, and executed `sh -c true` in `main`. The source checkout remained clean. All task sandboxes were deleted; final running, starting, and paused sandbox counts were zero, and the isolated server and ublk daemon were stopped. These results do not represent agent success rates or task verifier scores. ### Project checks - [x] `cargo fmt --all -- --check` - [x] `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - [x] Rust library/CLI tests: **1,096 passed** — 968 library tests, 124 CLI unit tests, and 4 CLI process integration tests. Capability tests were also validated separately in isolated namespaces. - [x] `make -C services fmt-check vet` and `go test ./...` from `services/` - [x] Capability-runner, service-installation, and buildctl-installation shell checks - [x] Server, CLI, planner, ublk daemon, and complete Compose guest-image builds - [x] Compose real-VM E2E: **6/6 passed** (`python3 -u compose-image/test_e2e.py`) - [x] BuildKit real-VM E2E: **9/9 passed** (`python3 -u scripts/buildkit/test.py`) - [x] OpenAPI source and bindings regenerated with OpenAPI Generator 7.22.0; documentation updated Compose E2E covers planning without allocations, build args/target/context/`.dockerignore`, literal dollars, local image dependency scheduling, successful and failed build cleanup, CLI startup, service DNS/ports/health, original image defaults, pause/resume, fork isolation, snapshot restore, and rejection/failure cleanup. BuildKit E2E covers concurrency, instruction/cache mounts, deadlines, external Dockerfiles/secrets, cancellation/isolation, numeric users, publication after client exit, startup overrides, and terminal progress. Validation fixes include an over-capacity recovery-heartbeat fixture, the guest image builder's missing Protobuf development package, and a redundant `Result` wrapper. Full unrelated hardware/storage suites, optional registry-push E2E, and benchmark solving/scoring were not run in this validation round; the aggregate `make test-unit` target was not used. ### Image placement validation - Gateway and scheduler race checks pass with `go test -race ./gateway/internal ./scheduler/internal`. - Coverage includes YAML/JSON image references, explicit interpolation and profiles, multiple-image placement, cache eviction, unhealthy/full/draining nodes, scheduler inventory recovery, retained build-result routing, serialized heartbeat publication, and the JSON error contract for gateway validation. - All 4 capability tests pass with the existing non-root runner in isolated namespaces. - Compose and BuildKit E2E were rerun through an isolated gateway, scheduler, and runtime, using the generated files without node markers or routing headers. - A real-VM recovery probe passed after worker cleanup, binding expiry, runtime restart, and scheduler restart: build-ID routing and local-image inventories recovered through heartbeats, the image started successfully, and explicit deletion left zero running/starting/paused sandboxes. Image scheduling returns a retryable 503 until a fresh inventory arrives after scheduler restart. ## Risks and reviewer notes Node-local images require eligible capacity holding all required digests; cache eviction can require rebuilding, and creation fails when no such capacity is available. Registry mode provides distribution across nodes. Partial failures may leave imported or pushed images for normal cache/registry retention to reclaim. Review the server planning endpoint, image-only publication/cleanup, build-result routing, cache inventory replacement, scheduler eligibility checks, and writable service-drive allocation. The plain snapshotter retains active metadata in memory, so independent Docker/containerd/snapshotter restarts are unsupported. Recreating a container reuses its writable rootfs; creating a new sandbox resets it. ## Checklist - [x] One coherent Compose feature, including its image-build, lifecycle, and routing support. - [x] New behavior and regressions are covered by tests. - [x] Logs/examples contain no credentials, tokens, private registry references, or test-server settings. - [x] Generated bindings were updated from their source schema. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:22:30 +00:00