[PR #398] [MERGED] feat: attach image drives to warm starts #405

New issue

Closed

opened 2026-10-29 14:23:02 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:23:02 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/398
Author: @Primary33
Created: 10/20/2026
Status: ✅ Merged
Merged: 10/26/2026
Merged by: @yingdi-shan

Base: main ← Head: feat/image-mounts

📝 Commits (1)

136e662 feat: attach image drives to warm starts

📊 Changes

19 files changed (+1186 additions, -273 deletions)

View changed files

📝 crates/aenv/src/client/sandboxes.rs (+57 -9)
📝 crates/aenv/src/commands/start.rs (+168 -3)
📝 docs/src/concepts/sandboxes/starting.md (+37 -0)
📝 docs/src/getting-started/aenv-cli/sandboxes.md (+2 -0)
📝 src/api/generated/src/models.rs (+17 -0)
📝 src/api/impls/sandbox.rs (+331 -175)
📝 src/api/openapi.yml (+9 -0)
📝 src/sandbox/extra_drive.rs (+4 -2)
📝 src/sandbox/firecracker/config.rs (+38 -10)
📝 src/sandbox/firecracker/sandbox.rs (+227 -38)
📝 src/sandbox/mod.rs (+1 -1)
📝 src/snapshot/backends/tests.rs (+2 -2)
📝 src/snapshot/store/posix_tests.rs (+1 -1)
📝 src/snapshot/types/drive.rs (+1 -1)
📝 src/snapshot/types/manifest.rs (+37 -8)
📝 src/snapshot/types/snapshot.rs (+11 -0)
📝 src/types/extra_drive.rs (+111 -12)
📝 src/types/mod.rs (+1 -1)
📝 tests/integration/snapshot_attached_drive.rs (+131 -10)

📄 Description

What

Add attachedDrives to POST /v2/sandboxes, so warm starts can attach image drives the same way cold starts do. aenv start --image-mount PATH=IMAGE (writable) and --image-mount-ro PATH=IMAGE build attachedDrives entries for both warm and cold starts; CLI warm starts now use v2.

Why

Mount models, datasets, or code from an image without managing persistent volumes or building a template for each combination.

Requested in a maintainer discussion. Builds on merged #405. Replaces the earlier imageMounts design in this PR, following the review.

Scope and non-goals

Reuses the existing AttachedDrive schema, defaults, and resolve_attached_drives; the cold-start path is unchanged.
No new imageMounts API, no catalog records, and no sweep task.
Also fixes read-only volumes on cold start: they are mounted through envd with ro,noload after boot instead of by init.

Design and behavior changes

ExtraDrive.volume: bool becomes ExtraDriveKind { Physical, LaunchSlot, Volume }. Drives added at a snapshot launch are LaunchSlot and take reserved Firecracker slots ahead of volumes.
On capture, LaunchSlot drives are committed to the manifest like physical drives but keep their slots, because the VM state records them under the slot device IDs. physical_extra_drive_count now counts only drives with their own device IDs; the attached drives after them occupy the first slots, and launch-time drives use the slots that follow.
Relaunch, fork, and pause/resume restore these drives from the snapshot without extra code. Slot drives already in the VM state are bound before snapshot load; drives new to this launch are bound after load and sized like a fresh boot (diskSizeMB is the target size).
Snapshots with volumes or attached drives reject explicit attachedDrives, including []. volumeMounts keeps the #405 rule. Attached drives plus explicit volumes must fit in the snapshot's free slots, and attached drive paths must not overlap volume mount paths (HTTP 400).

Compatibility and operations

Public API or generated protocol: optional attachedDrives on NewSandboxV2; regenerated models.
Configuration or defaults: none. readOnly still defaults to true in the API; --image-mount sends false.
Snapshot manifest, artifact layout, or storage format: no new fields. physical_extra_drive_count may now be smaller than the number of attached drives. Persisted ExtraDrive keeps the boolean volume field for physical drives and volumes; LaunchSlot is written as "launch_slot".
Upgrade and rollback: older servers misplace slot drives in new snapshots and fail to parse paused sandboxes with LaunchSlot drives; upgrade servers and the CLI together.
Host requirements, permissions, ports, or dependencies: none.

Validation

Validated on Ubuntu 24.04 as a non-root user with isolated state and dependencies.

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Commands and results:

make agentenv-server            # OpenAPI Generator 7.22.0; NewSandboxV2 output unchanged
make fmt && make clippy         # pass
make test-unit                  # 1218 passed, 0 failed; capability/install script checks pass
cargo test -p agentenv --test integration snapshot_attached_drive   # 4 passed
make test-agent-integration     # integration 22 passed, 1 ignored; orchestrator 2 passed; OSS E2E 9 passed
scripts/tests/e2e/suites/15_volume.sh   # 56 passed

New integration test launch_time_attached_drive_keeps_its_slot_across_captures covers launch-time attach, capture, relaunch, fork, and a second capture.

Skipped checks and reasons:

make -C services test: services/ is unchanged.
16_volume_randomized.sh: needs a multi-node Compose deployment.
posix_startup_pack_publish_resolve_and_resume: ignored by default; needs startup-pack configuration.

Risks and reviewer notes

Slot bookkeeping is the main risk: see FirecrackerSandbox::snapshot_config_for_launch, prepare_resume_drives, and the manifest count in pause_to_dir (src/sandbox/firecracker/sandbox.rs), plus SandboxSnapshotManifest::extra_drives and free_drive_slots. API validation is in create_sandbox_from_snapshot (src/api/impls/sandbox.rs); the v1 sandboxes_post body moved there unchanged apart from the new checks. Warm-start attached drives are not added to MMDS imageConfigs.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/398 **Author:** [@Primary33](https://github.com/Primary33) **Created:** 10/20/2026 **Status:** ✅ Merged **Merged:** 10/26/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `feat/image-mounts` --- ### 📝 Commits (1) - [`136e662`](https://github.com/kvcache-ai/AgentENV/commit/136e6624cf07c391e467b45dda719fe68a911858) feat: attach image drives to warm starts ### 📊 Changes **19 files changed** (+1186 additions, -273 deletions) <details> <summary>View changed files</summary> 📝 `crates/aenv/src/client/sandboxes.rs` (+57 -9) 📝 `crates/aenv/src/commands/start.rs` (+168 -3) 📝 `docs/src/concepts/sandboxes/starting.md` (+37 -0) 📝 `docs/src/getting-started/aenv-cli/sandboxes.md` (+2 -0) 📝 `src/api/generated/src/models.rs` (+17 -0) 📝 `src/api/impls/sandbox.rs` (+331 -175) 📝 `src/api/openapi.yml` (+9 -0) 📝 `src/sandbox/extra_drive.rs` (+4 -2) 📝 `src/sandbox/firecracker/config.rs` (+38 -10) 📝 `src/sandbox/firecracker/sandbox.rs` (+227 -38) 📝 `src/sandbox/mod.rs` (+1 -1) 📝 `src/snapshot/backends/tests.rs` (+2 -2) 📝 `src/snapshot/store/posix_tests.rs` (+1 -1) 📝 `src/snapshot/types/drive.rs` (+1 -1) 📝 `src/snapshot/types/manifest.rs` (+37 -8) 📝 `src/snapshot/types/snapshot.rs` (+11 -0) 📝 `src/types/extra_drive.rs` (+111 -12) 📝 `src/types/mod.rs` (+1 -1) 📝 `tests/integration/snapshot_attached_drive.rs` (+131 -10) </details> ### 📄 Description ## What Add `attachedDrives` to `POST /v2/sandboxes`, so warm starts can attach image drives the same way cold starts do. `aenv start --image-mount PATH=IMAGE` (writable) and `--image-mount-ro PATH=IMAGE` build `attachedDrives` entries for both warm and cold starts; CLI warm starts now use v2. ## Why Mount models, datasets, or code from an image without managing persistent volumes or building a template for each combination. ## Related issue Requested in a maintainer discussion. Builds on merged #405. Replaces the earlier `imageMounts` design in this PR, following the review. ## Scope and non-goals - Reuses the existing `AttachedDrive` schema, defaults, and `resolve_attached_drives`; the cold-start path is unchanged. - No new `imageMounts` API, no catalog records, and no sweep task. - Also fixes read-only volumes on cold start: they are mounted through envd with `ro,noload` after boot instead of by init. ## Design and behavior changes - `ExtraDrive.volume: bool` becomes `ExtraDriveKind { Physical, LaunchSlot, Volume }`. Drives added at a snapshot launch are `LaunchSlot` and take reserved Firecracker slots ahead of volumes. - On capture, `LaunchSlot` drives are committed to the manifest like physical drives but keep their slots, because the VM state records them under the slot device IDs. `physical_extra_drive_count` now counts only drives with their own device IDs; the attached drives after them occupy the first slots, and launch-time drives use the slots that follow. - Relaunch, fork, and pause/resume restore these drives from the snapshot without extra code. Slot drives already in the VM state are bound before snapshot load; drives new to this launch are bound after load and sized like a fresh boot (`diskSizeMB` is the target size). - Snapshots with volumes or attached drives reject explicit `attachedDrives`, including `[]`. `volumeMounts` keeps the #405 rule. Attached drives plus explicit volumes must fit in the snapshot's free slots, and attached drive paths must not overlap volume mount paths (HTTP 400). ## Compatibility and operations - Public API or generated protocol: optional `attachedDrives` on `NewSandboxV2`; regenerated models. - Configuration or defaults: none. `readOnly` still defaults to `true` in the API; `--image-mount` sends `false`. - Snapshot manifest, artifact layout, or storage format: no new fields. `physical_extra_drive_count` may now be smaller than the number of attached drives. Persisted `ExtraDrive` keeps the boolean `volume` field for physical drives and volumes; `LaunchSlot` is written as `"launch_slot"`. - Upgrade and rollback: older servers misplace slot drives in new snapshots and fail to parse paused sandboxes with `LaunchSlot` drives; upgrade servers and the CLI together. - Host requirements, permissions, ports, or dependencies: none. ## Validation Validated on Ubuntu 24.04 as a non-root user with isolated state and dependencies. - [x] `make fmt` - [x] `make clippy` - [x] `make test-unit` - [x] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [x] Generated clients/server regenerated with the documented `make` target - [x] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results: ```text make agentenv-server # OpenAPI Generator 7.22.0; NewSandboxV2 output unchanged make fmt && make clippy # pass make test-unit # 1218 passed, 0 failed; capability/install script checks pass cargo test -p agentenv --test integration snapshot_attached_drive # 4 passed make test-agent-integration # integration 22 passed, 1 ignored; orchestrator 2 passed; OSS E2E 9 passed scripts/tests/e2e/suites/15_volume.sh # 56 passed ``` New integration test `launch_time_attached_drive_keeps_its_slot_across_captures` covers launch-time attach, capture, relaunch, fork, and a second capture. Skipped checks and reasons: - `make -C services test`: `services/` is unchanged. - `16_volume_randomized.sh`: needs a multi-node Compose deployment. - `posix_startup_pack_publish_resolve_and_resume`: ignored by default; needs startup-pack configuration. ## Risks and reviewer notes Slot bookkeeping is the main risk: see `FirecrackerSandbox::snapshot_config_for_launch`, `prepare_resume_drives`, and the manifest count in `pause_to_dir` (`src/sandbox/firecracker/sandbox.rs`), plus `SandboxSnapshotManifest::extra_drives` and `free_drive_slots`. API validation is in `create_sandbox_from_snapshot` (`src/api/impls/sandbox.rs`); the v1 `sandboxes_post` body moved there unchanged apart from the new checks. Warm-start attached drives are not added to MMDS `imageConfigs`. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:23:02 +00:00