[PR #400] fix: restore paused state when resume races shutdown

bsakaguchi commented

2026-10-29 14:23:03 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/400
Author: @mvanhorn
Created: 10/20/2026
Status: 🔄 Open

Base: main ← Head: fix/372-resume-shutdown-rollback

📝 Commits (1)

de0b6ad fix: restore paused state when resume races shutdown

📊 Changes

2 files changed (+237 additions, -8 deletions)

View changed files

📝 src/orchestrator/service.rs (+19 -7)
📝 src/orchestrator/tests.rs (+218 -1)

📄 Description

What

A paused sandbox whose resume overlaps shutdown is left mid-resume. Later resume, pause, delete, and keep-alive calls wait on that sandbox until they time out, and the next startup discards its paused memory snapshot and writable layer. Resume commits the store and the persisted record from paused to resuming before launch checks whether the process is still accepting work. The shutdown rejection at the start of launch returns immediately and is the only failure there that does not restore the paused record. A resume with no paused runtime state used to fail only after that same commit, so it could leave the same transitional record behind.

Paused runtime state is now required before the paused-to-resuming update, so a missing snapshot returns an internal error and does not mark the record resuming. If launch is rejected because shutdown has started and the plan is a resume, the existing failed-launch rollback restores the paused state and the persisted record, then the original shutdown error is returned. A create rejected at that gate still writes nothing. New tests cover a missing paused state, shutdown during the resuming mark, and a resume launch that starts after shutdown is already set.

Why

Paused runtime state is now required before the paused-to-resuming update, so a missing snapshot returns an internal error and does not mark the record resuming. If launch is rejected because shutdown has started and the plan is a resume, the existing failed-launch rollback restores the paused state and the persisted record, then the original shutdown error is returned. A create rejected at that gate still writes nothing. New tests cover a missing paused state, shutdown during the resuming mark, and a resume launch that starts after shutdown is already set.

Closes #372

Scope and non-goals

Covered in the summary above.

Design and behavior changes

Covered in the summary above.

Compatibility and operations

Not applicable to this change.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Ran make test locally on macOS for the AgentENV change; it fails the same way on the base branch, so the failure predates this change (it fails the same way on upstream/main). Tests for the AgentENV change live in src/orchestrator/tests.rs.

Risks and reviewer notes

Scoped to the files listed above; no other caller or consumer changes.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
No test file is among the paths the AgentENV change changes (src/orchestrator/service.rs and src/orchestrator/tests.rs).
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/400 **Author:** [@mvanhorn](https://github.com/mvanhorn) **Created:** 10/20/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/372-resume-shutdown-rollback` --- ### 📝 Commits (1) - [`de0b6ad`](https://github.com/kvcache-ai/AgentENV/commit/de0b6ad5abd26c59fd0b59a204b46f3bf4abcad8) fix: restore paused state when resume races shutdown ### 📊 Changes **2 files changed** (+237 additions, -8 deletions) <details> <summary>View changed files</summary> 📝 `src/orchestrator/service.rs` (+19 -7) 📝 `src/orchestrator/tests.rs` (+218 -1) </details> ### 📄 Description ## What A paused sandbox whose resume overlaps shutdown is left mid-resume. Later resume, pause, delete, and keep-alive calls wait on that sandbox until they time out, and the next startup discards its paused memory snapshot and writable layer. Resume commits the store and the persisted record from paused to resuming before launch checks whether the process is still accepting work. The shutdown rejection at the start of launch returns immediately and is the only failure there that does not restore the paused record. A resume with no paused runtime state used to fail only after that same commit, so it could leave the same transitional record behind. Paused runtime state is now required before the paused-to-resuming update, so a missing snapshot returns an internal error and does not mark the record resuming. If launch is rejected because shutdown has started and the plan is a resume, the existing failed-launch rollback restores the paused state and the persisted record, then the original shutdown error is returned. A create rejected at that gate still writes nothing. New tests cover a missing paused state, shutdown during the resuming mark, and a resume launch that starts after shutdown is already set. ## Why Paused runtime state is now required before the paused-to-resuming update, so a missing snapshot returns an internal error and does not mark the record resuming. If launch is rejected because shutdown has started and the plan is a resume, the existing failed-launch rollback restores the paused state and the persisted record, then the original shutdown error is returned. A create rejected at that gate still writes nothing. New tests cover a missing paused state, shutdown during the resuming mark, and a resume launch that starts after shutdown is already set. ## Related issue Closes #372 ## Scope and non-goals Covered in the summary above. ## Design and behavior changes Covered in the summary above. ## Compatibility and operations Not applicable to this change. ## Validation - [ ] `make fmt` - [ ] `make clippy` - [ ] `make test-unit` - [ ] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [ ] Documentation updated - [ ] Benchmarks or performance comparison completed Ran `make test` locally on macOS for the AgentENV change; it fails the same way on the base branch, so the failure predates this change (it fails the same way on upstream/main). Tests for the AgentENV change live in `src/orchestrator/tests.rs`. ## Risks and reviewer notes Scoped to the files listed above; no other caller or consumer changes. ## Checklist - [ ] The PR contains one coherent change and no unrelated formatting or refactoring. - [ ] New behavior is covered by tests, or I explained why testing is impractical. No test file is among the paths the AgentENV change changes (`src/orchestrator/service.rs` and `src/orchestrator/tests.rs`). - [ ] Logs and examples contain no credentials, tokens, or private registry information. - [ ] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-29 14:23:03 +00:00

bsakaguchi referenced this issue

2026-10-29 14:23:06 +00:00

[PR #409] [MERGED] fix(sandbox): announce guest MAC changes via arp_notify #412