[GH-ISSUE #42] Direct Firecracker start failures can retain sandbox resources #19

New issue

Closed

opened 2026-10-29 14:18:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:18:28 +00:00

Owner

Originally created by @morluto on GitHub (Jul 28, 2026).
Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/42

Originally assigned to: @LSX-s-Software on GitHub.

Problem

Two Firecracker startup failure paths can retain resources:

The public FirecrackerSandbox::start() returns when start_nowait() or readiness fails without awaiting stop().
Fresh startup applies fallible egress-policy configuration before storing the allocated network slot on the sandbox.

Orchestrator-managed launch paths already perform explicit cleanup. This issue is limited to direct/public starts and the network-policy failure window before ownership is recorded.

A. Public start transaction

start_nowait
  +-- error --> stop --> return original error
  `-- ok ----> wait_for_ready
                 +-- error --> stop --> return original error
                 `-- ok ----> state: Running

B. Network slot ordering

before: allocate -> set policy -> store ownership
                       |
                       `-- error -> manager slot remains allocated

after:  allocate -> store ownership -> set policy
                                      +-- error -> stop/Drop releases slot
                                      `-- ok ---> continue boot

Current behavior

Direct callers of start() can leave initialized runtime state and daemon-owned ublk resources retained until daemon shutdown. If fresh-start egress configuration fails before self.network_slot is assigned, the namespace objects are dropped but the network manager does not receive the slot allocation back.

Expected behavior

Treat public startup and readiness as one transaction.
Await stop() after either operation fails while returning the primary startup error.
Record ownership of an allocated network slot before running fallible configuration.
Leave existing orchestrator cleanup and daemon shutdown behavior unchanged.

Impact

Repeated failure paths can reduce the available ublk or network-slot pool and erode node density over time. This is especially relevant to AgentENV because it repeatedly starts and resumes large numbers of short-lived Firecracker environments.

Originally created by @morluto on GitHub (Jul 28, 2026). Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/42 Originally assigned to: @LSX-s-Software on GitHub. ## Problem Two Firecracker startup failure paths can retain resources: 1. The public `FirecrackerSandbox::start()` returns when `start_nowait()` or readiness fails without awaiting `stop()`. 2. Fresh startup applies fallible egress-policy configuration before storing the allocated network slot on the sandbox. Orchestrator-managed launch paths already perform explicit cleanup. This issue is limited to direct/public starts and the network-policy failure window before ownership is recorded. ```text A. Public start transaction start_nowait +-- error --> stop --> return original error `-- ok ----> wait_for_ready +-- error --> stop --> return original error `-- ok ----> state: Running B. Network slot ordering before: allocate -> set policy -> store ownership | `-- error -> manager slot remains allocated after: allocate -> store ownership -> set policy +-- error -> stop/Drop releases slot `-- ok ---> continue boot ``` ## Current behavior Direct callers of `start()` can leave initialized runtime state and daemon-owned ublk resources retained until daemon shutdown. If fresh-start egress configuration fails before `self.network_slot` is assigned, the namespace objects are dropped but the network manager does not receive the slot allocation back. ## Expected behavior - Treat public startup and readiness as one transaction. - Await `stop()` after either operation fails while returning the primary startup error. - Record ownership of an allocated network slot before running fallible configuration. - Leave existing orchestrator cleanup and daemon shutdown behavior unchanged. ## Impact Repeated failure paths can reduce the available ublk or network-slot pool and erode node density over time. This is especially relevant to AgentENV because it repeatedly starts and resumes large numbers of short-lived Firecracker environments.

bsakaguchi closed this issue

2026-10-29 14:18:28 +00:00

bsakaguchi referenced this issue

2026-10-29 14:19:55 +00:00

[PR #19] [MERGED] feat: expose snapshot rootfs image references #121

bsakaguchi referenced this issue

2026-10-29 14:20:04 +00:00

[PR #38] [MERGED] feat: add standalone aenv-snapshot-image export tool #133