[GH-ISSUE #46] feat: support per-sandbox disk I/O rate limiting via Firecracker TokenBucket #21

New issue

Closed

opened 2026-10-29 14:18:28 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:18:28 +00:00

Owner

Originally created by @rogeroger-yu on GitHub (Jul 28, 2026).
Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/46

Summary

Add configurable per-sandbox disk I/O rate limiting using Firecracker's built-in TokenBucket rate limiter. This provides hard bandwidth (bytes/sec) and IOPS caps for each sandbox's root disk, ensuring no single sandbox can monopolize host disk I/O.

Motivation

Currently all sandboxes on a node share disk I/O resources without any isolation. A single sandbox performing heavy disk I/O can starve other sandboxes, causing performance degradation and unpredictable latency.

Proposed Solution

Wire Firecracker's per-drive RateLimiter into the sandbox lifecycle:

Configuration: New [machine.disk_rate_limit] section in config/default.toml
- bandwidth_bytes_per_sec (default: 100 MB/s)
- bandwidth_burst_bytes (default: 10 MB)
- iops (default: 3000)
- iops_burst (default: 500)
- refill_time_ms (default: 1000ms)
Fresh boot path: Rate limiter applied at add_drive() time
Resume path: Rate limiter applied via PATCH /drives/{drive_id} after VM resume

Test Results

Scenario	Baseline	Rate Limited	Reduction
Sequential Write (256 MB)	~1.8 GB/s	~12-28 MB/s	93-99%
Sequential Read (256 MB)	~3.3 GB/s	~12.5 MB/s	99%

Originally created by @rogeroger-yu on GitHub (Jul 28, 2026). Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/46 ## Summary Add configurable per-sandbox disk I/O rate limiting using Firecracker's built-in TokenBucket rate limiter. This provides hard bandwidth (bytes/sec) and IOPS caps for each sandbox's root disk, ensuring no single sandbox can monopolize host disk I/O. ## Motivation Currently all sandboxes on a node share disk I/O resources without any isolation. A single sandbox performing heavy disk I/O can starve other sandboxes, causing performance degradation and unpredictable latency. ## Proposed Solution Wire Firecracker's per-drive `RateLimiter` into the sandbox lifecycle: - **Configuration**: New `[machine.disk_rate_limit]` section in `config/default.toml` - `bandwidth_bytes_per_sec` (default: 100 MB/s) - `bandwidth_burst_bytes` (default: 10 MB) - `iops` (default: 3000) - `iops_burst` (default: 500) - `refill_time_ms` (default: 1000ms) - **Fresh boot path**: Rate limiter applied at `add_drive()` time - **Resume path**: Rate limiter applied via `PATCH /drives/{drive_id}` after VM resume ## Test Results | Scenario | Baseline | Rate Limited | Reduction | |----------|----------|--------------|-----------| | Sequential Write (256 MB) | ~1.8 GB/s | ~12-28 MB/s | 93-99% | | Sequential Read (256 MB) | ~3.3 GB/s | ~12.5 MB/s | 99% |

bsakaguchi closed this issue

2026-10-29 14:18:28 +00:00

bsakaguchi commented

2026-10-29 14:18:29 +00:00

Author

Owner

@yingdi-shan commented on GitHub (Jul 29, 2026):

Despite the design in #47 still needs further discussion, hard per-sandbox I/O limits remain necessary in some scenarios. Firecracker’s TokenBucket provides a clear limit on guest-visible disk throughput and can already serve many practical use cases.

@yingdi-shan commented on GitHub (Jul 29, 2026): Despite the design in #47 still needs further discussion, hard per-sandbox I/O limits remain necessary in some scenarios. Firecracker’s TokenBucket provides a clear limit on guest-visible disk throughput and can already serve many practical use cases.

bsakaguchi referenced this issue

2026-10-29 14:19:56 +00:00

[PR #21] [CLOSED] Update README by removing research citations #122