[GH-ISSUE #320] OverlayBD full-file cache: entries held open by long-lived sandboxes are never evicted, so a node stops caching new templates #84

New issue

Closed

opened 2026-10-29 14:19:22 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:19:22 +00:00

Owner

Originally created by @nblintao on GitHub (Sep 25, 2026).
Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/320

Originally assigned to: @huajq on GitHub.

Affected component

OverlayBD / image resolution

Problem statement

The full-file cache never evicts an entry that is open. A running sandbox keeps its template's memory-snapshot layers (and its rootfs lower layers) open for its whole lifetime, so the layers of every template that has at least one live sandbox stay pinned, however long ago they were last read.

When the pinned layers reach the pool's eviction watermark (90% of cacheSizeGB), eviction has nothing it is allowed to evict, is_full stays set, and every refill is refused with ENOSPC. From then on, any template that is not already cached resumes fully on demand: each read goes to the source for that request only, nothing is kept, and concurrent resumes of the same template do not share fetched blocks. Resumes that touch many pages can then exceed envd.init_timeout_secs, and the create fails.

The failure is a cliff rather than a gradual slowdown: templates that are already cached keep working, every new template degrades at once, and the capacity is held by sandboxes that may no longer be doing anything.

Use case

A node that serves many templates with long-lived sandboxes. For example, agent or evaluation workloads that build one template per task, start a burst of sandboxes from it, and keep each sandbox for the duration of the task (tens of minutes to hours). Sandboxes can also outlive their users when a client exits without deleting them, in which case they run until their timeout.

With memory layers of roughly 0.5–1.5 GiB per template and the default max_size_gb = 100, about 60–90 templates that each have one live sandbox are enough to pin the memory-snapshot cache. We hit this on a bare-metal x86_64 node running about 800 concurrent sandboxes.

Current behavior and workarounds

Code paths, at upstream main 04f8786:

evictable_cache_ids_by_lru only returns entries for which entry_is_busy is false, and entry_is_busy is true whenever open_count > 0. Open entries are skipped regardless of last_access.
publish_pressure_locked keeps is_full set while current_bytes is at or above the watermark.
do_refill_block_generic returns ENOSPC while is_full is set, so try_preadv2_generic falls back to pread_from_source_generic block by block, with no caching and no deduplication between concurrent readers.
The background downloader backs off on ENOSPC while the pool is full, so the layer is never filled either.

What we observed on a node in this state (build based on upstream 187f654, storage/ unmodified):

image-cache/memory-blocks sat at 90 GiB of 100 GiB. Almost all of it belonged to templates whose remaining sandboxes were idle (median CPU below 1%).
Entries created for newly started templates had zero allocated blocks in their data file and no meta.bin, while the entries of templates with running sandboxes were fully populated.
Source bytes read per sandbox create were about 3–4 times those of the same workload earlier, when the pool had room.
32 concurrent creates from one template (4 vCPU, 8 GiB) all failed with sandbox failed to become ready ... timed out waiting for envd after 105 probes in 60001 ms. Earlier the same day, the same template had started 32 sandboxes on the same node in about 1.5 s each.
Deleting the idle sandboxes let the pool evict their entries. New entries were populated again, and 32 concurrent cold creates of another template became ready in about 1.4 s each.

Workarounds are to delete idle sandboxes, or to raise max_size_gb, which only moves the threshold and needs a restart. Neither addresses the pinning.

Desired behavior

With the cache at its watermark, a template that is being started still gets its blocks cached. Capacity held by entries that have not been read recently is reclaimed first, whether or not the entry is open.
Concurrent resumes of the same template fetch each block from the source once, even while the pool is under pressure.
An open entry that loses cached blocks keeps working, and later reads refill it normally.
A reader never observes a block while it is being evicted.

Acceptance: with a small cache (for example max_size_gb = 2) filled by open entries that are no longer read, starting sandboxes from a new template populates that template's cache entry, and later resumes of it are served from the cache.

Proposed approach

For discussion:

Make reads safe against concurrent eviction. CacheEntry::read_block returns a zero-copy Bytes into the entry's mmap without holding refill_eviction_barrier; as we read it, that is why open entries cannot be evicted today. Holding a read guard of the barrier for as long as the returned Bytes lives (for example as part of its owner), and treating a failed try_read as a miss, would make eviction wait for in-flight reads.
Let pressure eviction consider open entries by last_access, using block-level eviction that keeps the entry (CacheEntry::evict_all_blocks) instead of evict_entry, so open files keep refilling afterwards. An idle-age threshold would protect the working set of active sandboxes.
Deduplicate concurrent misses on the same block even when the block cannot be admitted.

Compatibility and operational impact

None of the three changes needs an API, configuration, or on-disk format change. The behavior change is that blocks of idle open entries can be evicted, so a sandbox that touches them again pays a source read. If changing the default is undesirable, (2) could be gated by a setting such as an idle age for open entries, off by default. The rootfs cache (remote-blocks) uses the same backend and behaves the same way.

Alternatives considered

Raising max_size_gb delays the cliff by a constant factor, takes disk space that sandboxes also need, and requires a restart.
Deleting idle sandboxes or shortening sandbox timeouts depends on every client, and legitimate long-lived sandboxes still pin their templates.
PhotonLibOS's C++ FileCachePool evicts open files by truncating them. The Rust cache deliberately does not, presumably because of the zero-copy read path, so (1) looks like a prerequisite either way.

Non-goals

Changing the default cache sizes.
Cross-node placement or P2P distribution.

Contribution

I can help test or review it

Pre-submission checklist

I searched existing issues and discussions and did not find a duplicate.
I described a concrete problem and use case, not only a proposed technology.
I understand that acceptance of the problem does not imply approval of a particular implementation.

Originally created by @nblintao on GitHub (Sep 25, 2026). Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/320 Originally assigned to: @huajq on GitHub. ### Affected component OverlayBD / image resolution ### Problem statement The full-file cache never evicts an entry that is open. A running sandbox keeps its template's memory-snapshot layers (and its rootfs lower layers) open for its whole lifetime, so the layers of every template that has at least one live sandbox stay pinned, however long ago they were last read. When the pinned layers reach the pool's eviction watermark (90% of `cacheSizeGB`), eviction has nothing it is allowed to evict, `is_full` stays set, and every refill is refused with `ENOSPC`. From then on, any template that is not already cached resumes fully on demand: each read goes to the source for that request only, nothing is kept, and concurrent resumes of the same template do not share fetched blocks. Resumes that touch many pages can then exceed `envd.init_timeout_secs`, and the create fails. The failure is a cliff rather than a gradual slowdown: templates that are already cached keep working, every new template degrades at once, and the capacity is held by sandboxes that may no longer be doing anything. ### Use case A node that serves many templates with long-lived sandboxes. For example, agent or evaluation workloads that build one template per task, start a burst of sandboxes from it, and keep each sandbox for the duration of the task (tens of minutes to hours). Sandboxes can also outlive their users when a client exits without deleting them, in which case they run until their timeout. With memory layers of roughly 0.5–1.5 GiB per template and the default `max_size_gb = 100`, about 60–90 templates that each have one live sandbox are enough to pin the memory-snapshot cache. We hit this on a bare-metal x86_64 node running about 800 concurrent sandboxes. ### Current behavior and workarounds Code paths, at upstream main 04f8786: - [`evictable_cache_ids_by_lru`](https://github.com/kvcache-ai/AgentENV/blob/04f87864da78d3fb3c2f96bf6a2b719017edc843/storage/overlaybd/src/backend/cache/full_file_cache/cache_pool.rs#L522-L544) only returns entries for which `entry_is_busy` is false, and `entry_is_busy` is true whenever `open_count > 0`. Open entries are skipped regardless of `last_access`. - `publish_pressure_locked` keeps `is_full` set while `current_bytes` is at or above the watermark. - [`do_refill_block_generic`](https://github.com/kvcache-ai/AgentENV/blob/04f87864da78d3fb3c2f96bf6a2b719017edc843/storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs#L585-L591) returns `ENOSPC` while `is_full` is set, so `try_preadv2_generic` falls back to `pread_from_source_generic` block by block, with no caching and no deduplication between concurrent readers. - The background downloader backs off on `ENOSPC` while the pool is full, so the layer is never filled either. What we observed on a node in this state (build based on upstream 187f654, `storage/` unmodified): - `image-cache/memory-blocks` sat at 90 GiB of 100 GiB. Almost all of it belonged to templates whose remaining sandboxes were idle (median CPU below 1%). - Entries created for newly started templates had zero allocated blocks in their `data` file and no `meta.bin`, while the entries of templates with running sandboxes were fully populated. - Source bytes read per sandbox create were about 3–4 times those of the same workload earlier, when the pool had room. - 32 concurrent creates from one template (4 vCPU, 8 GiB) all failed with `sandbox failed to become ready ... timed out waiting for envd after 105 probes in 60001 ms`. Earlier the same day, the same template had started 32 sandboxes on the same node in about 1.5 s each. - Deleting the idle sandboxes let the pool evict their entries. New entries were populated again, and 32 concurrent cold creates of another template became ready in about 1.4 s each. Workarounds are to delete idle sandboxes, or to raise `max_size_gb`, which only moves the threshold and needs a restart. Neither addresses the pinning. ### Desired behavior - With the cache at its watermark, a template that is being started still gets its blocks cached. Capacity held by entries that have not been read recently is reclaimed first, whether or not the entry is open. - Concurrent resumes of the same template fetch each block from the source once, even while the pool is under pressure. - An open entry that loses cached blocks keeps working, and later reads refill it normally. - A reader never observes a block while it is being evicted. Acceptance: with a small cache (for example `max_size_gb = 2`) filled by open entries that are no longer read, starting sandboxes from a new template populates that template's cache entry, and later resumes of it are served from the cache. ### Proposed approach For discussion: 1. Make reads safe against concurrent eviction. [`CacheEntry::read_block`](https://github.com/kvcache-ai/AgentENV/blob/04f87864da78d3fb3c2f96bf6a2b719017edc843/storage/overlaybd/src/backend/cache/full_file_cache/cache_entry.rs#L329-L351) returns a zero-copy `Bytes` into the entry's mmap without holding `refill_eviction_barrier`; as we read it, that is why open entries cannot be evicted today. Holding a read guard of the barrier for as long as the returned `Bytes` lives (for example as part of its owner), and treating a failed `try_read` as a miss, would make eviction wait for in-flight reads. 2. Let pressure eviction consider open entries by `last_access`, using block-level eviction that keeps the entry (`CacheEntry::evict_all_blocks`) instead of `evict_entry`, so open files keep refilling afterwards. An idle-age threshold would protect the working set of active sandboxes. 3. Deduplicate concurrent misses on the same block even when the block cannot be admitted. ### Compatibility and operational impact None of the three changes needs an API, configuration, or on-disk format change. The behavior change is that blocks of idle open entries can be evicted, so a sandbox that touches them again pays a source read. If changing the default is undesirable, (2) could be gated by a setting such as an idle age for open entries, off by default. The rootfs cache (`remote-blocks`) uses the same backend and behaves the same way. ### Alternatives considered - Raising `max_size_gb` delays the cliff by a constant factor, takes disk space that sandboxes also need, and requires a restart. - Deleting idle sandboxes or shortening sandbox timeouts depends on every client, and legitimate long-lived sandboxes still pin their templates. - PhotonLibOS's C++ `FileCachePool` evicts open files by truncating them. The Rust cache deliberately does not, presumably because of the zero-copy read path, so (1) looks like a prerequisite either way. ### Non-goals - Changing the default cache sizes. - Cross-node placement or P2P distribution. ### Contribution I can help test or review it ### Pre-submission checklist - [x] I searched existing issues and discussions and did not find a duplicate. - [x] I described a concrete problem and use case, not only a proposed technology. - [x] I understand that acceptance of the problem does not imply approval of a particular implementation.

bsakaguchi closed this issue

2026-10-29 14:19:22 +00:00

bsakaguchi referenced this issue

2026-10-29 14:20:33 +00:00

[PR #84] [MERGED] test(image): cover ACR referrer discovery at the regctl boundary #167