[PR #345] [MERGED] feat(overlaybd): add runtime cache limits and mmap-aware eviction #366

New issue

Closed

opened 2026-10-29 14:22:39 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:22:39 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/345
Author: @huang-jl
Created: 10/8/2026
Status: ✅ Merged
Merged: 10/22/2026
Merged by: @yingdi-shan

Base: main ← Head: hjl/cache-mmap-limits

📝 Commits (4)

fb9faa9 feat(overlaybd): allow runtime cache capacity updates
302482e feat(overlaybd): track cache mmap usage
80f7abe feat(overlaybd): evict idle cache on mmap pressure
be159bb feat(overlaybd): add file cache tracing

📊 Changes

6 files changed (+1111 additions, -86 deletions)

View changed files

📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_entry.rs (+45 -3)
📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_pool.rs (+372 -73)
📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs (+14 -9)
📝 storage/overlaybd/src/backend/cache/mod.rs (+1 -0)
📝 storage/overlaybd/src/backend/cache/tests.rs (+567 -0)
📝 storage/util/src/mmap_region.rs (+112 -1)

📄 Description

What

Add runtime-adjustable cache data and mmap capacities.
Track page-aligned mmap usage through MMapRegion::on_unmap, decrementing it only after successful unmapping.
Extend the existing LRU eviction to handle both data and mmap pressure, including idle entries with no cached data.
Default mmap capacity to 32 TiB with a 90% GC watermark. Preserve protection for open entries and in-flight refills.
Use separate, fixed per-pass eviction budgets. Limits remain asynchronous GC thresholds.

Why

Sparse cache files map their full logical size, while disk capacity accounting only counts populated cache blocks. Virtual address usage can therefore grow until mmap fails with ENOMEM, even when the cache is still below its disk limit.

An independent mmap limit allows GC to reclaim idle mappings before address-space pressure becomes critical. Runtime capacity updates allow tuning without restarting the backend.

Design and behavior changes

Eviction starts when either cached data usage or mmap usage exceeds its watermark. mmap pressure can therefore trigger reclamation even when disk usage remains below its limit.
Both pressures reclaim idle entries, including entries with no cached data. Open entries and entries with in-flight refills remain protected.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/345 **Author:** [@huang-jl](https://github.com/huang-jl) **Created:** 10/8/2026 **Status:** ✅ Merged **Merged:** 10/22/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `hjl/cache-mmap-limits` --- ### 📝 Commits (4) - [`fb9faa9`](https://github.com/kvcache-ai/AgentENV/commit/fb9faa92e3299378f69acbc53c0155d51d5ad379) feat(overlaybd): allow runtime cache capacity updates - [`302482e`](https://github.com/kvcache-ai/AgentENV/commit/302482e14cf2f0080b0a11df74dba419ce7b3ef5) feat(overlaybd): track cache mmap usage - [`80f7abe`](https://github.com/kvcache-ai/AgentENV/commit/80f7abeb7ad6a7784d8efbadd7a2fa0f36fc6192) feat(overlaybd): evict idle cache on mmap pressure - [`be159bb`](https://github.com/kvcache-ai/AgentENV/commit/be159bbdbbd5e8674e2d08d0b6c9adcb5bbf91cf) feat(overlaybd): add file cache tracing ### 📊 Changes **6 files changed** (+1111 additions, -86 deletions) <details> <summary>View changed files</summary> 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_entry.rs` (+45 -3) 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_pool.rs` (+372 -73) 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs` (+14 -9) 📝 `storage/overlaybd/src/backend/cache/mod.rs` (+1 -0) 📝 `storage/overlaybd/src/backend/cache/tests.rs` (+567 -0) 📝 `storage/util/src/mmap_region.rs` (+112 -1) </details> ### 📄 Description ## What - Add runtime-adjustable cache data and mmap capacities. - Track page-aligned mmap usage through `MMapRegion::on_unmap`, decrementing it only after successful unmapping. - Extend the existing LRU eviction to handle both data and mmap pressure, including idle entries with no cached data. - Default mmap capacity to 32 TiB with a 90% GC watermark. Preserve protection for open entries and in-flight refills. - Use separate, fixed per-pass eviction budgets. Limits remain asynchronous GC thresholds. ## Why Sparse cache files map their full logical size, while disk capacity accounting only counts populated cache blocks. Virtual address usage can therefore grow until `mmap` fails with `ENOMEM`, even when the cache is still below its disk limit. An independent mmap limit allows GC to reclaim idle mappings before address-space pressure becomes critical. Runtime capacity updates allow tuning without restarting the backend. ## Design and behavior changes - Eviction starts when either cached data usage or mmap usage exceeds its watermark. mmap pressure can therefore trigger reclamation even when disk usage remains below its limit. - Both pressures reclaim idle entries, including entries with no cached data. Open entries and entries with in-flight refills remain protected. ## Validation  - [x] `make fmt` - [x] `make clippy` - [ ] `make test-unit` - [ ] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [ ] Documentation updated - [ ] Benchmarks or performance comparison completed --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:22:39 +00:00