[PR #201] [MERGED] refactor(overlaybd): decouple LocalFile from io_uring #249

New issue

Closed

opened 2026-10-29 14:21:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:21:28 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/201
Author: @huang-jl
Created: 8/21/2026
Status: ✅ Merged
Merged: 8/24/2026
Merged by: @yingdi-shan

Base: main ← Head: hjl/overlaybd-portable-local-io

📝 Commits (3)

df52096 refactor(overlaybd): decouple local files from io_uring
d779bec refactor(overlaybd): gate io_uring fast path
06e1689 fix(overlaybd): offload unbounded-latency LocalFile syscalls

📊 Changes

41 files changed (+671 additions, -957 deletions)

View changed files

📝 Cargo.lock (+0 -1)
📝 crates/e2e-tests/tests/snapshot_oss_e2e_test.rs (+3 -5)
📝 src/sandbox/firecracker/overlaybd_snapshot.rs (+1 -4)
📝 src/sandbox/ublk/overlaybd.rs (+9 -27)
📝 src/snapshot/p2p.rs (+3 -12)
📝 src/snapshot/repository/backends/common/acr/source_image.rs (+1 -6)
📝 src/snapshot/repository/backends/common/mod.rs (+6 -11)
📝 src/snapshot/repository/backends/posixfs/artifacts.rs (+5 -20)
📝 storage/overlaybd/Cargo.toml (+1 -1)
📝 storage/overlaybd/src/backend/cache/cached_fs.rs (+10 -8)
📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs (+16 -6)
📝 storage/overlaybd/src/backend/cache/tests.rs (+10 -31)
📝 storage/overlaybd/src/backend/local.rs (+351 -188)
📝 storage/overlaybd/src/backend/registryfs_v2.rs (+2 -6)
📝 storage/overlaybd/src/backend/switch.rs (+18 -24)
📝 storage/overlaybd/src/backend/tar.rs (+7 -1)
📝 storage/overlaybd/src/compression/zfile.rs (+11 -13)
📝 storage/overlaybd/src/config.rs (+2 -3)
📝 storage/overlaybd/src/dense_export.rs (+3 -15)
📝 storage/overlaybd/src/ext4_stat.rs (+1 -2)

...and 21 more files

📄 Description

What

Make OverlayBD LocalFile usable on Unix platforms without requiring Linux io_uring, while preserving the existing io_uring fast path for ublk consumers.

Why

Allow other platform (e.g., macOS) to use overlaybd crate.

Design and behavior changes

Remove IoRingHandle from LocalFile constructors.
Use synchronous positional I/O for ordinary VirtualFile operations.
Pass io_uring explicitly through the context-aware I/O methods.
Make io_uring support optional in storage-util and overlaybd.
Explicitly enable the io-uring feature for uvm-ublk and uvm-ublk-daemon.
Remove the ImageService-owned ring pool and transient io_uring fallback workers.
Update affected callers and tests for the new constructors and I/O model.

Compatibility and operations

Public API or generated protocol: N/A.
Configuration or defaults: The nr_io_rings configuration is useless.
Snapshot manifest, artifact layout, or storage format: N/A.
Upgrade and rollback: N/A.
Host requirements, permissions, ports, or dependencies: N/A.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Compared:

Baseline: 836711761f20612c9d322f0cdcb7412314426e2b

The full ublk_overlaybd Criterion benchmark was run on the same NVMe host, pinned to one CPU, with 50 samples across log, hybrid, and sparse upper modes.

No performance regression was detected:

Read throughput changes ranged from approximately -0.75% to +1.13%.
Log-structured 4 KiB writes improved by approximately 1.5%–2.0%.
Hybrid 4 KiB writes improved by approximately 2.2%–3.2%.
Sparse write changes remained within the benchmark noise threshold.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/201 **Author:** [@huang-jl](https://github.com/huang-jl) **Created:** 8/21/2026 **Status:** ✅ Merged **Merged:** 8/24/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `hjl/overlaybd-portable-local-io` --- ### 📝 Commits (3) - [`df52096`](https://github.com/kvcache-ai/AgentENV/commit/df5209684d218d293ce2878bf30680e96cca5365) refactor(overlaybd): decouple local files from io_uring - [`d779bec`](https://github.com/kvcache-ai/AgentENV/commit/d779becb64dcaa23835ae17f1c9899676ed3f887) refactor(overlaybd): gate io_uring fast path - [`06e1689`](https://github.com/kvcache-ai/AgentENV/commit/06e1689b82c474764c840757ef9a4497d97e8a28) fix(overlaybd): offload unbounded-latency LocalFile syscalls ### 📊 Changes **41 files changed** (+671 additions, -957 deletions) <details> <summary>View changed files</summary> 📝 `Cargo.lock` (+0 -1) 📝 `crates/e2e-tests/tests/snapshot_oss_e2e_test.rs` (+3 -5) 📝 `src/sandbox/firecracker/overlaybd_snapshot.rs` (+1 -4) 📝 `src/sandbox/ublk/overlaybd.rs` (+9 -27) 📝 `src/snapshot/p2p.rs` (+3 -12) 📝 `src/snapshot/repository/backends/common/acr/source_image.rs` (+1 -6) 📝 `src/snapshot/repository/backends/common/mod.rs` (+6 -11) 📝 `src/snapshot/repository/backends/posixfs/artifacts.rs` (+5 -20) 📝 `storage/overlaybd/Cargo.toml` (+1 -1) 📝 `storage/overlaybd/src/backend/cache/cached_fs.rs` (+10 -8) 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs` (+16 -6) 📝 `storage/overlaybd/src/backend/cache/tests.rs` (+10 -31) 📝 `storage/overlaybd/src/backend/local.rs` (+351 -188) 📝 `storage/overlaybd/src/backend/registryfs_v2.rs` (+2 -6) 📝 `storage/overlaybd/src/backend/switch.rs` (+18 -24) 📝 `storage/overlaybd/src/backend/tar.rs` (+7 -1) 📝 `storage/overlaybd/src/compression/zfile.rs` (+11 -13) 📝 `storage/overlaybd/src/config.rs` (+2 -3) 📝 `storage/overlaybd/src/dense_export.rs` (+3 -15) 📝 `storage/overlaybd/src/ext4_stat.rs` (+1 -2) _...and 21 more files_ </details> ### 📄 Description ## What Make OverlayBD `LocalFile` usable on Unix platforms without requiring Linux io_uring, while preserving the existing io_uring fast path for ublk consumers. ## Why Allow other platform (e.g., macOS) to use overlaybd crate. ## Design and behavior changes - Remove `IoRingHandle` from `LocalFile` constructors. - Use synchronous positional I/O for ordinary `VirtualFile` operations. - Pass io_uring explicitly through the context-aware I/O methods. - Make io_uring support optional in `storage-util` and `overlaybd`. - Explicitly enable the `io-uring` feature for `uvm-ublk` and `uvm-ublk-daemon`. - Remove the ImageService-owned ring pool and transient io_uring fallback workers. - Update affected callers and tests for the new constructors and I/O model. ## Compatibility and operations  - Public API or generated protocol: N/A. - Configuration or defaults: The `nr_io_rings` configuration is useless. - Snapshot manifest, artifact layout, or storage format: N/A. - Upgrade and rollback: N/A. - Host requirements, permissions, ports, or dependencies: N/A. ## Validation  - [x] `make fmt` - [x] `make clippy` - [x] `make test-unit` - [x] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [ ] Documentation updated - [x] Benchmarks or performance comparison completed Compared: - Baseline: `836711761f20612c9d322f0cdcb7412314426e2b` The full `ublk_overlaybd` Criterion benchmark was run on the same NVMe host, pinned to one CPU, with 50 samples across log, hybrid, and sparse upper modes. No performance regression was detected: - Read throughput changes ranged from approximately `-0.75%` to `+1.13%`. - Log-structured 4 KiB writes improved by approximately `1.5%–2.0%`. - Hybrid 4 KiB writes improved by approximately `2.2%–3.2%`. - Sparse write changes remained within the benchmark noise threshold. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:21:28 +00:00