[PR #223] [MERGED] feat(overlaybd): port the crate to macOS behind a platform sys module #265

New issue

Closed

opened 2026-10-29 14:21:37 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:21:37 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/223
Author: @huang-jl
Created: 8/27/2026
Status: ✅ Merged
Merged: 8/31/2026
Merged by: @yingdi-shan

Base: main ← Head: hjl/overlaybd-macos-sys

📝 Commits (3)

228b4d9 refactor(overlaybd): abstract platform syscalls into a sys module
0c8eb32 feat(overlaybd): reject the sparse upper layout on macOS
78165b3 fix(overlaybd): reject unaligned extents when scanning a sparse source

📊 Changes

23 files changed (+1958 additions, -281 deletions)

View changed files

📝 storage/overlaybd/src/backend/cache/cached_fs.rs (+6 -76)
📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_entry.rs (+35 -28)
📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_pool.rs (+7 -22)
📝 storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs (+20 -3)
📝 storage/overlaybd/src/backend/cache/mod.rs (+4 -1)
📝 storage/overlaybd/src/backend/cache/tests.rs (+21 -30)
📝 storage/overlaybd/src/backend/local.rs (+56 -114)
📝 storage/overlaybd/src/dense_export.rs (+4 -1)
📝 storage/overlaybd/src/image/image_file.rs (+12 -0)
📝 storage/overlaybd/src/lib.rs (+1 -0)
📝 storage/overlaybd/src/lsmt/file/helper.rs (+44 -0)
📝 storage/overlaybd/src/lsmt/file/readwrite.rs (+17 -1)
📝 storage/overlaybd/src/lsmt/file/tests.rs (+117 -5)
📝 storage/overlaybd/src/lsmt/file/types.rs (+23 -0)
➕ storage/overlaybd/src/sys/fs_space.rs (+116 -0)
➕ storage/overlaybd/src/sys/mod.rs (+92 -0)
➕ storage/overlaybd/src/sys/open_flags.rs (+87 -0)
➕ storage/overlaybd/src/sys/page_cache.rs (+57 -0)
➕ storage/overlaybd/src/sys/release_space.rs (+287 -0)
➕ storage/overlaybd/src/sys/reserve_space.rs (+190 -0)

...and 3 more files

📄 Description

What

Makes overlaybd build and pass its tests on macOS (arm64). The starting point was 25 compile errors from a cargo check on a real macOS machine, every one of them a bare libc::/nix:: call whose ABI or availability differs on Darwin.

This adds storage/overlaybd/src/sys/, split by capability rather than by platform, each file owns one portable signature plus its own per-OS imp arms. The capabilities are extended attributes (8 entry points), space release, space reservation, page-cache eviction, filesystem capacity, the direct-IO open flag, and sparse-extent trustworthiness.

Two further commits reject the sparse upper layout on macOS at all three entry points that create or open an upper, and turn a pre-existing platform-independent hazard — a source file whose size is not a multiple of 512 silently losing its tail block from the index — into an explicit error.

Why

Platform detail and overlaybd's own logic were interleaved. Syscall specifics sat inline in the cache, the LSMT layer, and the local-file backend, and the same two-phase xattr size query existed as two separately written, subtly different copies. Adding a second platform on those terms would have meant scattering cfg(target_os) through overlaybd code.

Design and behavior changes

No changes for users.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/223 **Author:** [@huang-jl](https://github.com/huang-jl) **Created:** 8/27/2026 **Status:** ✅ Merged **Merged:** 8/31/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `hjl/overlaybd-macos-sys` --- ### 📝 Commits (3) - [`228b4d9`](https://github.com/kvcache-ai/AgentENV/commit/228b4d9460dd0e40ca9cd628391bd8e2f132a476) refactor(overlaybd): abstract platform syscalls into a sys module - [`0c8eb32`](https://github.com/kvcache-ai/AgentENV/commit/0c8eb320cd45043957c4ed6df069be69be79e7e7) feat(overlaybd): reject the sparse upper layout on macOS - [`78165b3`](https://github.com/kvcache-ai/AgentENV/commit/78165b3a0a34cafcf89221e84c8c7edc60038688) fix(overlaybd): reject unaligned extents when scanning a sparse source ### 📊 Changes **23 files changed** (+1958 additions, -281 deletions) <details> <summary>View changed files</summary> 📝 `storage/overlaybd/src/backend/cache/cached_fs.rs` (+6 -76) 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_entry.rs` (+35 -28) 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_pool.rs` (+7 -22) 📝 `storage/overlaybd/src/backend/cache/full_file_cache/cache_store.rs` (+20 -3) 📝 `storage/overlaybd/src/backend/cache/mod.rs` (+4 -1) 📝 `storage/overlaybd/src/backend/cache/tests.rs` (+21 -30) 📝 `storage/overlaybd/src/backend/local.rs` (+56 -114) 📝 `storage/overlaybd/src/dense_export.rs` (+4 -1) 📝 `storage/overlaybd/src/image/image_file.rs` (+12 -0) 📝 `storage/overlaybd/src/lib.rs` (+1 -0) 📝 `storage/overlaybd/src/lsmt/file/helper.rs` (+44 -0) 📝 `storage/overlaybd/src/lsmt/file/readwrite.rs` (+17 -1) 📝 `storage/overlaybd/src/lsmt/file/tests.rs` (+117 -5) 📝 `storage/overlaybd/src/lsmt/file/types.rs` (+23 -0) ➕ `storage/overlaybd/src/sys/fs_space.rs` (+116 -0) ➕ `storage/overlaybd/src/sys/mod.rs` (+92 -0) ➕ `storage/overlaybd/src/sys/open_flags.rs` (+87 -0) ➕ `storage/overlaybd/src/sys/page_cache.rs` (+57 -0) ➕ `storage/overlaybd/src/sys/release_space.rs` (+287 -0) ➕ `storage/overlaybd/src/sys/reserve_space.rs` (+190 -0) _...and 3 more files_ </details> ### 📄 Description ## What Makes overlaybd build and pass its tests on macOS (arm64). The starting point was 25 compile errors from a cargo check on a real macOS machine, every one of them a bare libc::/nix:: call whose ABI or availability differs on Darwin. This adds storage/overlaybd/src/sys/, split by capability rather than by platform, each file owns one portable signature plus its own per-OS imp arms. The capabilities are extended attributes (8 entry points), space release, space reservation, page-cache eviction, filesystem capacity, the direct-IO open flag, and sparse-extent trustworthiness. Two further commits reject the sparse upper layout on macOS at all three entry points that create or open an upper, and turn a pre-existing platform-independent hazard — a source file whose size is not a multiple of 512 silently losing its tail block from the index — into an explicit error. ## Why Platform detail and overlaybd's own logic were interleaved. Syscall specifics sat inline in the cache, the LSMT layer, and the local-file backend, and the same two-phase xattr size query existed as two separately written, subtly different copies. Adding a second platform on those terms would have meant scattering cfg(target_os) through overlaybd code. ## Design and behavior changes No changes for users. ## Validation - [x] `make fmt` - [x] `make clippy` - [x] `make test-unit` - [ ] Relevant Rust integration tests - [ ] `make -C services test` (required when `services/` changes) - [ ] Generated clients/server regenerated with the documented `make` target - [ ] Documentation updated - [ ] Benchmarks or performance comparison completed --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:21:37 +00:00