[PR #1031] [MERGED] [Store] feat: Implement a FileStorage component to manage the lifecycle of key-value data #1968

New issue

Closed

opened 2026-10-29 21:31:46 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:31:46 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1031
Author: @zhuxinjie-nz
Created: 11/6/2025
Status: ✅ Merged
Merged: 12/9/2025
Merged by: @ykwd

Base: main ← Head: file-storage-dev

📝 Commits (10+)

d31a3e4 [Store] feat: Implement a FileStorage component to manage the lifecycle of key-value data, including creation, reading, and synchronizing metadata with the master.
37a53af Merge branch 'main' into file-storage-dev
cb1d694 fix(store): Error handling logic in FileStorage class
067d20b fix(store): clang format
623135a fix(store): StorageBackend clang format
cc6325a [Store] feat: Add test cases for FileStorage
4a8db30 fix(store): Improve error handling logic in FileStorage::BatchOffload
3f5983d Merge remote-tracking branch 'origin/main' into file-storage-dev
f34e42c fix(store): Make FileStorage compatible with master's buffer_descriptor metadata changes
44f8da9 fix(store): FileStorage Clang Format

📊 Changes

31 files changed (+2222 additions, -220 deletions)

View changed files

📝 mooncake-store/include/client_metric.h (+4 -1)
📝 mooncake-store/include/client_service.h (+47 -0)
➕ mooncake-store/include/file_storage.h (+179 -0)
📝 mooncake-store/include/master_client.h (+28 -0)
📝 mooncake-store/include/master_config.h (+15 -0)
📝 mooncake-store/include/master_service.h (+41 -0)
📝 mooncake-store/include/pyclient.h (+2 -0)
📝 mooncake-store/include/real_client.h (+1 -1)
📝 mooncake-store/include/replica.h (+66 -4)
📝 mooncake-store/include/rpc_service.h (+10 -0)
📝 mooncake-store/include/segment.h (+62 -0)
📝 mooncake-store/include/storage_backend.h (+22 -28)
📝 mooncake-store/include/types.h (+16 -4)
📝 mooncake-store/src/CMakeLists.txt (+1 -0)
📝 mooncake-store/src/client_service.cpp (+42 -0)
➕ mooncake-store/src/file_storage.cpp (+622 -0)
📝 mooncake-store/src/master.cpp (+9 -0)
📝 mooncake-store/src/master_client.cpp (+54 -0)
📝 mooncake-store/src/master_service.cpp (+144 -0)
📝 mooncake-store/src/real_client.cpp (+13 -2)

...and 11 more files

📄 Description

The FileStorage class will be responsible for managing the entire lifecycle of key-value objects, including:

Storing new objects (Put)
Retrieving existing objects (Get)
Deleting objects
Tracking object metadata (size, location, timestamp)
Notifying the master node of metadata changes (e.g., after offload completion)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1031 **Author:** [@zhuxinjie-nz](https://github.com/zhuxinjie-nz) **Created:** 11/6/2025 **Status:** ✅ Merged **Merged:** 12/9/2025 **Merged by:** [@ykwd](https://github.com/ykwd) **Base:** `main` ← **Head:** `file-storage-dev` --- ### 📝 Commits (10+) - [`d31a3e4`](https://github.com/kvcache-ai/Mooncake/commit/d31a3e45838b3b7c638dfdb400c07de07a79a8c6) [Store] feat: Implement a FileStorage component to manage the lifecycle of key-value data, including creation, reading, and synchronizing metadata with the master. - [`37a53af`](https://github.com/kvcache-ai/Mooncake/commit/37a53af70752168714ef0f2fd839affdcd934fa4) Merge branch 'main' into file-storage-dev - [`cb1d694`](https://github.com/kvcache-ai/Mooncake/commit/cb1d6942325e1ce403de81a66dfc182445e0937e) fix(store): Error handling logic in FileStorage class - [`067d20b`](https://github.com/kvcache-ai/Mooncake/commit/067d20bc43577a752713fc628d575e9b079b6385) fix(store): clang format - [`623135a`](https://github.com/kvcache-ai/Mooncake/commit/623135a8f00e79fce6d9652a7aa4333353bbd309) fix(store): StorageBackend clang format - [`cc6325a`](https://github.com/kvcache-ai/Mooncake/commit/cc6325a77f032b577207539571f138311cd1132a) [Store] feat: Add test cases for FileStorage - [`4a8db30`](https://github.com/kvcache-ai/Mooncake/commit/4a8db301845516eef2816d77e160dc4a9df9e89d) fix(store): Improve error handling logic in FileStorage::BatchOffload - [`3f5983d`](https://github.com/kvcache-ai/Mooncake/commit/3f5983daea3ad4264380ce381d9e405be0e5d42a) Merge remote-tracking branch 'origin/main' into file-storage-dev - [`f34e42c`](https://github.com/kvcache-ai/Mooncake/commit/f34e42c321ea0a263495883731a1782f9488105d) fix(store): Make FileStorage compatible with master's buffer_descriptor metadata changes - [`44f8da9`](https://github.com/kvcache-ai/Mooncake/commit/44f8da97d811c12fc15096e93266574268f4f519) fix(store): FileStorage Clang Format ### 📊 Changes **31 files changed** (+2222 additions, -220 deletions) <details> <summary>View changed files</summary> 📝 `mooncake-store/include/client_metric.h` (+4 -1) 📝 `mooncake-store/include/client_service.h` (+47 -0) ➕ `mooncake-store/include/file_storage.h` (+179 -0) 📝 `mooncake-store/include/master_client.h` (+28 -0) 📝 `mooncake-store/include/master_config.h` (+15 -0) 📝 `mooncake-store/include/master_service.h` (+41 -0) 📝 `mooncake-store/include/pyclient.h` (+2 -0) 📝 `mooncake-store/include/real_client.h` (+1 -1) 📝 `mooncake-store/include/replica.h` (+66 -4) 📝 `mooncake-store/include/rpc_service.h` (+10 -0) 📝 `mooncake-store/include/segment.h` (+62 -0) 📝 `mooncake-store/include/storage_backend.h` (+22 -28) 📝 `mooncake-store/include/types.h` (+16 -4) 📝 `mooncake-store/src/CMakeLists.txt` (+1 -0) 📝 `mooncake-store/src/client_service.cpp` (+42 -0) ➕ `mooncake-store/src/file_storage.cpp` (+622 -0) 📝 `mooncake-store/src/master.cpp` (+9 -0) 📝 `mooncake-store/src/master_client.cpp` (+54 -0) 📝 `mooncake-store/src/master_service.cpp` (+144 -0) 📝 `mooncake-store/src/real_client.cpp` (+13 -2) _...and 11 more files_ </details> ### 📄 Description The FileStorage class will be responsible for managing the entire lifecycle of key-value objects, including: - Storing new objects (Put) - Retrieving existing objects (Get) - Deleting objects - Tracking object metadata (size, location, timestamp) - Notifying the master node of metadata changes (e.g., after offload completion) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:31:46 +00:00