[PR #437] [MERGED] [Store] Enable Client SSD Offload And Storage Persistence #1564

New issue

Closed

opened 2026-10-29 21:28:29 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:28:29 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/437
Author: @SgtPepperr
Created: 6/2/2025
Status: ✅ Merged
Merged: 7/2/2025
Merged by: @stmatengss

Base: main ← Head: client-ssd-persistence

📝 Commits (10+)

951c093 enable client ssd offload and storage persistence
159442d add storage_root_path in all tests setup() initialization and add related description in doc
b02d7bd clean up headers and improve code readability - Added consistent Doxygen-style comments to all header files - Removed redundant code and outdated comments - Optimized function execution logic in LocalFile
9e25771 Revert "add storage_root_path in all tests setup() initialization and add related description in doc"
4d5f4dc Restore the high-level API to its original state and modify it to introduce the storage path through environment variables.
240e56a add local_file_test and thread_pool_test
cd275dc feat(client_ssd_offload): implement async writes and fix locking bugs
5443d6d add support for remove , remove_all , isexist interface etc.
d78414e feat(kvcache): implement cluster isolation with session IDs
6e93a21 edit two parameters client get, add persisitence path in client rather than store_py.cpp

📊 Changes

35 files changed (+2297 additions, -132 deletions)

View changed files

📝 .github/workflows/ci.yml (+29 -0)
📝 mooncake-integration/store/store_py.cpp (+96 -30)
📝 mooncake-integration/store/store_py.h (+3 -0)
📝 mooncake-store/include/client.h (+27 -9)
📝 mooncake-store/include/ha_helper.h (+4 -1)
➕ mooncake-store/include/local_file.h (+97 -0)
📝 mooncake-store/include/master_client.h (+6 -0)
📝 mooncake-store/include/master_service.h (+10 -1)
📝 mooncake-store/include/rpc_service.h (+33 -9)
➕ mooncake-store/include/storage_backend.h (+152 -0)
➕ mooncake-store/include/thread_pool.h (+69 -0)
📝 mooncake-store/include/transfer_task.h (+93 -3)
📝 mooncake-store/include/types.h (+65 -4)
📝 mooncake-store/src/CMakeLists.txt (+3 -0)
📝 mooncake-store/src/client.cpp (+163 -37)
📝 mooncake-store/src/ha_helper.cpp (+5 -3)
➕ mooncake-store/src/local_file.cpp (+224 -0)
📝 mooncake-store/src/master.cpp (+7 -3)
📝 mooncake-store/src/master_client.cpp (+34 -0)
📝 mooncake-store/src/master_service.cpp (+13 -2)

...and 15 more files

📄 Description

I am currently working on the KVcache SSD offload feature for the client side in the Mooncake project. The primary implementation approach involves

Initialize: specifying the storage path for files during the initialization phase. If the storage path is invalid, subsequent persistence operations will fail.
Put: During the put stage, for each successful put request, the client writes the data to the designated storage path using POSIX file write operations.
Get: In the get stage, for each failed get request, the system attempts to locate the corresponding KVcache from the local storage path. If found, the data is read from the file and returned correctly.

The persistence functionality is enabled through a precompiled parameter USE_CLIENT_PERSISTENCE.

[TODO] Currently, file write operations are still performed synchronously, but the related thread pool asynchronous interfaces have been implemented and will be modified to asynchronous operations in a subsequent commit after debugging is completed.

The current code represents the initial implementation of KVcache persistence on the client side. Future work will focus on refining the existing implementation, including adding comments, removing redundant code, improving readability and extensibility, adding test code, and updating documentation.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/437 **Author:** [@SgtPepperr](https://github.com/SgtPepperr) **Created:** 6/2/2025 **Status:** ✅ Merged **Merged:** 7/2/2025 **Merged by:** [@stmatengss](https://github.com/stmatengss) **Base:** `main` ← **Head:** `client-ssd-persistence` --- ### 📝 Commits (10+) - [`951c093`](https://github.com/kvcache-ai/Mooncake/commit/951c093c50621a0c4935a28a67faa091e104d912) enable client ssd offload and storage persistence - [`159442d`](https://github.com/kvcache-ai/Mooncake/commit/159442d44ed5d139968cfc0e697d8d89376688ca) add storage_root_path in all tests setup() initialization and add related description in doc - [`b02d7bd`](https://github.com/kvcache-ai/Mooncake/commit/b02d7bd7e70d868c13cb44905cd23254ecd46a0d) clean up headers and improve code readability - Added consistent Doxygen-style comments to all header files - Removed redundant code and outdated comments - Optimized function execution logic in LocalFile - [`9e25771`](https://github.com/kvcache-ai/Mooncake/commit/9e257710467e3cff5445af7f189943e76adfc821) Revert "add storage_root_path in all tests setup() initialization and add related description in doc" - [`4d5f4dc`](https://github.com/kvcache-ai/Mooncake/commit/4d5f4dc6ab4dc88f51be484346fd38b4cdb0a54a) Restore the high-level API to its original state and modify it to introduce the storage path through environment variables. - [`240e56a`](https://github.com/kvcache-ai/Mooncake/commit/240e56a273c29e221868574810583e93edd7f700) add local_file_test and thread_pool_test - [`cd275dc`](https://github.com/kvcache-ai/Mooncake/commit/cd275dca0888ba63891858becb33e459bb7dacd0) feat(client_ssd_offload): implement async writes and fix locking bugs - [`5443d6d`](https://github.com/kvcache-ai/Mooncake/commit/5443d6d63d3aaf69461330140aa48978f9e976b1) add support for remove , remove_all , isexist interface etc. - [`d78414e`](https://github.com/kvcache-ai/Mooncake/commit/d78414ec50c413f4c4e0e25a22ed90f609912b55) feat(kvcache): implement cluster isolation with session IDs - [`6e93a21`](https://github.com/kvcache-ai/Mooncake/commit/6e93a21c505a92e628776bdbde2e9ab8636628fd) edit two parameters client get, add persisitence path in client rather than store_py.cpp ### 📊 Changes **35 files changed** (+2297 additions, -132 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/ci.yml` (+29 -0) 📝 `mooncake-integration/store/store_py.cpp` (+96 -30) 📝 `mooncake-integration/store/store_py.h` (+3 -0) 📝 `mooncake-store/include/client.h` (+27 -9) 📝 `mooncake-store/include/ha_helper.h` (+4 -1) ➕ `mooncake-store/include/local_file.h` (+97 -0) 📝 `mooncake-store/include/master_client.h` (+6 -0) 📝 `mooncake-store/include/master_service.h` (+10 -1) 📝 `mooncake-store/include/rpc_service.h` (+33 -9) ➕ `mooncake-store/include/storage_backend.h` (+152 -0) ➕ `mooncake-store/include/thread_pool.h` (+69 -0) 📝 `mooncake-store/include/transfer_task.h` (+93 -3) 📝 `mooncake-store/include/types.h` (+65 -4) 📝 `mooncake-store/src/CMakeLists.txt` (+3 -0) 📝 `mooncake-store/src/client.cpp` (+163 -37) 📝 `mooncake-store/src/ha_helper.cpp` (+5 -3) ➕ `mooncake-store/src/local_file.cpp` (+224 -0) 📝 `mooncake-store/src/master.cpp` (+7 -3) 📝 `mooncake-store/src/master_client.cpp` (+34 -0) 📝 `mooncake-store/src/master_service.cpp` (+13 -2) _...and 15 more files_ </details> ### 📄 Description I am currently working on the KVcache SSD offload feature for the client side in the Mooncake project. The primary implementation approach involves * Initialize: specifying the storage path for files during the initialization phase. If the storage path is invalid, subsequent persistence operations will fail. * Put: During the put stage, for each successful put request, the client writes the data to the designated storage path using POSIX file write operations. * Get: In the get stage, for each failed get request, the system attempts to locate the corresponding KVcache from the local storage path. If found, the data is read from the file and returned correctly. The persistence functionality is enabled through a precompiled parameter `USE_CLIENT_PERSISTENCE`. [TODO] Currently, file write operations are still performed synchronously, but the related thread pool asynchronous interfaces have been implemented and will be modified to asynchronous operations in a subsequent commit after debugging is completed. The current code represents the initial implementation of KVcache persistence on the client side. Future work will focus on refining the existing implementation, including adding comments, removing redundant code, improving readability and extensibility, adding test code, and updating documentation. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:28:29 +00:00