[GH-ISSUE #943] [Bug]: Will the SGLang service, acting as a client, automatically reload the keys it previously stored to SSD back into the master server after it crashes and is restarted? #303

New issue

Closed

opened 2026-10-29 21:12:19 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 21:12:19 +00:00

Owner

Originally created by @jieyang-su on GitHub (Oct 21, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/943

Bug Report

SGLang prefill adds this config that MOONCAKE_STORAGE_ROOT_DIR=/data/Mooncake_storage/

When I shut down the SGLang service for the first time, Mooncake master deregisters the corresponding client.
Since SSD store is enabled, I expected the keys that were flushed to disk to be reloaded into memory when the client restarts, but my test shows they are not recovered.
Are there any extra configuration flags or APIs that must be set to force the master (or the re-registered client) to re-populate the cache from SSD on restart?

du -sh /data/Mooncake_storage/ 2.5G /data/Mooncake_storage/

I1021 06:05:23.408679 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 0.00 B | Keys: 0 (soft-pinned: 0) | Clients: 0 | Requests (Success/Total): Put=80/80, Get=40/40, Exist=40/40, Del=40/40, DelAll=0/0, Ping=43872/43872, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:05:33.408847 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 0.00 B | Keys: 0 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=80/80, Get=40/40, Exist=40/40, Del=40/40, DelAll=0/0, Ping=43952/43952, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:05:43.409021 11605 rpc_service.cpp:48] Master Metrics: Storage: 32.00 KB / 400.00 GB (0.0%) | Keys: 8 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=96/96, Get=48/48, Exist=48/48, Del=40/40, DelAll=0/0, Ping=44032/44032, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:05:53.409194 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 400.00 GB (0.0%) | Keys: 0 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=96/96, Get=48/48, Exist=48/48, Del=48/48, DelAll=0/0, Ping=44112/44112, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:06:03.409366 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 400.00 GB (0.0%) | Keys: 0 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=96/96, Get=48/48, Exist=48/48, Del=48/48, DelAll=0/0, Ping=44192/44192, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @jieyang-su on GitHub (Oct 21, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/943 ### Bug Report <img width="1406" height="368" alt="Image" src="https://github.com/user-attachments/assets/69a4289d-2130-4d74-84cc-27fda7073e05" /> SGLang prefill adds this config that MOONCAKE_STORAGE_ROOT_DIR=/data/Mooncake_storage/ When I shut down the SGLang service for the first time, Mooncake master deregisters the corresponding client. Since SSD store is enabled, I expected the keys that were flushed to disk to be reloaded into memory when the client restarts, but my test shows they are not recovered. Are there any extra configuration flags or APIs that must be set to force the master (or the re-registered client) to re-populate the cache from SSD on restart? > du -sh /data/Mooncake_storage/ **2.5G /data/Mooncake_storage/** `I1021 06:05:23.408679 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 0.00 B | Keys: 0 (soft-pinned: 0) | Clients: 0 | Requests (Success/Total): Put=80/80, Get=40/40, Exist=40/40, Del=40/40, DelAll=0/0, Ping=43872/43872, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:05:33.408847 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 0.00 B | Keys: 0 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=80/80, Get=40/40, Exist=40/40, Del=40/40, DelAll=0/0, Ping=43952/43952, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:05:43.409021 11605 rpc_service.cpp:48] Master Metrics: Storage: 32.00 KB / 400.00 GB (0.0%) | Keys: 8 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=96/96, Get=48/48, Exist=48/48, Del=40/40, DelAll=0/0, Ping=44032/44032, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:05:53.409194 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 400.00 GB (0.0%) | Keys: 0 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=96/96, Get=48/48, Exist=48/48, Del=48/48, DelAll=0/0, Ping=44112/44112, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B I1021 06:06:03.409366 11605 rpc_service.cpp:48] Master Metrics: Storage: 0.00 B / 400.00 GB (0.0%) | Keys: 0 (soft-pinned: 0) | Clients: 8 | Requests (Success/Total): Put=96/96, Get=48/48, Exist=48/48, Del=48/48, DelAll=0/0, Ping=44192/44192, | Eviction: Success/Attempts=0/0, keys=0, size=0.00 B` ### Before submitting... - [ ] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:12:19 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-29 21:12:20 +00:00

Author

Owner

@SgtPepperr commented on GitHub (Oct 22, 2025):

The current implementation does not support reloading KVCache from SSD back into the memory pool; we may consider adding this capability in the future.

@SgtPepperr commented on GitHub (Oct 22, 2025): The current implementation does not support reloading KVCache from SSD back into the memory pool; we may consider adding this capability in the future.

bsakaguchi commented

2026-10-29 21:12:20 +00:00

Author

Owner

@jieyang-su commented on GitHub (Oct 22, 2025):

The current implementation does not support reloading KVCache from SSD back into the memory pool; we may consider adding this capability in the future.

Thank you for your reply @SgtPepperr , I have another question: when the client restarts and the key no longer exists in RAM but is still present on the SSD, how can I verify that the client will indeed fetch the corresponding key from the SSD?

@jieyang-su commented on GitHub (Oct 22, 2025): > The current implementation does not support reloading KVCache from SSD back into the memory pool; we may consider adding this capability in the future. Thank you for your reply @SgtPepperr , I have another question: when the client restarts and the key no longer exists in RAM but is still present on the SSD, how can I verify that the client will indeed fetch the corresponding key from the SSD?

bsakaguchi commented

2026-10-29 21:12:20 +00:00

Author

Owner

@SgtPepperr commented on GitHub (Oct 22, 2025):

The current implementation does not support reloading KVCache from SSD back into the memory pool; we may consider adding this capability in the future.

Thank you for your reply @SgtPepperr , I have another question: when the client restarts and the key no longer exists in RAM but is still present on the SSD, how can I verify that the client will indeed fetch the corresponding key from the SSD?

你可以尝试使用stress_cluster_benchmark.py的测试程序（开启SSD功能），先运行一次prefill，把key value进行put保存，然后关闭该测试程序（client），再运行一次decode，你会发现所有的kvcache仍能正确完成读取，这里的数据都来自于SSD。

@SgtPepperr commented on GitHub (Oct 22, 2025): > > The current implementation does not support reloading KVCache from SSD back into the memory pool; we may consider adding this capability in the future. > > Thank you for your reply [@SgtPepperr](https://github.com/SgtPepperr) , I have another question: when the client restarts and the key no longer exists in RAM but is still present on the SSD, how can I verify that the client will indeed fetch the corresponding key from the SSD? 你可以尝试使用stress_cluster_benchmark.py的测试程序（开启SSD功能），先运行一次prefill，把key value进行put保存，然后关闭该测试程序（client），再运行一次decode，你会发现所有的kvcache仍能正确完成读取，这里的数据都来自于SSD。

bsakaguchi referenced this issue

2026-10-29 21:27:41 +00:00

[PR #303] [MERGED] fix(transfer_engine): skip metric report for zero bytes transferred #1465