kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #1670] [Feature Request]: Will DRAM-CXL-SSD tiered storage be supported in Mooncake #492

New issue

Closed

opened 2026-10-29 21:15:25 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 21:15:25 +00:00

Owner

Originally created by @qiuweit7 on GitHub (Mar 16, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1670

Originally assigned to: @stmatengss on GitHub.

Describe your feature request

DRAM-CXL-SSD Tiered Storage Implementation

Feature Description

This submission aims to implement the DRAM-CXL-SSD tiered storage core capability for Mooncake Store: the design retains hot data in ultra-low-latency DRAM, places warm data in a shareable CXL memory pool, and writes cold data to high-capacity SSD.

We are currently planning to merge this functionality into the community codebase. This Issue is submitted to synchronize the changes and solicit community feedback.

Core Changes Overview

1. Client Initialization with Multi-Protocol Support

Files:

mooncake-store/src/real_client.cpp
mooncake-store/src/client_service.cpp

Changes:

Added DispatchProtocols logic during client creation to implement protocol-to-storage level mapping:
- tcp / rdma → StorageLevel::RAM
- cxl → StorageLevel::CXL
Mount RAM Segment and CXL Segment separately according to storage levels.

2. Mount Implementation

Files:

mooncake-store/src/client_service.cpp
mooncake-transfer-engine/src/transfer_engine_impl.cpp

Changes:

Adjusted transfer_engine memory registration logic (registerLocalMemory) to perform memory registration with the appropriate transport layer based on protocol selection.

3. Level Allocation Strategy

File: mooncake-store/include/allocation_strategy.h

Changes:

Added LevelAllocationStrategy class for tiered storage scenarios
Route allocation requests based on ReplicateConfig.preferred_storage_level:
- StorageLevel::RAM → allocateRAM() method
- StorageLevel::CXL → allocateCXL() method

// Select allocation method based on specified storage level
ReplicateConfig config;
config.preferred_storage_level = StorageLevel::CXL;  // or StorageLevel::RAM

if (config.preferred_storage_level == StorageLevel::RAM) {
    return allocateRAM(...);
} else if (config.preferred_storage_level == StorageLevel::CXL) {
    return allocateCXL(...);
}

4. Protocol-Matched Data Transfer

Files:

mooncake-store/src/transfer_task.cpp
mooncake-transfer-engine/src/multi_transport.cpp

Changes:

Added protocol handling logic to the submitTransfer interface

Select protocol based on replica's storage level:

// Select protocol based on replica storage level
std::string proto = level_protocols_[replica.get_storage_level()];

submitTransfer(requests, proto);

Modified selectTransport to check whether the target segment's protocol list contains the requested proto, thereby obtaining the corresponding transport.

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @qiuweit7 on GitHub (Mar 16, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1670 Originally assigned to: @stmatengss on GitHub. ### Describe your feature request # DRAM-CXL-SSD Tiered Storage Implementation ## Feature Description This submission aims to implement the **DRAM-CXL-SSD tiered storage** core capability for Mooncake Store: the design retains hot data in ultra-low-latency DRAM, places warm data in a shareable CXL memory pool, and writes cold data to high-capacity SSD. We are currently planning to merge this functionality into the community codebase. This Issue is submitted to synchronize the changes and solicit community feedback. ------ ## Core Changes Overview ### 1. Client Initialization with Multi-Protocol Support **Files**: - `mooncake-store/src/real_client.cpp` - `mooncake-store/src/client_service.cpp` **Changes**: - Added `DispatchProtocols` logic during `client` creation to implement protocol-to-storage level mapping: - `tcp` / `rdma` → `StorageLevel::RAM` - `cxl` → `StorageLevel::CXL` - Mount RAM Segment and CXL Segment separately according to storage levels. ------ ### 2. Mount Implementation **Files**: - `mooncake-store/src/client_service.cpp` - `mooncake-transfer-engine/src/transfer_engine_impl.cpp` **Changes**: - Adjusted `transfer_engine` memory registration logic (`registerLocalMemory`) to perform memory registration with the appropriate transport layer based on protocol selection. ------ ### 3. Level Allocation Strategy **File**: `mooncake-store/include/allocation_strategy.h` **Changes**: - Added `LevelAllocationStrategy` class for tiered storage scenarios - Route allocation requests based on `ReplicateConfig.preferred_storage_level`: - `StorageLevel::RAM` → `allocateRAM()` method - `StorageLevel::CXL` → `allocateCXL()` method ```cpp // Select allocation method based on specified storage level ReplicateConfig config; config.preferred_storage_level = StorageLevel::CXL; // or StorageLevel::RAM if (config.preferred_storage_level == StorageLevel::RAM) { return allocateRAM(...); } else if (config.preferred_storage_level == StorageLevel::CXL) { return allocateCXL(...); } ``` ### 4. Protocol-Matched Data Transfer **Files**: - `mooncake-store/src/transfer_task.cpp` - `mooncake-transfer-engine/src/multi_transport.cpp` **Changes**: - Added protocol handling logic to the `submitTransfer` interface - Select protocol based on replica's storage level: ```cpp // Select protocol based on replica storage level std::string proto = level_protocols_[replica.get_storage_level()]; submitTransfer(requests, proto); ``` - Modified `selectTransport` to check whether the target segment's protocol list contains the requested `proto`, thereby obtaining the corresponding `transport`. ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi

2026-10-29 21:15:25 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:15:26 +00:00

Author

Owner

@stmatengss commented on GitHub (Mar 17, 2026):

Good feature. Hope to see related PRs.

@stmatengss commented on GitHub (Mar 17, 2026): Good feature. Hope to see related PRs.

bsakaguchi commented

2026-10-29 21:15:26 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 16, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jun 16, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:15:26 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 24, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jun 24, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.