[PR #147] [CLOSED] [Feature] Add shared memory transport and refactor metadata format #1356

New issue

Closed

opened 2026-10-29 21:26:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:26:43 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/147
Author: @alogfans
Created: 3/17/2025
Status: ❌ Closed

Base: main ← Head: transport-refactor-stage-1

📝 Commits (2)

18d08a6 [feature] update transfer engine metadata format and metadata manager
988ba83 [fix] fix bugs not passing metadata tests

📊 Changes

46 files changed (+1078 additions, -892 deletions)

View changed files

📝 README.md (+3 -0)
📝 doc/en/transfer-engine.md (+4 -4)
📝 doc/zh/transfer-engine.md (+4 -4)
📝 mooncake-integration/vllm/vllm_adaptor.cpp (+2 -2)
📝 mooncake-integration/vllm/vllm_adaptor.h (+1 -1)
📝 mooncake-p2p-store/build.sh (+2 -1)
📝 mooncake-store/src/client.cpp (+5 -1)
📝 mooncake-store/tests/distributed_object_store_provider.py (+6 -6)
📝 mooncake-store/tests/stress_cluster_benchmark.py (+16 -7)
📝 mooncake-store/tests/stress_workload_test.cpp (+9 -7)
📝 mooncake-transfer-engine/example/transfer_engine_bench.cpp (+1 -1)
📝 mooncake-transfer-engine/include/error.h (+1 -0)
➕ mooncake-transfer-engine/include/segment.h (+84 -0)
➕ mooncake-transfer-engine/include/segment_cache.h (+72 -0)
📝 mooncake-transfer-engine/include/topology.h (+1 -1)
📝 mooncake-transfer-engine/include/transfer_engine.h (+2 -3)
📝 mooncake-transfer-engine/include/transfer_metadata.h (+14 -52)
➖ mooncake-transfer-engine/include/transport/cxl_transport/cxl_transport.h (+0 -84)
📝 mooncake-transfer-engine/include/transport/nvmeof_transport/nvmeof_transport.h (+1 -1)
📝 mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h (+0 -14)

...and 26 more files

📄 Description

This patch is the first stage of Transfer Engine's code refactoring.

Regarding that in many AI data centers, there are multiple link between two storage medias, we want Transfer Engine to use optimal link to reach higher throughput. E.g., two processes in one machine may use shared memory or nvlink transports.

But the current implementation requires to use EITHER rdma or TCP, which is not suitable for extension. Also, there are many dead code in the transport code. So I plan to refact the metadata & transport impl code to make Transfer Engine more organizable.

Overally, this patch changes the following things:

Metadata format, see mooncake-transfer-engine/include/segment.h, we also extract some code for coding & decoding segment desc in the seperte file.
We uniform terms SegmentID and SegmentHandle. In the future, we will remove the term local_server_name and replace it with local_segment_name to avoid ambiguous.
Add shared memory transports, currently only enable local-to-local data transfer, the remaining part will be fixed at stage 2
Remove dead code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/147 **Author:** [@alogfans](https://github.com/alogfans) **Created:** 3/17/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `transport-refactor-stage-1` --- ### 📝 Commits (2) - [`18d08a6`](https://github.com/kvcache-ai/Mooncake/commit/18d08a65addb9d2fbf6c2130d198856410db2a57) [feature] update transfer engine metadata format and metadata manager - [`988ba83`](https://github.com/kvcache-ai/Mooncake/commit/988ba83aaedaeec4dc01c636f81e464c1f539845) [fix] fix bugs not passing metadata tests ### 📊 Changes **46 files changed** (+1078 additions, -892 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+3 -0) 📝 `doc/en/transfer-engine.md` (+4 -4) 📝 `doc/zh/transfer-engine.md` (+4 -4) 📝 `mooncake-integration/vllm/vllm_adaptor.cpp` (+2 -2) 📝 `mooncake-integration/vllm/vllm_adaptor.h` (+1 -1) 📝 `mooncake-p2p-store/build.sh` (+2 -1) 📝 `mooncake-store/src/client.cpp` (+5 -1) 📝 `mooncake-store/tests/distributed_object_store_provider.py` (+6 -6) 📝 `mooncake-store/tests/stress_cluster_benchmark.py` (+16 -7) 📝 `mooncake-store/tests/stress_workload_test.cpp` (+9 -7) 📝 `mooncake-transfer-engine/example/transfer_engine_bench.cpp` (+1 -1) 📝 `mooncake-transfer-engine/include/error.h` (+1 -0) ➕ `mooncake-transfer-engine/include/segment.h` (+84 -0) ➕ `mooncake-transfer-engine/include/segment_cache.h` (+72 -0) 📝 `mooncake-transfer-engine/include/topology.h` (+1 -1) 📝 `mooncake-transfer-engine/include/transfer_engine.h` (+2 -3) 📝 `mooncake-transfer-engine/include/transfer_metadata.h` (+14 -52) ➖ `mooncake-transfer-engine/include/transport/cxl_transport/cxl_transport.h` (+0 -84) 📝 `mooncake-transfer-engine/include/transport/nvmeof_transport/nvmeof_transport.h` (+1 -1) 📝 `mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h` (+0 -14) _...and 26 more files_ </details> ### 📄 Description This patch is the first stage of Transfer Engine's code refactoring. Regarding that in many AI data centers, there are multiple link between two storage medias, we want Transfer Engine to use optimal link to reach higher throughput. E.g., two processes in one machine may use shared memory or nvlink transports. But the current implementation requires to use EITHER rdma or TCP, which is not suitable for extension. Also, there are many dead code in the transport code. So I plan to refact the metadata & transport impl code to make Transfer Engine more organizable. Overally, this patch changes the following things: - Metadata format, see `mooncake-transfer-engine/include/segment.h`, we also extract some code for coding & decoding segment desc in the seperte file. - We uniform terms `SegmentID` and `SegmentHandle`. In the future, we will remove the term `local_server_name` and replace it with `local_segment_name` to avoid ambiguous. - Add shared memory transports, currently only enable local-to-local data transfer, the remaining part will be fixed at stage 2 - Remove dead code --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:26:43 +00:00