[PR #170] [CLOSED] [TransferEngine] Add shared memory transport and refactor metadata format #1369

New issue

Closed

opened 2026-10-29 21:26:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:26:49 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/170
Author: @alogfans
Created: 3/25/2025
Status: ❌ Closed

Base: main ← Head: transfer-engine-v2

📝 Commits (3)

c8bc443 [FEATURE] add new directory for maintaining next transfer engine based on #147
cb13668 fix typos
e0b4c56 [FIX] fix typos of CANNELED

📊 Changes

80 files changed (+10624 additions, -14 deletions)

View changed files

📝 CMakeLists.txt (+1 -0)
📝 doc/en/transfer-engine.md (+1 -1)
📝 doc/zh/transfer-engine.md (+1 -1)
➕ mooncake-ngxl-exp/CMakeLists.txt (+5 -0)
➕ mooncake-ngxl-exp/README.md (+4 -0)
➕ mooncake-ngxl-exp/example/CMakeLists.txt (+5 -0)
➕ mooncake-ngxl-exp/example/http-metadata-server/go.mod (+34 -0)
➕ mooncake-ngxl-exp/example/http-metadata-server/main.go (+79 -0)
➕ mooncake-ngxl-exp/example/memory_pool.cpp (+114 -0)
➕ mooncake-ngxl-exp/example/transfer_engine_bench.cpp (+427 -0)
➕ mooncake-ngxl-exp/include/CMakeLists.txt (+1 -0)
➕ mooncake-ngxl-exp/include/common.h (+363 -0)
➕ mooncake-ngxl-exp/include/common/base/status.h (+283 -0)
➕ mooncake-ngxl-exp/include/config.h (+57 -0)
➕ mooncake-ngxl-exp/include/error.h (+40 -0)
➕ mooncake-ngxl-exp/include/memory_location.h (+40 -0)
➕ mooncake-ngxl-exp/include/multi_transport.h (+66 -0)
➕ mooncake-ngxl-exp/include/segment.h (+84 -0)
➕ mooncake-ngxl-exp/include/segment_cache.h (+72 -0)
➕ mooncake-ngxl-exp/include/topology.h (+100 -0)

...and 60 more files

📄 Description

According to issue #162, we recreate a pull request based on #134 #147. All modified code is in mooncake-ngxl directory.

This patch is the first stage of Transfer Engine's code refactoring.

Regarding that in many AI data centers, there are multiple link between two storage medias, we want Transfer Engine to use optimal link to reach higher throughput. E.g., two processes in one machine may use shared memory or nvlink transports.

But the current implementation requires to use EITHER rdma or TCP, which is not suitable for extension. Also, there are many dead code in the transport code. So I plan to refact the metadata & transport impl code to make Transfer Engine more organizable.

Overally, this patch changes the following things:

Metadata format, see mooncake-transfer-engine/include/segment.h, we also extract some code for coding & decoding segment desc in the seperte file.
We uniform terms SegmentID and SegmentHandle. In the future, we will remove the term local_server_name and replace it with local_segment_name to avoid ambiguous.
Add shared memory transports, currently only enable local-to-local data transfer, the remaining part will be fixed at stage 2
Remove dead code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/170 **Author:** [@alogfans](https://github.com/alogfans) **Created:** 3/25/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `transfer-engine-v2` --- ### 📝 Commits (3) - [`c8bc443`](https://github.com/kvcache-ai/Mooncake/commit/c8bc4439f369c5453c553a9010b3aa6fac430edb) [FEATURE] add new directory for maintaining next transfer engine based on #147 - [`cb13668`](https://github.com/kvcache-ai/Mooncake/commit/cb1366866a3f7f26c46ab38f82467ff8123a305e) fix typos - [`e0b4c56`](https://github.com/kvcache-ai/Mooncake/commit/e0b4c5615e4de23fb2cb7d2326a2b7a34b83010b) [FIX] fix typos of CANNELED ### 📊 Changes **80 files changed** (+10624 additions, -14 deletions) <details> <summary>View changed files</summary> 📝 `CMakeLists.txt` (+1 -0) 📝 `doc/en/transfer-engine.md` (+1 -1) 📝 `doc/zh/transfer-engine.md` (+1 -1) ➕ `mooncake-ngxl-exp/CMakeLists.txt` (+5 -0) ➕ `mooncake-ngxl-exp/README.md` (+4 -0) ➕ `mooncake-ngxl-exp/example/CMakeLists.txt` (+5 -0) ➕ `mooncake-ngxl-exp/example/http-metadata-server/go.mod` (+34 -0) ➕ `mooncake-ngxl-exp/example/http-metadata-server/main.go` (+79 -0) ➕ `mooncake-ngxl-exp/example/memory_pool.cpp` (+114 -0) ➕ `mooncake-ngxl-exp/example/transfer_engine_bench.cpp` (+427 -0) ➕ `mooncake-ngxl-exp/include/CMakeLists.txt` (+1 -0) ➕ `mooncake-ngxl-exp/include/common.h` (+363 -0) ➕ `mooncake-ngxl-exp/include/common/base/status.h` (+283 -0) ➕ `mooncake-ngxl-exp/include/config.h` (+57 -0) ➕ `mooncake-ngxl-exp/include/error.h` (+40 -0) ➕ `mooncake-ngxl-exp/include/memory_location.h` (+40 -0) ➕ `mooncake-ngxl-exp/include/multi_transport.h` (+66 -0) ➕ `mooncake-ngxl-exp/include/segment.h` (+84 -0) ➕ `mooncake-ngxl-exp/include/segment_cache.h` (+72 -0) ➕ `mooncake-ngxl-exp/include/topology.h` (+100 -0) _...and 60 more files_ </details> ### 📄 Description According to issue #162, we recreate a pull request based on #134 #147. All modified code is in `mooncake-ngxl` directory. --- This patch is the first stage of Transfer Engine's code refactoring. Regarding that in many AI data centers, there are multiple link between two storage medias, we want Transfer Engine to use optimal link to reach higher throughput. E.g., two processes in one machine may use shared memory or nvlink transports. But the current implementation requires to use EITHER rdma or TCP, which is not suitable for extension. Also, there are many dead code in the transport code. So I plan to refact the metadata & transport impl code to make Transfer Engine more organizable. Overally, this patch changes the following things: - Metadata format, see mooncake-transfer-engine/include/segment.h, we also extract some code for coding & decoding segment desc in the seperte file. - We uniform terms SegmentID and SegmentHandle. In the future, we will remove the term local_server_name and replace it with local_segment_name to avoid ambiguous. - Add shared memory transports, currently only enable local-to-local data transfer, the remaining part will be fixed at stage 2 - Remove dead code --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:26:49 +00:00