[PR #796] [MERGED] [TransferEngine Next] Preview version of Mooncake Transfer Engine NexT #1829

New issue

Closed

opened 2026-10-29 21:30:41 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:30:41 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/796
Author: @alogfans
Created: 9/1/2025
Status: ✅ Merged
Merged: 9/22/2025
Merged by: @alogfans

Base: dev/tent ← Head: dev/tent-preview

📝 Commits (10+)

52d9e20 Add cudaDeviceSynchronize after first address mapping
9e64313 Squash all changes of Mooncake TE v1
088909f Fix build bugs
03da25f Fix CI problems
9018b1d Fix typos
3902c58 Fix CI bugs
bcb8901 Fix CI problem in etcd plugin
e3d6d65 Fix etcd dependency
51cf6b5 Add -ltransfer_engine_v1 in p2p store
73c2bb6 Fix CI problem in p2pstore

📊 Changes

108 files changed (+15631 additions, -24 deletions)

View changed files

📝 dependencies.sh (+1 -0)
➕ doc/en/v1/tebench-usage.md (+91 -0)
📝 mooncake-integration/transfer_engine/transfer_engine_py.cpp (+293 -13)
📝 mooncake-integration/transfer_engine/transfer_engine_py.h (+4 -0)
📝 mooncake-p2p-store/build.sh (+3 -1)
📝 mooncake-transfer-engine/CMakeLists.txt (+2 -0)
➕ mooncake-transfer-engine/benchmark/CMakeLists.txt (+3 -0)
➕ mooncake-transfer-engine/benchmark/bench_runner.h (+70 -0)
➕ mooncake-transfer-engine/benchmark/main.cpp (+166 -0)
➕ mooncake-transfer-engine/benchmark/tev0_backend.cpp (+315 -0)
➕ mooncake-transfer-engine/benchmark/tev0_backend.h (+98 -0)
➕ mooncake-transfer-engine/benchmark/tev1_backend.cpp (+277 -0)
➕ mooncake-transfer-engine/benchmark/tev1_backend.h (+100 -0)
➕ mooncake-transfer-engine/benchmark/utils.cpp (+142 -0)
➕ mooncake-transfer-engine/benchmark/utils.h (+176 -0)
➕ mooncake-transfer-engine/config/cluster-topology.json (+658 -0)
➕ mooncake-transfer-engine/config/transfer-engine.json (+66 -0)
📝 mooncake-transfer-engine/example/CMakeLists.txt (+1 -1)
➕ mooncake-transfer-engine/include/v1/common/config.h (+72 -0)
➕ mooncake-transfer-engine/include/v1/common/status.h (+199 -0)

...and 80 more files

📄 Description

This PR includes the revised version of Mooncake Transfer Engine, which introduces brand new design and adds lots of new features. It will not merged to the main branch, instead we add a seperate branch called dev/tent.

Users may try using it at their own risk (as we didn't use it in production before).

Key features:

Supports mixed use of RDMA, NVLink (including regular and multi-node), TCP, GDS, IOUring, Shared Memory and CXL over DRAM/VRAM/NVMe SSD. That is, TENT can use NVLink if it is available in some specific requests, while others still use RDMA.
Better path selection policy used in RDMA transport, making better fairness instead of random selection.
More structureful code.
Replace massive environment variables with a unified configuration that is easy to maintain.
New benchmarking tool.
...

To enable it in Python package, use environment variable USE_TEV1=1.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/796 **Author:** [@alogfans](https://github.com/alogfans) **Created:** 9/1/2025 **Status:** ✅ Merged **Merged:** 9/22/2025 **Merged by:** [@alogfans](https://github.com/alogfans) **Base:** `dev/tent` ← **Head:** `dev/tent-preview` --- ### 📝 Commits (10+) - [`52d9e20`](https://github.com/kvcache-ai/Mooncake/commit/52d9e20cbfef937dba8e5c510a38d7356da9027d) Add cudaDeviceSynchronize after first address mapping - [`9e64313`](https://github.com/kvcache-ai/Mooncake/commit/9e643132a409a737104f279c070df06482a0c95e) Squash all changes of Mooncake TE v1 - [`088909f`](https://github.com/kvcache-ai/Mooncake/commit/088909f9e5fad68e8614d3a0a1da686a2ede4a26) Fix build bugs - [`03da25f`](https://github.com/kvcache-ai/Mooncake/commit/03da25fa453d649beb76507b92a7d1c999ee641c) Fix CI problems - [`9018b1d`](https://github.com/kvcache-ai/Mooncake/commit/9018b1d568e6b9ff3a2545a1e42214ca580cb2fc) Fix typos - [`3902c58`](https://github.com/kvcache-ai/Mooncake/commit/3902c58e18cde7016b118f8a2f33eade3ecd307d) Fix CI bugs - [`bcb8901`](https://github.com/kvcache-ai/Mooncake/commit/bcb8901a9033a0fb8b8d45d8a07e52e65240f7da) Fix CI problem in etcd plugin - [`e3d6d65`](https://github.com/kvcache-ai/Mooncake/commit/e3d6d6598aaad7e1ac6fc3cbf2b4c9e349780ffd) Fix etcd dependency - [`51cf6b5`](https://github.com/kvcache-ai/Mooncake/commit/51cf6b59c2f834ac146213e788d84c4899e47ba9) Add -ltransfer_engine_v1 in p2p store - [`73c2bb6`](https://github.com/kvcache-ai/Mooncake/commit/73c2bb68a0a9b012f4ad0be116e1ebcec5f3a242) Fix CI problem in p2pstore ### 📊 Changes **108 files changed** (+15631 additions, -24 deletions) <details> <summary>View changed files</summary> 📝 `dependencies.sh` (+1 -0) ➕ `doc/en/v1/tebench-usage.md` (+91 -0) 📝 `mooncake-integration/transfer_engine/transfer_engine_py.cpp` (+293 -13) 📝 `mooncake-integration/transfer_engine/transfer_engine_py.h` (+4 -0) 📝 `mooncake-p2p-store/build.sh` (+3 -1) 📝 `mooncake-transfer-engine/CMakeLists.txt` (+2 -0) ➕ `mooncake-transfer-engine/benchmark/CMakeLists.txt` (+3 -0) ➕ `mooncake-transfer-engine/benchmark/bench_runner.h` (+70 -0) ➕ `mooncake-transfer-engine/benchmark/main.cpp` (+166 -0) ➕ `mooncake-transfer-engine/benchmark/tev0_backend.cpp` (+315 -0) ➕ `mooncake-transfer-engine/benchmark/tev0_backend.h` (+98 -0) ➕ `mooncake-transfer-engine/benchmark/tev1_backend.cpp` (+277 -0) ➕ `mooncake-transfer-engine/benchmark/tev1_backend.h` (+100 -0) ➕ `mooncake-transfer-engine/benchmark/utils.cpp` (+142 -0) ➕ `mooncake-transfer-engine/benchmark/utils.h` (+176 -0) ➕ `mooncake-transfer-engine/config/cluster-topology.json` (+658 -0) ➕ `mooncake-transfer-engine/config/transfer-engine.json` (+66 -0) 📝 `mooncake-transfer-engine/example/CMakeLists.txt` (+1 -1) ➕ `mooncake-transfer-engine/include/v1/common/config.h` (+72 -0) ➕ `mooncake-transfer-engine/include/v1/common/status.h` (+199 -0) _...and 80 more files_ </details> ### 📄 Description This PR includes the revised version of Mooncake Transfer Engine, which introduces brand new design and adds lots of new features. It will not merged to the main branch, instead we add a seperate branch called `dev/tent`. Users may try using it at their own risk (as we didn't use it in production before). Key features: - Supports mixed use of RDMA, NVLink (including regular and multi-node), TCP, GDS, IOUring, Shared Memory and CXL over DRAM/VRAM/NVMe SSD. That is, TENT can use NVLink if it is available in some specific requests, while others still use RDMA. - Better path selection policy used in RDMA transport, making better fairness instead of random selection. - More structureful code. - Replace massive environment variables with a unified configuration that is easy to maintain. - New benchmarking tool. - ... To enable it in Python package, use environment variable `USE_TEV1=1`. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:30:41 +00:00