[GH-ISSUE #780] [RFC]: [TransferEngine] Introduce generic NVMeoF transport implementation #242

New issue

Closed

opened 2026-10-29 21:11:25 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:11:25 +00:00

Owner

Originally created by @nickyc975 on GitHub (Aug 25, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/780

Changes proposed

The current implementation of NVMeoF transport is incomplete, which hinders Moocake users from using SSDs to expand the KV cache storage space. We propose to implement full support of NVMeoF, including the following changes:

Extend TransferEngine to support registering files as shared buffers;
Introduce complete generic NVMeoF transport, including NVMeoF targets that provide file-sharing capabilities and NVMeoF initiators for accessing remote files. This implementation will coexist with the existing NVMeoF transport.

Based on TransferEngine's support for NVMeoF, we further propose to extend mooncake-store and its Python binding to support mounting file segments. This includes the following changes:

Enable segment allocators to support segments with 0 as base address. OffsetAllocator already supports this, while CachelibAllocator can be adapted by adding a non-zero offset to the incoming base address;
Add a "type" field to the Segment type to indicate whether it is a memory segment or a file segment;
Add corresponding interfaces for registering and unregistering file-type segments.

Details about the proposed NVMeoF transport

The proposed NVMeoF transport will not support GDS for now, making it general to different scenarios. GDS support might be added in the future;
The proposed NVMeoF transport will be based on Linux kernel nvme and nvmet drivers, and will migrate to SPDK userspace drivers in the future.
Direct-IO is not currently supported because allocators cannot ensure buffer alignment. To fully utilize the performance of high-end SSDs, more work needs to be done on allocators.

References

libnvme: https://github.com/linux-nvme/libnvme
Linux nvme target configuration: RedHat: Configuring NVMe over fabrics using NVMe/RDMA
SPDK: https://github.com/spdk/spdk

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @nickyc975 on GitHub (Aug 25, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/780 ### Changes proposed The current implementation of NVMeoF transport is incomplete, which hinders Moocake users from using SSDs to expand the KV cache storage space. We propose to implement full support of NVMeoF, including the following changes: 1. Extend TransferEngine to support registering files as shared buffers; 2. Introduce complete generic NVMeoF transport, including NVMeoF targets that provide file-sharing capabilities and NVMeoF initiators for accessing remote files. This implementation will coexist with the existing NVMeoF transport. Based on TransferEngine's support for NVMeoF, we further propose to extend mooncake-store and its Python binding to support mounting file segments. This includes the following changes: 1. Enable segment allocators to support segments with 0 as base address. OffsetAllocator already supports this, while CachelibAllocator can be adapted by adding a non-zero offset to the incoming base address; 2. Add a "type" field to the Segment type to indicate whether it is a memory segment or a file segment; 3. Add corresponding interfaces for registering and unregistering file-type segments. ### Details about the proposed NVMeoF transport 1. The proposed NVMeoF transport will not support GDS for now, making it general to different scenarios. GDS support might be added in the future; 2. The proposed NVMeoF transport will be based on Linux kernel nvme and nvmet drivers, and will migrate to SPDK userspace drivers in the future. 3. Direct-IO is not currently supported because allocators cannot ensure buffer alignment. To fully utilize the performance of high-end SSDs, more work needs to be done on allocators. ### References 1. libnvme: [https://github.com/linux-nvme/libnvme](https://github.com/linux-nvme/libnvme) 2. Linux nvme target configuration: [RedHat: Configuring NVMe over fabrics using NVMe/RDMA](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/managing_storage_devices/configuring-nvme-over-fabrics-using-nvme-rdma_managing-storage-devices) 3. SPDK: [https://github.com/spdk/spdk](https://github.com/spdk/spdk) ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi

2026-10-29 21:11:25 +00:00

closed this issue
added the
stale

auto-closed

enhancement
labels

bsakaguchi commented

2026-10-29 21:11:26 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 27, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jun 27, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:11:26 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 5, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 5, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:27:21 +00:00

[PR #245] [MERGED] [Add new python http server] Python http server #1427