[PR #704] [MERGED] Add ability to do RDMA without nvidia-peermem #1758

New issue

Closed

opened 2026-10-29 21:30:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:30:05 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/704
Author: @misterwilliam
Created: 7/31/2025
Status: ✅ Merged
Merged: 8/9/2025
Merged by: @stmatengss

Base: main ← Head: main

📝 Commits (1)

d68b4e8 Add ability to do RDMA without nvidia-peermem

📊 Changes

3 files changed (+92 additions, -17 deletions)

View changed files

📝 mooncake-common/common.cmake (+6 -2)
📝 mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h (+8 -1)
📝 mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp (+78 -14)

📄 Description

Migrate from ibv_reg_mr() to ibv_reg_dmabuf_mr(). This allows us to not need nvidia-peermem for RDMA because ibv_reg_dmabuf_mr() does not require nvidia-peermem on nvidia GPUs while ibv_reg_mr() does require nvidia-peermem.

Need to introduce a new struct (MrMeta) to track starting address of MR because ibv_reg_dmabuf_mr() is iova based and doesn't set mr->address to starting address, so we need to track it ourselves.

Closes #674

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/704 **Author:** [@misterwilliam](https://github.com/misterwilliam) **Created:** 7/31/2025 **Status:** ✅ Merged **Merged:** 8/9/2025 **Merged by:** [@stmatengss](https://github.com/stmatengss) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (1) - [`d68b4e8`](https://github.com/kvcache-ai/Mooncake/commit/d68b4e80d117a7a174ef8d91d2cfb16185ecabb9) Add ability to do RDMA without nvidia-peermem ### 📊 Changes **3 files changed** (+92 additions, -17 deletions) <details> <summary>View changed files</summary> 📝 `mooncake-common/common.cmake` (+6 -2) 📝 `mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h` (+8 -1) 📝 `mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp` (+78 -14) </details> ### 📄 Description Migrate from ibv_reg_mr() to ibv_reg_dmabuf_mr(). This allows us to not need nvidia-peermem for RDMA because ibv_reg_dmabuf_mr() does not require nvidia-peermem on nvidia GPUs while ibv_reg_mr() does require nvidia-peermem. Need to introduce a new struct (MrMeta) to track starting address of MR because ibv_reg_dmabuf_mr() is iova based and doesn't set mr->address to starting address, so we need to track it ourselves. Closes #674 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:30:05 +00:00