[PR #1234] [MERGED] [Feature] Support H20 intraNode nvlink by introducing fallback mechanism to leverage cudaIPC

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2092

bsakaguchi commented

2026-10-29 21:32:43 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1234
Author: @TTThanos
Created: 12/18/2025
Status: ✅ Merged
Merged: 12/20/2025
Merged by: @stmatengss

Base: main ← Head: NewFeature_H20_IntraNode

📝 Commits (4)

3ab0df3 Support Fallback strategy in AllocPinnedMemory in nvlink_transport which supports intraNode nvlink transmission
ddb3a3f Change supportFabricMem logic to support fallback CudaMalloc when CuMemCreate file for Fabric handle
efafeec Refine log print for cudaMalloc
fcafc52 Define Macro to avoid build error in MUSA environment

📊 Changes

3 files changed (+56 additions, -4 deletions)

View changed files

📝 mooncake-transfer-engine/include/gpu_vendor/musa.h (+2 -0)
📝 mooncake-transfer-engine/include/transport/nvlink_transport/nvlink_transport.h (+1 -1)
📝 mooncake-transfer-engine/src/transport/nvlink_transport/nvlink_transport.cpp (+53 -3)

📄 Description

…ich supports intraNode nvlink transmission

delete redundant variable in nvlink_transport.h

Description

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

The functionality is tested using transfer_engine_bench on 8 * H20.

NVLINK volume can be observered via nvidia-smi dmon.

To ensure intraNode nvlink feasible, you need to set USE_MNNVL=ON when make your project and set environment variable
MC_MC_USE_NVLINK_IPC=1 when running your test example. For H20 case, Mooncake will automatically fallback to classical memory allocation and gurantee vaild memory handle can be accessed by other GPU card through CudaIPC realated API.

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1234 **Author:** [@TTThanos](https://github.com/TTThanos) **Created:** 12/18/2025 **Status:** ✅ Merged **Merged:** 12/20/2025 **Merged by:** [@stmatengss](https://github.com/stmatengss) **Base:** `main` ← **Head:** `NewFeature_H20_IntraNode` --- ### 📝 Commits (4) - [`3ab0df3`](https://github.com/kvcache-ai/Mooncake/commit/3ab0df380b6219d7ada9bb663a812fe8a66e8f78) Support Fallback strategy in AllocPinnedMemory in nvlink_transport which supports intraNode nvlink transmission - [`ddb3a3f`](https://github.com/kvcache-ai/Mooncake/commit/ddb3a3f8d26b655a066783fe6f0bfa96986ab037) Change supportFabricMem logic to support fallback CudaMalloc when CuMemCreate file for Fabric handle - [`efafeec`](https://github.com/kvcache-ai/Mooncake/commit/efafeecf87e7061b1bdcbfb8a73c37ae15ff02a2) Refine log print for cudaMalloc - [`fcafc52`](https://github.com/kvcache-ai/Mooncake/commit/fcafc52442b3ff8da7faae09f0018bbe1798abf2) Define Macro to avoid build error in MUSA environment ### 📊 Changes **3 files changed** (+56 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `mooncake-transfer-engine/include/gpu_vendor/musa.h` (+2 -0) 📝 `mooncake-transfer-engine/include/transport/nvlink_transport/nvlink_transport.h` (+1 -1) 📝 `mooncake-transfer-engine/src/transport/nvlink_transport/nvlink_transport.cpp` (+53 -3) </details> ### 📄 Description …ich supports intraNode nvlink transmission delete redundant variable in nvlink_transport.h ## Description  ## Type of Change * Types - [ ] Bug fix - [ ] New feature - [x] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? The functionality is tested using transfer_engine_bench on 8 * H20. <img width="2560" height="838" alt="image" src="https://github.com/user-attachments/assets/edeb5ce6-dce0-4c0d-a891-67e00258cc57" /> NVLINK volume can be observered via nvidia-smi dmon. ![Uploading image.png…]() To ensure intraNode nvlink feasible, you need to set USE_MNNVL=ON when make your project and set environment variable MC_MC_USE_NVLINK_IPC=1 when running your test example. For H20 case, Mooncake will automatically fallback to classical memory allocation and gurantee vaild memory handle can be accessed by other GPU card through CudaIPC realated API. ## Checklist - [x] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

closed this issue
added the
pull-request
label

Rows
Columns

[PR #1234] [MERGED] [Feature] Support H20 intraNode nvlink by introducing fallback mechanism to leverage cudaIPC #2092