[PR #1197] [CLOSED] [Feature] Intranode NVLINK Transport support in Hopper GPU

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2065

bsakaguchi commented

2026-10-29 21:32:31 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1197
Author: @TTThanos
Created: 12/11/2025
Status: ❌ Closed

Base: main ← Head: Feature_IntraNode_NVLINK

📝 Commits (10+)

22c41e8 Support NVLINK transfer intraNode in H20---Tested on example/transfer_engine_bench
7ac0c14 Simplified the fb transfer phase using one socket | automatically dectect Fabric/Posix handle | design gracefully shutdown mechanism
e8ac05b Modify transfer_engine_bench to be compatible with USE_MNNVL
b78105c Delete redundant log and set shutdown() to public method
f5d9fb7 pre-commit modify format of Codes committed previously
d372f04 Use abstract namespace to establish socket connectin between server and client side
fb54c20 delete debug log in metadata_plugin.cpp
8b98792 Clean log print and remove useless functions
937cb54 Add requestedHandleTypes detection method in nvlink_allocator
428320c Merge branch 'main' into Feature_IntraNode_NVLINK

📊 Changes

7 files changed (+621 additions, -36 deletions)

View changed files

📝 mooncake-transfer-engine/example/transfer_engine_bench.cpp (+14 -1)
📝 mooncake-transfer-engine/include/transfer_metadata.h (+1 -0)
📝 mooncake-transfer-engine/include/transport/nvlink_transport/nvlink_transport.h (+35 -1)
📝 mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp (+55 -1)
📝 mooncake-transfer-engine/src/transfer_metadata.cpp (+22 -5)
📝 mooncake-transfer-engine/src/transfer_metadata_plugin.cpp (+1 -2)
📝 mooncake-transfer-engine/src/transport/nvlink_transport/nvlink_transport.cpp (+493 -26)

📄 Description

Description

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

This patch is validated by transfer_engine_bench which use one GPU to transport data to another GPU on 8 * H20 machine.

The topo of the machine is below:

The screenshot of the test result is:

You can observe the nvlink volume through nvidia-smi dmon when running the test:

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1197 **Author:** [@TTThanos](https://github.com/TTThanos) **Created:** 12/11/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `Feature_IntraNode_NVLINK` --- ### 📝 Commits (10+) - [`22c41e8`](https://github.com/kvcache-ai/Mooncake/commit/22c41e8f0b9cb0ac8b65ab0d69d3a75f9734e86f) Support NVLINK transfer intraNode in H20---Tested on example/transfer_engine_bench - [`7ac0c14`](https://github.com/kvcache-ai/Mooncake/commit/7ac0c1490e49645dc0a93620f2c868d54826386b) Simplified the fb transfer phase using one socket | automatically dectect Fabric/Posix handle | design gracefully shutdown mechanism - [`e8ac05b`](https://github.com/kvcache-ai/Mooncake/commit/e8ac05b870e99b5a7062934f224aad4e3306412a) Modify transfer_engine_bench to be compatible with USE_MNNVL - [`b78105c`](https://github.com/kvcache-ai/Mooncake/commit/b78105c51d1696f73536ce6855ddf573f876bd99) Delete redundant log and set shutdown() to public method - [`f5d9fb7`](https://github.com/kvcache-ai/Mooncake/commit/f5d9fb7a3ba480b2e77ea5b49241c4ad428b973d) pre-commit modify format of Codes committed previously - [`d372f04`](https://github.com/kvcache-ai/Mooncake/commit/d372f0484d1393a19d0da76c3a88e687407b885f) Use abstract namespace to establish socket connectin between server and client side - [`fb54c20`](https://github.com/kvcache-ai/Mooncake/commit/fb54c20bd1a75e7dce1c3f887ba3067f38e043e5) delete debug log in metadata_plugin.cpp - [`8b98792`](https://github.com/kvcache-ai/Mooncake/commit/8b9879205891ee3f1b2d4b3c8c56952a71bf233f) Clean log print and remove useless functions - [`937cb54`](https://github.com/kvcache-ai/Mooncake/commit/937cb54d8a1dc79e2f3c5fa3ac4edd307992d84b) Add requestedHandleTypes detection method in nvlink_allocator - [`428320c`](https://github.com/kvcache-ai/Mooncake/commit/428320cddf2f983fae68b0d6d84082f12c2589bd) Merge branch 'main' into Feature_IntraNode_NVLINK ### 📊 Changes **7 files changed** (+621 additions, -36 deletions) <details> <summary>View changed files</summary> 📝 `mooncake-transfer-engine/example/transfer_engine_bench.cpp` (+14 -1) 📝 `mooncake-transfer-engine/include/transfer_metadata.h` (+1 -0) 📝 `mooncake-transfer-engine/include/transport/nvlink_transport/nvlink_transport.h` (+35 -1) 📝 `mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp` (+55 -1) 📝 `mooncake-transfer-engine/src/transfer_metadata.cpp` (+22 -5) 📝 `mooncake-transfer-engine/src/transfer_metadata_plugin.cpp` (+1 -2) 📝 `mooncake-transfer-engine/src/transport/nvlink_transport/nvlink_transport.cpp` (+493 -26) </details> ### 📄 Description ## Description  ## Type of Change * Types - [ ] Bug fix - [x] New feature - [x] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? This patch is validated by transfer_engine_bench which use one GPU to transport data to another GPU on 8 * H20 machine. The topo of the machine is below: <img width="1666" height="566" alt="image" src="https://github.com/user-attachments/assets/8d7e51ca-ca3a-4ef2-9c78-d81b62ff6f4d" /> The screenshot of the test result is: <img width="3832" height="1038" alt="image" src="https://github.com/user-attachments/assets/be64e670-f42e-4061-8818-05694212d1fe" /> You can observe the nvlink volume through nvidia-smi dmon when running the test: <img width="1476" height="1236" alt="image" src="https://github.com/user-attachments/assets/772e78cf-e035-4ade-a4e9-f8456dba1c81" /> ## Checklist - [ ] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

closed this issue
added the
pull-request
label

Rows
Columns

[PR #1197] [CLOSED] [Feature] Intranode NVLINK Transport support in Hopper GPU #2065