[PR #1320] [CLOSED] [TE] IntraNode nvlink isolation: isolate intraNode nvlink connection from previous nvlink_transport

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2159

bsakaguchi commented

2026-10-29 21:33:13 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1320
Author: @TTThanos
Created: 1/4/2026
Status: ❌ Closed

Base: main ← Head: nvlink_intra_isolation

📝 Commits (6)

175b050 Add early mem backend detection method in NVLINK_allocator
239a3e9 Change enumerate type in allocator.py for different mem backend
e321c35 Isolate nvlink intraNode transport from nvlink_transport and modify corresponding transfer_engine_bench
9b69a56 IntraNode transport isolation to be compatible with SGlang
082a3e0 Merge branch 'main' into nvlink_intra_isolation
ced3d8a Modify the code style using pre-commit check

📊 Changes

13 files changed (+819 additions, -29 deletions)

View changed files

📝 docs/source/design/transfer-engine/index.md (+9 -9)
📝 mooncake-integration/allocator.py (+82 -4)
📝 mooncake-integration/transfer_engine/transfer_engine_py.cpp (+31 -7)
📝 mooncake-transfer-engine/example/transfer_engine_bench.cpp (+56 -1)
📝 mooncake-transfer-engine/example/transfer_engine_bench_with_notify.cpp (+2 -0)
📝 mooncake-transfer-engine/example/transfer_engine_validator.cpp (+2 -0)
📝 mooncake-transfer-engine/include/gpu_vendor/mnnvl.h (+6 -0)
➕ mooncake-transfer-engine/include/transport/nvlink_transport/intranode_nvlink_transport.h (+85 -0)
📝 mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp (+37 -0)
📝 mooncake-transfer-engine/src/multi_transport.cpp (+3 -0)
📝 mooncake-transfer-engine/src/transfer_engine_impl.cpp (+20 -7)
📝 mooncake-transfer-engine/src/transfer_metadata.cpp (+3 -1)
➕ mooncake-transfer-engine/src/transport/nvlink_transport/intranode_nvlink_transport.cpp (+483 -0)

📄 Description

Description

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

By adding a new subtransport class IntraNodeNvlinkTransport to handle the situation of intraNode kv cache transmission.
Along with Autodetect mechanism in PR https://github.com/kvcache-ai/Mooncake/pull/1296/files/056a45bf34e75e92b7530c3c10f82e72c0136c40..07b3bb1f618ecabbb0febeb91f2d796035634d85
SGlang can automatically use Intranode_nvlink connection when set MC_FORCE_MNNVL.

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1320 **Author:** [@TTThanos](https://github.com/TTThanos) **Created:** 1/4/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `nvlink_intra_isolation` --- ### 📝 Commits (6) - [`175b050`](https://github.com/kvcache-ai/Mooncake/commit/175b0501faae86bb7f053650f8a915dcfba9bff2) Add early mem backend detection method in NVLINK_allocator - [`239a3e9`](https://github.com/kvcache-ai/Mooncake/commit/239a3e9a17ba1eafffb1e210cbe3e118d33934eb) Change enumerate type in allocator.py for different mem backend - [`e321c35`](https://github.com/kvcache-ai/Mooncake/commit/e321c35a939f116db8179f69434ea0eadb315f70) Isolate nvlink intraNode transport from nvlink_transport and modify corresponding transfer_engine_bench - [`9b69a56`](https://github.com/kvcache-ai/Mooncake/commit/9b69a560ebdba6d045282b0893c2b6588531b5a8) IntraNode transport isolation to be compatible with SGlang - [`082a3e0`](https://github.com/kvcache-ai/Mooncake/commit/082a3e002ef958f38215334e425300be84840d32) Merge branch 'main' into nvlink_intra_isolation - [`ced3d8a`](https://github.com/kvcache-ai/Mooncake/commit/ced3d8a081002aa872021d5dfd7bbe553b7a9d8a) Modify the code style using pre-commit check ### 📊 Changes **13 files changed** (+819 additions, -29 deletions) <details> <summary>View changed files</summary> 📝 `docs/source/design/transfer-engine/index.md` (+9 -9) 📝 `mooncake-integration/allocator.py` (+82 -4) 📝 `mooncake-integration/transfer_engine/transfer_engine_py.cpp` (+31 -7) 📝 `mooncake-transfer-engine/example/transfer_engine_bench.cpp` (+56 -1) 📝 `mooncake-transfer-engine/example/transfer_engine_bench_with_notify.cpp` (+2 -0) 📝 `mooncake-transfer-engine/example/transfer_engine_validator.cpp` (+2 -0) 📝 `mooncake-transfer-engine/include/gpu_vendor/mnnvl.h` (+6 -0) ➕ `mooncake-transfer-engine/include/transport/nvlink_transport/intranode_nvlink_transport.h` (+85 -0) 📝 `mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp` (+37 -0) 📝 `mooncake-transfer-engine/src/multi_transport.cpp` (+3 -0) 📝 `mooncake-transfer-engine/src/transfer_engine_impl.cpp` (+20 -7) 📝 `mooncake-transfer-engine/src/transfer_metadata.cpp` (+3 -1) ➕ `mooncake-transfer-engine/src/transport/nvlink_transport/intranode_nvlink_transport.cpp` (+483 -0) </details> ### 📄 Description ## Description  ## Type of Change * Types - [ ] Bug fix - [x] New feature - [ ] Transfer Engine - [ ] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? By adding a new subtransport class IntraNodeNvlinkTransport to handle the situation of intraNode kv cache transmission. Along with Autodetect mechanism in PR https://github.com/kvcache-ai/Mooncake/pull/1296/files/056a45bf34e75e92b7530c3c10f82e72c0136c40..07b3bb1f618ecabbb0febeb91f2d796035634d85 SGlang can automatically use Intranode_nvlink connection when set MC_FORCE_MNNVL. <img width="2216" height="992" alt="image" src="https://github.com/user-attachments/assets/ea0045c5-1103-406d-aad7-36037f7a4782" /> ## Checklist - [x] I have performed a self-review of my own code. - [ ] I have updated the documentation. - [ ] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

closed this issue
added the
pull-request
label

Rows
Columns

[PR #1320] [CLOSED] [TE] IntraNode nvlink isolation: isolate intraNode nvlink connection from previous nvlink_transport #2159