[PR #173] [MERGED] Feature: LL nvlink p2p #448

New issue

Closed

opened 2026-10-26 10:00:51 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 10:00:51 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/173
Author: @cywork121
Created: 5/22/2025
Status: ✅ Merged
Merged: 5/23/2025
Merged by: @LyricZhao

Base: main ← Head: DeepEP/ll_nvlink_load_store

📝 Commits (1)

0c02143 Feature: LL nvlink p2p

📊 Changes

2 files changed (+34 additions, -5 deletions)

View changed files

📝 csrc/kernels/internode_ll.cu (+32 -4)
📝 deep_ep/buffer.py (+2 -1)

📄 Description

This PR introduces the use of NVLink P2P copy for intra-node data transfer in low-latency scenarios. For smaller-scale decode instances requirements, we found this approach enhances the overall decoding throughput. Specifically in our inference service, the 4-node decode efficiency demonstrates a 27% improvement.

The following test data was obtained using H20:

Dispatch #EP	Latency(us)	bandwidth(GB/s)	Combine #EP	Latency(us)	bandwidth(GB/s)
8	172 👉🏻 37	43.5 👉🏻 203.2	8	319 👉🏻 59	45.5 👉🏻 244.2
16	193 👉🏻 119	38.8 👉🏻 62.9	16	335 👉🏻 199	43.3 👉🏻 72.7
32	198 👉🏻 157	37.8 👉🏻 47.8	32	344 👉🏻 280	42.1 👉🏻 51.7
64	203 👉🏻 176	36.9 👉🏻 42.6	64	351 👉🏻 318	41.3 👉🏻 45.6

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/173 **Author:** [@cywork121](https://github.com/cywork121) **Created:** 5/22/2025 **Status:** ✅ Merged **Merged:** 5/23/2025 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `DeepEP/ll_nvlink_load_store` --- ### 📝 Commits (1) - [`0c02143`](https://github.com/deepseek-ai/DeepEP/commit/0c02143a9d4e551b3b800a6327fe8eb3af514f7a) Feature: LL nvlink p2p ### 📊 Changes **2 files changed** (+34 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `csrc/kernels/internode_ll.cu` (+32 -4) 📝 `deep_ep/buffer.py` (+2 -1) </details> ### 📄 Description This PR introduces the use of NVLink P2P copy for intra-node data transfer in low-latency scenarios. For smaller-scale decode instances requirements, we found this approach enhances the overall decoding throughput. Specifically in our inference service, the 4-node decode efficiency demonstrates a 27% improvement. The following test data was obtained using H20: Dispatch #EP | Latency(us) | bandwidth(GB/s) | Combine #EP | Latency(us) | bandwidth(GB/s) -- | -- | -- | -- | -- | -- 8 | 172 👉🏻 37 | 43.5 👉🏻 203.2 | 8 | 319 👉🏻 59 | 45.5 👉🏻 244.2 16 | 193 👉🏻 119 | 38.8 👉🏻 62.9 | 16 | 335 👉🏻 199 | 43.3 👉🏻 72.7 32 | 198 👉🏻 157 | 37.8 👉🏻 47.8 | 32 | 344 👉🏻 280 | 42.1 👉🏻 51.7 64 | 203 👉🏻 176 | 36.9 👉🏻 42.6 | 64 | 351 👉🏻 318 | 41.3 👉🏻 45.6 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 10:00:51 +00:00