[PR #501] [MERGED] Support cuMem API in cross process shared memory management #600

New issue

Closed

opened 2026-10-27 21:04:25 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:04:25 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/501
Author: @shifangx
Created: 11/24/2025
Status: ✅ Merged
Merged: 11/25/2025
Merged by: @jershi425

Base: hybrid-ep ← Head: shifang/cuMem

📝 Commits (3)

d146713 Support cuMem API in cross process shared memory management
6d365cc Update configs.cuh
a28cb93 Fix tensor descriptions in buffer.py

📊 Changes

6 files changed (+205 additions, -24 deletions)

View changed files

📝 csrc/deep_ep.cpp (+140 -17)
📝 csrc/deep_ep.hpp (+39 -2)
📝 csrc/kernels/configs.cuh (+1 -1)
📝 csrc/kernels/exception.cuh (+12 -0)
📝 deep_ep/buffer.py (+6 -3)
📝 tests/test_intranode.py (+7 -1)

📄 Description

In order to support normal intranode EP8 on GB200, we cherry-pick pr-212 from main branch to hybrid-ep branch.

Unit test passed with DeepEP/tests/test_intranode.py, using sglang:blackwell-cu129-arm64.sqsh.

hybrid-ep branch + pr501, without --use-fabric, EP4

[tuning] Best combine: SMs 24, NVL chunk 16: 351.05 GB/s (NVL), t: 602.89 us

hybrid-ep branch + pr501, with --use-fabric, EP8

[tuning] Best combine: SMs 24, NVL chunk 15: 417.83 GB/s (NVL), t: 748.14 us

This pr also includes two tiny fixes for low_latency_dispatch, which are Fix tensor descriptions in buffer.py and Fix racing condition in large batch size.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/501 **Author:** [@shifangx](https://github.com/shifangx) **Created:** 11/24/2025 **Status:** ✅ Merged **Merged:** 11/25/2025 **Merged by:** [@jershi425](https://github.com/jershi425) **Base:** `hybrid-ep` ← **Head:** `shifang/cuMem` --- ### 📝 Commits (3) - [`d146713`](https://github.com/deepseek-ai/DeepEP/commit/d146713ee53adc4265d8c7eacf28561a145e052a) Support cuMem API in cross process shared memory management - [`6d365cc`](https://github.com/deepseek-ai/DeepEP/commit/6d365ccbf5a9b6fb7d1d46169b8b76431b0c667f) Update configs.cuh - [`a28cb93`](https://github.com/deepseek-ai/DeepEP/commit/a28cb9306cc0d9ea197ffd690e6c2ea3064ccaf5) Fix tensor descriptions in buffer.py ### 📊 Changes **6 files changed** (+205 additions, -24 deletions) <details> <summary>View changed files</summary> 📝 `csrc/deep_ep.cpp` (+140 -17) 📝 `csrc/deep_ep.hpp` (+39 -2) 📝 `csrc/kernels/configs.cuh` (+1 -1) 📝 `csrc/kernels/exception.cuh` (+12 -0) 📝 `deep_ep/buffer.py` (+6 -3) 📝 `tests/test_intranode.py` (+7 -1) </details> ### 📄 Description In order to support normal intranode EP8 on GB200, we cherry-pick [pr-212](https://github.com/deepseek-ai/DeepEP/pull/217) from main branch to hybrid-ep branch. Unit test passed with DeepEP/tests/test_intranode.py, using `sglang:blackwell-cu129-arm64.sqsh`. - hybrid-ep branch + pr501, without --use-fabric, EP4 ``` [tuning] Best combine: SMs 24, NVL chunk 16: 351.05 GB/s (NVL), t: 602.89 us ``` - hybrid-ep branch + pr501, with --use-fabric, EP8 ``` [tuning] Best combine: SMs 24, NVL chunk 15: 417.83 GB/s (NVL), t: 748.14 us ``` ------------------- This pr also includes two tiny fixes for low_latency_dispatch, which are [Fix tensor descriptions in buffer.py](https://github.com/deepseek-ai/DeepEP/pull/498) and [Fix racing condition in large batch size](https://github.com/deepseek-ai/DeepEP/pull/440). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:04:25 +00:00