[PR #763] [MERGED] DeepEP V2.5 #765

New issue

Closed

opened 2026-10-27 21:06:08 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:06:08 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/763
Author: @LyricZhao
Created: 9/23/2026
Status: ✅ Merged
Merged: 9/29/2026
Merged by: @LyricZhao

Base: main ← Head: v2.5

📝 Commits (2)

1a1ffba DeepEP V2.5
8c1d13a remove warpget opt

📊 Changes

163 files changed (+11700 additions, -17402 deletions)

View changed files

📝 .gitmodules (+3 -3)
📝 CMakeLists.txt (+7 -13)
📝 README.md (+272 -192)
➕ csrc/buffers/base.hpp (+63 -0)
➕ csrc/buffers/bucket.hpp (+374 -0)
➕ csrc/buffers/engram.hpp (+319 -0)
➕ csrc/buffers/ep.hpp (+1100 -0)
➕ csrc/buffers/pp.hpp (+148 -0)
➕ csrc/comm/api.hpp (+5 -0)
➕ csrc/comm/barrier.hpp (+44 -0)
➕ csrc/comm/handle.hpp (+32 -0)
➕ csrc/comm/stream.hpp (+29 -0)
➖ csrc/elastic/buffer.hpp (+0 -1382)
📝 csrc/indexing/main.cu (+26 -13)
➖ csrc/jit/api.hpp (+0 -20)
➖ csrc/jit/cache.hpp (+0 -34)
➖ csrc/jit/compiler.hpp (+0 -269)
➖ csrc/jit/device_runtime.hpp (+0 -67)
➖ csrc/jit/handle.hpp (+0 -166)
➖ csrc/jit/include_parser.hpp (+0 -80)

...and 80 more files

📄 Description

DeepEP V2.5 introduces dedicated communication buffers for EP, PP, Engram, and bucket collectives, together with new primitives for redundant-expert load balancing. GPU kernels are now compiled through DeepJIT, while installation builds the host C++ extension.

This release also fully removes V1 and its NVSHMEM dependency.

New Features

Dedicated buffer APIs @LyricZhao
- Split ElasticBuffer into EPBuffer, EngramBuffer, PPBuffer, and BucketBuffer, sharing the BufferBase lifecycle.
- Add BufferAllocator for planning symmetric tensor allocations before buffer construction.
Expert load balancing @cqzhangyu @LyricZhao
- Add EPBuffer.lb_prefetch_weights to transfer expert weights and quantization scales to redundant expert slots over NVLink.
- Add EPBuffer.lb_reduce_grads to accumulate redundant experts’ FP32 gradients into the original experts.
- These primitives support the dynamic expert-replication approach explored by MoonEP and UltraEP. Replication planning and token rerouting are supplied by the integrating framework.
Bucket collectives for CP and DP @jason-huang03 @bucket-xv @cqzhangyu @dromniscience @sphish
- Batched all-gather, reduce-scatter, and all-reduce across NVLink, RDMA, and hybrid domains.
- Sessions for communicating ordinary PyTorch tensors through temporary bucket storage.
EP improvements @LyricZhao
- Deferred dispatch/combine epilogues for communication-computation overlap.
- Cached expanded layouts and zero padding between experts.
Engram @tianr22 @sphish
- Multi-layer storage on GPU or CPU, with one completion hook per layer.
Build and runtime @LyricZhao
- Move GPU kernel compilation to DeepJIT.
- Remove installation-time NVCC compilation and GPU architecture selection.

Notes

Migrate from ElasticBuffer to the dedicated buffer APIs.
V1 APIs, legacy documentation, and NVSHMEM support have been removed.
Bucket collectives, Engram, and PP remain experimental.
Requires NVIDIA Hopper or newer GPUs, CUDA 13.1+, NCCL 2.32.3+, PyTorch 2.10+ with CUDA support, Python 3.10+, and a C++20 compiler.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/763 **Author:** [@LyricZhao](https://github.com/LyricZhao) **Created:** 9/23/2026 **Status:** ✅ Merged **Merged:** 9/29/2026 **Merged by:** [@LyricZhao](https://github.com/LyricZhao) **Base:** `main` ← **Head:** `v2.5` --- ### 📝 Commits (2) - [`1a1ffba`](https://github.com/deepseek-ai/DeepEP/commit/1a1ffbac1f9ba94e1f731f7647b5d6353fb38ade) DeepEP V2.5 - [`8c1d13a`](https://github.com/deepseek-ai/DeepEP/commit/8c1d13a89b8fd0ba09fc631bd6a29aef37b7602d) remove warpget opt ### 📊 Changes **163 files changed** (+11700 additions, -17402 deletions) <details> <summary>View changed files</summary> 📝 `.gitmodules` (+3 -3) 📝 `CMakeLists.txt` (+7 -13) 📝 `README.md` (+272 -192) ➕ `csrc/buffers/base.hpp` (+63 -0) ➕ `csrc/buffers/bucket.hpp` (+374 -0) ➕ `csrc/buffers/engram.hpp` (+319 -0) ➕ `csrc/buffers/ep.hpp` (+1100 -0) ➕ `csrc/buffers/pp.hpp` (+148 -0) ➕ `csrc/comm/api.hpp` (+5 -0) ➕ `csrc/comm/barrier.hpp` (+44 -0) ➕ `csrc/comm/handle.hpp` (+32 -0) ➕ `csrc/comm/stream.hpp` (+29 -0) ➖ `csrc/elastic/buffer.hpp` (+0 -1382) 📝 `csrc/indexing/main.cu` (+26 -13) ➖ `csrc/jit/api.hpp` (+0 -20) ➖ `csrc/jit/cache.hpp` (+0 -34) ➖ `csrc/jit/compiler.hpp` (+0 -269) ➖ `csrc/jit/device_runtime.hpp` (+0 -67) ➖ `csrc/jit/handle.hpp` (+0 -166) ➖ `csrc/jit/include_parser.hpp` (+0 -80) _...and 80 more files_ </details> ### 📄 Description DeepEP V2.5 introduces dedicated communication buffers for EP, PP, Engram, and bucket collectives, together with new primitives for redundant-expert load balancing. GPU kernels are now compiled through DeepJIT, while installation builds the host C++ extension. This release also fully removes V1 and its NVSHMEM dependency. ### New Features - Dedicated buffer APIs @LyricZhao - Split ElasticBuffer into EPBuffer, EngramBuffer, PPBuffer, and BucketBuffer, sharing the BufferBase lifecycle. - Add BufferAllocator for planning symmetric tensor allocations before buffer construction. - Expert load balancing @cqzhangyu @LyricZhao - Add EPBuffer.lb_prefetch_weights to transfer expert weights and quantization scales to redundant expert slots over NVLink. - Add EPBuffer.lb_reduce_grads to accumulate redundant experts’ FP32 gradients into the original experts. - These primitives support the dynamic expert-replication approach explored by MoonEP and UltraEP. Replication planning and token rerouting are supplied by the integrating framework. - Bucket collectives for CP and DP @jason-huang03 @bucket-xv @cqzhangyu @dromniscience @sphish - Batched all-gather, reduce-scatter, and all-reduce across NVLink, RDMA, and hybrid domains. - Sessions for communicating ordinary PyTorch tensors through temporary bucket storage. - EP improvements @LyricZhao - Deferred dispatch/combine epilogues for communication-computation overlap. - Cached expanded layouts and zero padding between experts. - Engram @tianr22 @sphish - Multi-layer storage on GPU or CPU, with one completion hook per layer. - Build and runtime @LyricZhao - Move GPU kernel compilation to DeepJIT. - Remove installation-time NVCC compilation and GPU architecture selection. ### Notes - Migrate from ElasticBuffer to the dedicated buffer APIs. - V1 APIs, legacy documentation, and NVSHMEM support have been removed. - Bucket collectives, Engram, and PP remain experimental. - Requires NVIDIA Hopper or newer GPUs, CUDA 13.1+, NCCL 2.32.3+, PyTorch 2.10+ with CUDA support, Python 3.10+, and a C++20 compiler. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:06:08 +00:00