[PR #249] Computation communication overlap #485

New issue

Open

opened 2026-10-27 21:02:59 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:02:59 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/249
Author: @fzyzcjy
Created: 6/24/2025
Status: 🔄 Open

Base: main ← Head: feat/dev_20250621

📝 Commits (10+)

db53053 more
2278722 more
661e188 more
5567637 more
20855ee more
ad11318 more
a960335 more
9a8d98a Merge branch 'feat/test_detailed_time' into feat/dev_20250621
681bdc5 cherry pick
7421672 Merge branch 'feat/num_processes' into feat/dev_20250621

📊 Changes

12 files changed (+1299 additions, -299 deletions)

View changed files

📝 csrc/deep_ep.cpp (+11 -2)
📝 csrc/deep_ep.hpp (+4 -1)
📝 csrc/kernels/api.cuh (+2 -1)
📝 csrc/kernels/exception.cuh (+12 -0)
📝 csrc/kernels/internode_ll.cu (+413 -265)
📝 deep_ep/buffer.py (+3 -2)
📝 setup.py (+5 -1)
📝 tests/test_internode.py (+1 -1)
📝 tests/test_intranode.py (+15 -7)
📝 tests/test_low_latency.py (+136 -16)
➕ tests/test_moe_layer.py (+642 -0)
📝 tests/utils.py (+55 -3)

📄 Description

WIP

Some quick dump:

Can use few SMs to run communication while most SMs to run computation. DeepEP with few SMs in low latency mode will almost still run in full speed.
- The change is pretty simple to support few-SM, just change num_warp_groups etc
- I can provide more info and experiment results if needed
I am waiting for fp4 kernel before doing more computation communication overlap, that's why this draft is pending and no progress at all
- but anyway the code here does show overlap and speedup b/t combine send and (the fp8 DeepGEMM) down gemm

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/249 **Author:** [@fzyzcjy](https://github.com/fzyzcjy) **Created:** 6/24/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/dev_20250621` --- ### 📝 Commits (10+) - [`db53053`](https://github.com/deepseek-ai/DeepEP/commit/db53053fe8ef86e644914436c71f23ed81ec1ffa) more - [`2278722`](https://github.com/deepseek-ai/DeepEP/commit/227872271bec60b14e59090c24480bbbbd5a0d82) more - [`661e188`](https://github.com/deepseek-ai/DeepEP/commit/661e188595ad9be9f0eb8ce7998e224bc7b9b60d) more - [`5567637`](https://github.com/deepseek-ai/DeepEP/commit/55676370a92da29f5a2c01ea92f2acc975e590ba) more - [`20855ee`](https://github.com/deepseek-ai/DeepEP/commit/20855ee5e1dbf8c15c323e76d5f75e46a8579359) more - [`ad11318`](https://github.com/deepseek-ai/DeepEP/commit/ad113181103973c45a829652937746c8f7f0dd9c) more - [`a960335`](https://github.com/deepseek-ai/DeepEP/commit/a9603352a2efaf1ec860032ab15107b1f3887fd9) more - [`9a8d98a`](https://github.com/deepseek-ai/DeepEP/commit/9a8d98a94a5326206f165d9c8970628d7ff3aef6) Merge branch 'feat/test_detailed_time' into feat/dev_20250621 - [`681bdc5`](https://github.com/deepseek-ai/DeepEP/commit/681bdc5a73f5702d39d77b498e71dc0a9ec439ff) cherry pick - [`7421672`](https://github.com/deepseek-ai/DeepEP/commit/7421672d672c4db94d8601061ff060625b597801) Merge branch 'feat/num_processes' into feat/dev_20250621 ### 📊 Changes **12 files changed** (+1299 additions, -299 deletions) <details> <summary>View changed files</summary> 📝 `csrc/deep_ep.cpp` (+11 -2) 📝 `csrc/deep_ep.hpp` (+4 -1) 📝 `csrc/kernels/api.cuh` (+2 -1) 📝 `csrc/kernels/exception.cuh` (+12 -0) 📝 `csrc/kernels/internode_ll.cu` (+413 -265) 📝 `deep_ep/buffer.py` (+3 -2) 📝 `setup.py` (+5 -1) 📝 `tests/test_internode.py` (+1 -1) 📝 `tests/test_intranode.py` (+15 -7) 📝 `tests/test_low_latency.py` (+136 -16) ➕ `tests/test_moe_layer.py` (+642 -0) 📝 `tests/utils.py` (+55 -3) </details> ### 📄 Description WIP --- Some quick dump: * Can use few SMs to run communication while most SMs to run computation. DeepEP with few SMs in low latency mode will almost still run in full speed. * The change is pretty simple to support few-SM, just change num_warp_groups etc * I can provide more info and experiment results if needed * I am waiting for fp4 kernel before doing more computation communication overlap, that's why this draft is pending and no progress at all * but anyway the code here does show overlap and speedup b/t combine send and (the fp8 DeepGEMM) down gemm --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-27 21:02:59 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#485

No description provided.

Rows
Columns