deepseek-ai/DeepEP
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00
Code Issues 308 Projects Packages Wiki Activity

[PR #232] [CLOSED] Optimize get_dispatch_layout from 170us to 45us #475

New issue
Closed
opened 2026-10-26 10:01:13 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 10:01:13 +00:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/232
Author: @fzyzcjy
Created: 6/20/2025
Status: ❌ Closed

Base: main ← Head: feat/opt_get_dispatch_layout

📝 Commits (10+)

📊 Changes

10 files changed (+249 additions, -22 deletions)

View changed files

📝 csrc/deep_ep.cpp (+135 -9)
📝 csrc/deep_ep.hpp (+33 -1)
📝 csrc/kernels/configs.cuh (+7 -0)
📝 csrc/kernels/exception.cuh (+12 -0)
📝 csrc/kernels/internode.cu (+6 -0)
📝 csrc/kernels/launch.cuh (+2 -0)
📝 csrc/kernels/layout.cu (+34 -3)
📝 setup.py (+1 -1)
📝 tests/test_intranode.py (+16 -7)
📝 tests/utils.py (+3 -1)

📄 Description

Quickly tested:

before

[layout] Kernel performance: 0.170 ms
[layout] Kernel performance: 0.178 ms

after

[layout] Kernel performance: 0.045 ms
[layout] Kernel performance: 0.046 ms

core change is simple, just change num SM.

The code diff is huge since I inherited code change from https://github.com/deepseek-ai/DeepEP/pull/218 but that's mostly unnecessary.

I will try to find out some time later to cleanup the code (eg still use few SM when need two stream overlap, and only use this one when do not enable two stream overlap etc), and here just quickly PR to know whether you think this looks acceptable and if anyone happens to need this then can copy-paste.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/232 **Author:** [@fzyzcjy](https://github.com/fzyzcjy) **Created:** 6/20/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/opt_get_dispatch_layout` --- ### 📝 Commits (10+) - [`be2eed8`](https://github.com/deepseek-ai/DeepEP/commit/be2eed8f6c40e67f17f0dd9335d6ddb113713af4) more - [`9ecb941`](https://github.com/deepseek-ai/DeepEP/commit/9ecb9411c719badba8814cb92f9d4bf2331f0a87) more - [`9683d94`](https://github.com/deepseek-ai/DeepEP/commit/9683d9453ed9132b288c46b4232067707db29a49) more - [`8cf6bd8`](https://github.com/deepseek-ai/DeepEP/commit/8cf6bd88ea51f993da7a177c27826c91111de7c9) more - [`acf108a`](https://github.com/deepseek-ai/DeepEP/commit/acf108ab4818d306fe973a010db6b76de40ef17d) more - [`3e2cede`](https://github.com/deepseek-ai/DeepEP/commit/3e2cede81cf741c6867af3f97f36a9af943c8d71) Revert "more" - [`45fa1af`](https://github.com/deepseek-ai/DeepEP/commit/45fa1af2428c56b2a2ec6df3b28904c687003e71) Revert "more" - [`443bfa8`](https://github.com/deepseek-ai/DeepEP/commit/443bfa8692c945539bec21e2b910fc22f67f7bc0) more - [`b986cce`](https://github.com/deepseek-ai/DeepEP/commit/b986cce27bf39167488415fefcf0ee2dfee24e0c) more - [`3ea6f58`](https://github.com/deepseek-ai/DeepEP/commit/3ea6f58265cfccec4112bc00a195458dda5503d5) more ### 📊 Changes **10 files changed** (+249 additions, -22 deletions) <details> <summary>View changed files</summary> 📝 `csrc/deep_ep.cpp` (+135 -9) 📝 `csrc/deep_ep.hpp` (+33 -1) 📝 `csrc/kernels/configs.cuh` (+7 -0) 📝 `csrc/kernels/exception.cuh` (+12 -0) 📝 `csrc/kernels/internode.cu` (+6 -0) 📝 `csrc/kernels/launch.cuh` (+2 -0) 📝 `csrc/kernels/layout.cu` (+34 -3) 📝 `setup.py` (+1 -1) 📝 `tests/test_intranode.py` (+16 -7) 📝 `tests/utils.py` (+3 -1) </details> ### 📄 Description Quickly tested: before ``` [layout] Kernel performance: 0.170 ms [layout] Kernel performance: 0.178 ms ``` after ``` [layout] Kernel performance: 0.045 ms [layout] Kernel performance: 0.046 ms ``` core change is simple, just change num SM. The code diff is huge since I inherited code change from https://github.com/deepseek-ai/DeepEP/pull/218 but that's mostly unnecessary. I will try to find out some time later to cleanup the code (eg still use few SM when need two stream overlap, and only use this one when do not enable two stream overlap etc), and here just quickly PR to know whether you think this looks acceptable and if anyone happens to need this then can copy-paste. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-26 10:01:13 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
hybrid-ep
chore/remove-clang-format-yapf
v2.5
nvDev
try-fix-cu128
mori-ep
antgroup-opt
compat-old-torch
add-linter
eager-rdma
v1.2.1
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepEP#475
No description provided.