[PR #784] [CLOSED] Skip unused source-order sorting for expanded dispatch #781

New issue

Closed

opened 2026-10-27 21:06:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:06:18 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/784
Author: @yuweih205
Created: 10/15/2026
Status: ❌ Closed

Base: main ← Head: perf/skip-unused-expanded-source-sort-1016

📝 Commits (1)

5189c07 Skip unused source-order sorting for expanded dispatch

📊 Changes

2 files changed (+31 additions, -13 deletions)

View changed files

📝 deep_ep/buffers/ep.py (+11 -7)
📝 tests/ep/test_ep.py (+20 -6)

📄 Description

EPHandle.deterministic_sort() computes a compact source-order permutation even when expanded dispatch never consumes it. This moves that sort into the compact branch and skips sorting entirely for cached expanded dispatch, where the copy epilogue already uses saved sorted slots. Fresh expanded dispatch keeps its stable expert/source sort and slot remapping.

The regression checks one sort for fresh dispatch and zero for cached expanded dispatch. It also fixes two existing repeated-dispatch comparisons that included undefined unused receive capacity; these now compare valid rows/assignments. The original failures were reproduced on unmodified upstream.

Validation used eight H200 GPUs per NVLink domain, Torch 2.13.0+cu132, CUDA 13.2 and loaded NCCL 2.32.3, with the unchanged native extension:

Independent input/route oracle: 12,288 rank-dispatch cases across baseline/candidate, BF16/FP8, compact/expanded, fresh/cached, CPU/GPU counts, padding, weights, empty/skewed routes, direct/deferred calls, and both record-stream policies.
Public EP suites completed with both CPU-sync modes; cached graph replay passed 64 rank/cases.
The production and public-test files in this PR are byte-identical to the final GPU-tested files. The independent oracle was inherited from the earlier run with identical production code.

Two independent EP8 nodes each ran five fresh processes over 32 cases, alternating baseline/candidate blocks. Geometric-mean dispatch-through-CUDA-completion wall time decreased 9.87% / 9.99%. Every case and repetition is retained; separate A/A controls show that some small gains overlap noise. This is serialized component latency, not training throughput or inter-node scaling.

Full protocol, per-case/per-process results, noise controls and reproducers.

Local verification: Python compilation, exact source-hash checks, benchmark execution-code equivalence, GPU-artifact coverage audit and git diff --check. GPU performance numbers are from the recorded October 15 runs, not a new run at the publication commit.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/784 **Author:** [@yuweih205](https://github.com/yuweih205) **Created:** 10/15/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `perf/skip-unused-expanded-source-sort-1016` --- ### 📝 Commits (1) - [`5189c07`](https://github.com/deepseek-ai/DeepEP/commit/5189c0784731754a1e210228bcdab4592cecf72c) Skip unused source-order sorting for expanded dispatch ### 📊 Changes **2 files changed** (+31 additions, -13 deletions) <details> <summary>View changed files</summary> 📝 `deep_ep/buffers/ep.py` (+11 -7) 📝 `tests/ep/test_ep.py` (+20 -6) </details> ### 📄 Description `EPHandle.deterministic_sort()` computes a compact source-order permutation even when expanded dispatch never consumes it. This moves that sort into the compact branch and skips sorting entirely for cached expanded dispatch, where the copy epilogue already uses saved sorted slots. Fresh expanded dispatch keeps its stable expert/source sort and slot remapping. The regression checks one sort for fresh dispatch and zero for cached expanded dispatch. It also fixes two existing repeated-dispatch comparisons that included undefined unused receive capacity; these now compare valid rows/assignments. The original failures were reproduced on unmodified upstream. Validation used eight H200 GPUs per NVLink domain, Torch 2.13.0+cu132, CUDA 13.2 and loaded NCCL 2.32.3, with the unchanged native extension: - Independent input/route oracle: 12,288 rank-dispatch cases across baseline/candidate, BF16/FP8, compact/expanded, fresh/cached, CPU/GPU counts, padding, weights, empty/skewed routes, direct/deferred calls, and both record-stream policies. - Public EP suites completed with both CPU-sync modes; cached graph replay passed 64 rank/cases. - The production and public-test files in this PR are byte-identical to the final GPU-tested files. The independent oracle was inherited from the earlier run with identical production code. Two independent EP8 nodes each ran five fresh processes over 32 cases, alternating baseline/candidate blocks. Geometric-mean dispatch-through-CUDA-completion wall time decreased **9.87% / 9.99%**. Every case and repetition is retained; separate A/A controls show that some small gains overlap noise. This is serialized component latency, not training throughput or inter-node scaling. [Full protocol, per-case/per-process results, noise controls and reproducers](https://gist.github.com/yuweih205/841a5fdcdf1c5ac9fd221d16fe334f71). Local verification: Python compilation, exact source-hash checks, benchmark execution-code equivalence, GPU-artifact coverage audit and `git diff --check`. GPU performance numbers are from the recorded October 15 runs, not a new run at the publication commit. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:06:18 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#781

No description provided.

Rows
Columns