[PR #785] [CLOSED] [RFC] Benchmark valid-prefix sorting and count prefetch for compact dispatch #783

New issue

Closed

opened 2026-10-27 21:06:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-27 21:06:18 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/deepseek-ai/DeepEP/pull/785
Author: @yuweih205
Created: 10/15/2026
Status: ❌ Closed

Base: main ← Head: experiments/valid-prefix-sort-prefetch-1016

📝 Commits (1)

1037538 Add valid-prefix sort and count-prefetch experiment

📊 Changes

3 files changed (+827 additions, -0 deletions)

View changed files

➕ tests/ep/bench_prefix_sort.py (+456 -0)
➕ tests/ep/prefix_sort_experiment.md (+44 -0)
➕ tests/ep/prefix_sort_results.json (+327 -0)

📄 Description

Add a standalone experimental benchmark that separates valid-prefix deterministic sorting from early receive-count copies for compact BF16 dispatch. It compares four factorial arms with both native sync-free and native deferred materialization, using the same independent GEMM work and synchronized end-to-end readiness timing.

This is a draft experiment/benchmark PR, with no production API or default change. On eight H200 GPUs, all eight measured scenarios remain slower than native deferred. Valid-prefix sorting helps relative to the matching full-capacity sort, while early count copies do not provide a consistent additional benefit. The documentation retains all six arms, all cases and all three fresh-process summaries.

The existing GPU study covers 1920 correctness/smoke rank/cases plus 1152 full-performance preflight rank/cases: independent dispatch-reference equality, counts, combine, cached replay, saved-view backward, both allocation streams, both record-stream policies, and uneven/duplicate/skewed/masked routes. The published execution code matches the measured script; only its docstring and historical-source log label changed. Local compilation, AST equivalence, evidence-count checks and git diff --check passed before publication.

See tests/ep/prefix_sort_experiment.md for reproduction, complete results and scope. This is a one-NVLink-domain compact BF16 experiment; it does not establish full-training gains or cover FP8/expanded/hybrid dispatch.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/deepseek-ai/DeepEP/pull/785 **Author:** [@yuweih205](https://github.com/yuweih205) **Created:** 10/15/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `experiments/valid-prefix-sort-prefetch-1016` --- ### 📝 Commits (1) - [`1037538`](https://github.com/deepseek-ai/DeepEP/commit/1037538e48d92047584c2807e4d6cda26d64dd50) Add valid-prefix sort and count-prefetch experiment ### 📊 Changes **3 files changed** (+827 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `tests/ep/bench_prefix_sort.py` (+456 -0) ➕ `tests/ep/prefix_sort_experiment.md` (+44 -0) ➕ `tests/ep/prefix_sort_results.json` (+327 -0) </details> ### 📄 Description Add a standalone experimental benchmark that separates valid-prefix deterministic sorting from early receive-count copies for compact BF16 dispatch. It compares four factorial arms with both native sync-free and native deferred materialization, using the same independent GEMM work and synchronized end-to-end readiness timing. This is a **draft experiment/benchmark PR**, with no production API or default change. On eight H200 GPUs, all eight measured scenarios remain slower than native deferred. Valid-prefix sorting helps relative to the matching full-capacity sort, while early count copies do not provide a consistent additional benefit. The documentation retains all six arms, all cases and all three fresh-process summaries. The existing GPU study covers 1920 correctness/smoke rank/cases plus 1152 full-performance preflight rank/cases: independent dispatch-reference equality, counts, combine, cached replay, saved-view backward, both allocation streams, both record-stream policies, and uneven/duplicate/skewed/masked routes. The published execution code matches the measured script; only its docstring and historical-source log label changed. Local compilation, AST equivalence, evidence-count checks and `git diff --check` passed before publication. See `tests/ep/prefix_sort_experiment.md` for reproduction, complete results and scope. This is a one-NVLink-domain compact BF16 experiment; it does not establish full-training gains or cover FP8/expanded/hybrid dispatch. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-27 21:06:18 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#783

No description provided.

Rows
Columns