[GH-ISSUE #343] 2x performance drop due to intrahost imbalance. #240

New issue

Closed

opened 2026-10-26 09:57:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:57:11 +00:00

Owner

Originally created by @isayoften on GitHub (Jul 30, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/343

I took your test_internode.py script and added a print statement to display the global tokens_per_expert distribution:

Using EP=32, num_experts=256, top_k=8, num_tokens=4096, and hidden_size=7168, I obtained the following distribution:

I also visualized the data grouped by rank and host:

As you can see, the distribution is nearly uniform, which leads to optimal performance.

Next, I simulated an imbalanced distribution that is balanced across nodes but imbalanced across GPUs within each host using the following code:

Here are the resulting token distributions:

This scenario results in nearly a 2x drop in performance:

Does this match your expectations?

Originally created by @isayoften on GitHub (Jul 30, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/343 I took your test_internode.py script and added a print statement to display the global tokens_per_expert distribution: <img width="580" height="112" alt="Image" src="https://github.com/user-attachments/assets/9cfc2128-8baf-4cc5-8d3c-b637a669e3bf" /> Using EP=32, num_experts=256, top_k=8, num_tokens=4096, and hidden_size=7168, I obtained the following distribution: <img width="500" height="400" alt="Image" src="https://github.com/user-attachments/assets/e026b773-daf1-4803-a9d8-f1ebe2e091a3" /> I also visualized the data grouped by rank and host: <img width="500" height="400" alt="Image" src="https://github.com/user-attachments/assets/745fe228-ecee-45ae-bc00-55c7fe505288" /> <img width="500" height="400" alt="Image" src="https://github.com/user-attachments/assets/8bdfa47c-ea53-46f4-877c-eaa65bdebbd7" /> <img width="1468" height="286" alt="Image" src="https://github.com/user-attachments/assets/2bcb8789-81e9-4c94-939b-2e2b23b152ad" /> As you can see, the distribution is nearly uniform, which leads to optimal performance. Next, I simulated an imbalanced distribution that is balanced across nodes but imbalanced across GPUs within each host using the following code: <img width="913" height="829" alt="Image" src="https://github.com/user-attachments/assets/536da31f-fa3b-463f-950b-0a041ba9f9da" /> Here are the resulting token distributions: <img width="500" height="400" alt="Image" src="https://github.com/user-attachments/assets/5e87ebac-9fa9-46f9-be76-d5fde154958e" /> <img width="500" height="400" alt="Image" src="https://github.com/user-attachments/assets/533130f6-e6c4-4e27-8bdc-a3b52b1f07c2" /> This scenario results in nearly a 2x drop in performance: <img width="1478" height="294" alt="Image" src="https://github.com/user-attachments/assets/58588ef1-d762-4341-8f55-8203f222af35" /> Does this match your expectations?

bsakaguchi closed this issue

2026-10-26 09:57:12 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#240

No description provided.

Rows
Columns