[GH-ISSUE #636] [Question] Why is elastic dispatch limited to ~512 warps? #379

New issue

Open

opened 2026-10-26 09:59:47 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:59:47 +00:00

Owner

Originally created by @lixingyii on GitHub (May 19, 2026).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/636

Hi, I have a question about the dispatch warp limit in the elastic dispatch path.

In csrc/kernels/elastic/dispatch.hpp, num_dispatch_warps is limited by:

math::ceil_div(512, num_sms)

so the total number of dispatch/sender warps is kept around 512.

I tried removing this limit and observed that performance can degrade when using more SMs. For example, in a single-node NVLink setup (Ranks: 1 x 8), increasing num_sms caused dispatch bandwidth to drop instead of improve.

Do you know the main reason for this 512-warp limit? Is it due to a specific bottleneck, such as TMA/proxy engine pressure, NVLink/L2 contention, atomic counter contention, grid synchronization overhead, or something else? Or was 512 chosen mainly as an empirical tuning value based on benchmarking?

Any insight would be appreciated. Thanks!

Originally created by @lixingyii on GitHub (May 19, 2026). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/636 Hi, I have a question about the dispatch warp limit in the elastic dispatch path. In `csrc/kernels/elastic/dispatch.hpp`, `num_dispatch_warps` is limited by: math::ceil_div(512, num_sms) so the total number of dispatch/sender warps is kept around 512. I tried removing this limit and observed that performance can degrade when using more SMs. For example, in a single-node NVLink setup (`Ranks: 1 x 8`), increasing `num_sms` caused dispatch bandwidth to drop instead of improve. Do you know the main reason for this 512-warp limit? Is it due to a specific bottleneck, such as TMA/proxy engine pressure, NVLink/L2 contention, atomic counter contention, grid synchronization overhead, or something else? Or was 512 chosen mainly as an empirical tuning value based on benchmarking? Any insight would be appreciated. Thanks!

bsakaguchi referenced this issue

2026-10-27 21:03:50 +00:00

[PR #379] [MERGED] Fix: avoid floating point exception #542