deepseek-ai/DeepEP

Fork 0

mirror of https://github.com/deepseek-ai/DeepEP.git synced 2026-11-05 00:47:32 +00:00

[GH-ISSUE #170] Can DeepEP run correctly in cudaGraph mode? #147

New issue

Open

opened 2026-10-26 09:55:13 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-26 09:55:13 +00:00

Owner

Originally created by @ghostplant on GitHub (May 17, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/170

Two questions:

Can DeepEP run normally in cudaGraph mode?
Does DeepEP perform "dropless" MoE dispatch? (i.e. no token discarded if tokens are heavily routed to a limited number of experts)

Originally created by @ghostplant on GitHub (May 17, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/170 Two questions: 1. Can DeepEP run normally in cudaGraph mode? 2. Does DeepEP perform "dropless" MoE dispatch? (i.e. no token discarded if tokens are heavily routed to a limited number of experts)

bsakaguchi commented

2026-10-26 09:55:14 +00:00

Author

Owner

@LyricZhao commented on GitHub (May 22, 2025):

Yes, but only for the normal kernels;
Yes; If you want to drop tokens, you should perform at the gate (masking some topk_idx into -1), DeepEP supports ignoring -1 expert selection (no send for such cases).

@LyricZhao commented on GitHub (May 22, 2025): 1. Yes, but only for the normal kernels; 2. Yes; If you want to drop tokens, you should perform at the gate (masking some `topk_idx` into `-1`), DeepEP supports ignoring `-1` expert selection (no send for such cases).

bsakaguchi commented

2026-10-26 09:55:14 +00:00

Author

Owner

@ghostplant commented on GitHub (May 22, 2025):

@LyricZhao Thank you, for drop-less dispatch, will the utilization still be that fast when gating selection is imbalanced (e.g. all tokens routed to the same GPU)?

@ghostplant commented on GitHub (May 22, 2025): @LyricZhao Thank you, for drop-less dispatch, will the utilization still be that fast when gating selection is imbalanced (e.g. all tokens routed to the same GPU)?

bsakaguchi commented

2026-10-26 09:55:14 +00:00

Author

Owner

@LyricZhao commented on GitHub (May 23, 2025):

will the utilization still be that fast when gating selection is imbalanced

The overall performance will be bound at the imbalanced rank. In the terms of the imbalanced rank itself, the utilization should be full.

@LyricZhao commented on GitHub (May 23, 2025): > will the utilization still be that fast when gating selection is imbalanced The overall performance will be bound at the imbalanced rank. In the terms of the imbalanced rank itself, the utilization should be full.

bsakaguchi commented

2026-10-26 09:55:14 +00:00

Author

Owner

@alpha-baby commented on GitHub (Jun 24, 2025):

Yes, but only for the normal kernels;

In sglang, low-latancy kernel could run normally in cudaGraph Mode!

Why only for the normal kernels?
@LyricZhao

@alpha-baby commented on GitHub (Jun 24, 2025): > 1. Yes, but only for the normal kernels; In sglang, low-latancy kernel could run normally in cudaGraph Mode! Why only for the normal kernels? @LyricZhao

bsakaguchi commented

2026-10-26 09:55:15 +00:00

Author

Owner

@MaoZiming commented on GitHub (Feb 20, 2026):

+1
just want to follow up to see if there is some effort to support cuda graph for low-latency kernel. Thanks!

@MaoZiming commented on GitHub (Feb 20, 2026): +1 just want to follow up to see if there is some effort to support cuda graph for low-latency kernel. Thanks!

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#147

No description provided.

Rows
Columns