[GH-ISSUE #159] question about the modification of "pybind11::gil_scoped_release release" #137

New issue

Open

opened 2026-10-26 09:55:05 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:55:05 +00:00

Owner

Originally created by @leehongming on GitHub (May 13, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/159

Hi,
A few days ago, the modification of "pybind11::gil_scoped_release release;" is added to Buffer::internode_dispatch, and you guys give some comments on it. In my understanding, the modification is supposed to optimize the performance of the inference system as the other threads such as kv cache is only stucked for a moment, not forever.
Is there such a possibility that without the modification of gil release, a dispatch timeout is going to happen because of deadlock? Have you ever encountered this kind of problem?

Originally created by @leehongming on GitHub (May 13, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/159 Hi, A few days ago, the modification of "pybind11::gil_scoped_release release;" is added to Buffer::internode_dispatch, and you guys give some comments on it. In my understanding, the modification is supposed to optimize the performance of the inference system as the other threads such as kv cache is only stucked for a moment, not forever. Is there such a possibility that without the modification of gil release, a dispatch timeout is going to happen because of deadlock? Have you ever encountered this kind of problem?

bsakaguchi commented

2026-10-26 09:55:06 +00:00

Author

Owner

@LyricZhao commented on GitHub (May 14, 2025):

Is there such a possibility that without the modification of gil release, a dispatch timeout is going to happen because of deadlock?

With GIL consideration, timeout only occurs with some ranks launched comm kernels, while some ranks not. I guess it really depends on how you implement your framework. I can not tell an exact answer about this.

Have you ever encountered this kind of problem?

No, internally at DeepSeek.

This PR is introduced by the SGLang team, @fzyzcjy do you have something to comment?

@LyricZhao commented on GitHub (May 14, 2025): > Is there such a possibility that without the modification of gil release, a dispatch timeout is going to happen because of deadlock? With GIL consideration, timeout only occurs with some ranks launched comm kernels, while some ranks not. I guess it really depends on how you implement your framework. I can not tell an exact answer about this. > Have you ever encountered this kind of problem? No, internally at DeepSeek. This PR is introduced by the SGLang team, @fzyzcjy do you have something to comment?

bsakaguchi commented

2026-10-26 09:55:06 +00:00

Author

Owner

@fzyzcjy commented on GitHub (May 14, 2025):

I have no more to comment

@fzyzcjy commented on GitHub (May 14, 2025): I have no more to comment

bsakaguchi referenced this issue

2026-10-26 10:00:54 +00:00

[PR #181] [MERGED] Fix dispatch timeout problem: concurrent submissions of qp should be forbidden. #452