[GH-ISSUE #233] hidden size 2048 of Qwen3-30B-A3B is not supported #181

New issue

Closed

opened 2026-10-26 09:55:58 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 09:55:58 +00:00

Owner

Originally created by @GuoxiangZu on GitHub (Jun 20, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/233

When we run the sglang inference with --enable-deepep-moe on Qwen3-30B-A3B, the following error occurs:

raceback (most recent call last):
  File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 2546, in run_scheduler_process
    scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, pp_rank, dp_rank)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 312, in __init__
    self.tp_worker = TpWorkerClass(
                     ^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/managers/tp_worker_overlap_thread.py", line 64, in __init__
    self.worker = TpModelWorker(
                  ^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/managers/tp_worker.py", line 78, in __init__
    self.model_runner = ModelRunner(
                        ^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 212, in __init__
    self.initialize(min_per_gpu_memory)
  File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 287, in initialize
    self.init_cuda_graphs()
  File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 1138, in init_cuda_graphs
    self.cuda_graph_runner = CudaGraphRunner(self)
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/model_executor/cuda_graph_runner.py", line 320, in __init__
    raise Exception(
Exception: Capture cuda graph failed: Failed: Assertion error /sgl-workspace/DeepEP/csrc/kernels/internode_ll.cu:381 'false && "Unsupported hidden"'

It is due to hidden size 2048 is not supported yet.

Originally created by @GuoxiangZu on GitHub (Jun 20, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/233 When we run the sglang inference with `--enable-deepep-moe` on Qwen3-30B-A3B, the following error occurs: ``` raceback (most recent call last): File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 2546, in run_scheduler_process scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, pp_rank, dp_rank) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 312, in __init__ self.tp_worker = TpWorkerClass( ^^^^^^^^^^^^^^ File "/sgl-workspace/sglang/python/sglang/srt/managers/tp_worker_overlap_thread.py", line 64, in __init__ self.worker = TpModelWorker( ^^^^^^^^^^^^^^ File "/sgl-workspace/sglang/python/sglang/srt/managers/tp_worker.py", line 78, in __init__ self.model_runner = ModelRunner( ^^^^^^^^^^^^ File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 212, in __init__ self.initialize(min_per_gpu_memory) File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 287, in initialize self.init_cuda_graphs() File "/sgl-workspace/sglang/python/sglang/srt/model_executor/model_runner.py", line 1138, in init_cuda_graphs self.cuda_graph_runner = CudaGraphRunner(self) ^^^^^^^^^^^^^^^^^^^^^ File "/sgl-workspace/sglang/python/sglang/srt/model_executor/cuda_graph_runner.py", line 320, in __init__ raise Exception( Exception: Capture cuda graph failed: Failed: Assertion error /sgl-workspace/DeepEP/csrc/kernels/internode_ll.cu:381 'false && "Unsupported hidden"' ``` It is due to hidden size 2048 is not supported yet.

bsakaguchi closed this issue

2026-10-26 09:55:58 +00:00

bsakaguchi commented

2026-10-26 09:55:58 +00:00

Author

Owner

@alpha-baby commented on GitHub (Jun 20, 2025):

github.com/deepseek-ai/DeepEP@9d4f7ef8ee/csrc/kernels/launch.cuh (L85-L92)

you can try add one line code, like:

 #define SWITCH_HIDDEN(case_macro) \ 
     switch (hidden) { \ 
         case 2048: case_macro(2048); \ 
         case 2560: case_macro(2560); \ 
         case 4096: case_macro(4096); \ 
         case 5120: case_macro(5120); \ 
         case 7168: case_macro(7168); \ 
         default: EP_HOST_ASSERT(false && "Unsupported hidden"); \ 
     } while (false)

@alpha-baby commented on GitHub (Jun 20, 2025): https://github.com/deepseek-ai/DeepEP/blob/9d4f7ef8eeedd9970c2bb8efe998e07e7bb9df5b/csrc/kernels/launch.cuh#L85-L92 you can try add one line code, like: ``` #define SWITCH_HIDDEN(case_macro) \ switch (hidden) { \ case 2048: case_macro(2048); \ case 2560: case_macro(2560); \ case 4096: case_macro(4096); \ case 5120: case_macro(5120); \ case 7168: case_macro(7168); \ default: EP_HOST_ASSERT(false && "Unsupported hidden"); \ } while (false) ```

bsakaguchi commented

2026-10-26 09:55:59 +00:00

Author

Owner

@LyricZhao commented on GitHub (Jun 20, 2025):

Fixed in github.com/deepseek-ai/DeepEP@7b0c25f864, thanks for pointing it out.

@LyricZhao commented on GitHub (Jun 20, 2025): Fixed in https://github.com/deepseek-ai/DeepEP/commit/7b0c25f864cc7a2d95d7e3c3c2b7c13b22116063, thanks for pointing it out.

bsakaguchi referenced this issue

2026-10-26 09:56:28 +00:00

[GH-ISSUE #270] A timeout error occurred when running test_internode.py on four devices. #200

bsakaguchi referenced this issue

2026-10-26 10:00:54 +00:00

[PR #181] [MERGED] Fix dispatch timeout problem: concurrent submissions of qp should be forbidden. #452