[GH-ISSUE #1893] 如何配置参数让 Qwen3.5-397B-A17B 模型按 Thinking Process 标准格式输出 #1181

New issue

Closed

opened 2026-10-29 14:49:38 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:49:38 +00:00

Owner

Originally created by @xiaolvtongxue-zt on GitHub (Mar 19, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1893

Reminder

I have read the above rules and searched the existing issues.

System Info

通过测试，使用
https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/Qwen3.5.md的文档介绍，启动了模型。Qwen3.5-397B-A17B。

python -m sglang.launch_server \
  --host 0.0.0.0 \
  --port 30005 \
  --model /path/to/qwen3.5 \
  --kt-weight-path /path/to/qwen3.5 \
  --kt-cpuinfer 60 \
  --kt-threadpool-count 2 \
  --kt-num-gpu-experts 1 \
  --kt-method BF16 \
  --attention-backend triton \
  --trust-remote-code \
  --mem-fraction-static 0.98 \
  --chunked-prefill-size 4096 \
  --max-running-requests 32 \
  --max-total-tokens 32000 \
  --served-model-name qwen3.5 \
  --enable-mixed-chunk \
  --tensor-parallel-size 4 \
  --enable-p2p-check \
  --disable-shared-experts-fusion \
  --disable-custom-all-reduce

但这个模型是思考跟非思考是一起的。直接调用的话，模型不能按照标准的思考来输出。

Thinking Process: **** [ENDTHINKFLAG]

这个要加入哪个参数才能适配回来？我通过介绍文档，没能找到对应的参数。

Reproduction

Thinking Process: **** [ENDTHINKFLAG]

Others

No response

Originally created by @xiaolvtongxue-zt on GitHub (Mar 19, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1893 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info 通过测试，使用 https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/Qwen3.5.md的文档介绍，启动了模型。Qwen3.5-397B-A17B。 ``` python -m sglang.launch_server \ --host 0.0.0.0 \ --port 30005 \ --model /path/to/qwen3.5 \ --kt-weight-path /path/to/qwen3.5 \ --kt-cpuinfer 60 \ --kt-threadpool-count 2 \ --kt-num-gpu-experts 1 \ --kt-method BF16 \ --attention-backend triton \ --trust-remote-code \ --mem-fraction-static 0.98 \ --chunked-prefill-size 4096 \ --max-running-requests 32 \ --max-total-tokens 32000 \ --served-model-name qwen3.5 \ --enable-mixed-chunk \ --tensor-parallel-size 4 \ --enable-p2p-check \ --disable-shared-experts-fusion \ --disable-custom-all-reduce ``` 但这个模型是思考跟非思考是一起的。直接调用的话，模型不能按照标准的思考来输出。 ``` Thinking Process: **** [ENDTHINKFLAG] ``` 这个要加入哪个参数才能适配回来？我通过介绍文档，没能找到对应的参数。 ### Reproduction ```text Thinking Process: **** [ENDTHINKFLAG] ``` ### Others _No response_

bsakaguchi closed this issue

2026-10-29 14:49:38 +00:00

bsakaguchi commented

2026-10-29 14:49:39 +00:00

Author

Owner

@chenht2022 commented on GitHub (Mar 19, 2026):

Maybe try --reasoning-parser qwen3_coder

@chenht2022 commented on GitHub (Mar 19, 2026): Maybe try `--reasoning-parser qwen3_coder`

bsakaguchi commented

2026-10-29 14:49:39 +00:00

Author