[GH-ISSUE #129] DeepSeek-R1 not outputting initial <think> token #70

New issue

Closed

opened 2026-10-29 14:33:35 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-29 14:33:35 +00:00

Owner

Originally created by @RodriMora on GitHub (Feb 10, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/129

I've been testing the latest update and it seems like it's not outputting the initial token in the output. It does output the final token though.

It used to work fine but I cannot reproduce it now. Tested with the same model files using llama.cpp and it works fine.

Originally created by @RodriMora on GitHub (Feb 10, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/129 I've been testing the latest update and it seems like it's not outputting the initial <think> token in the output. It does output the final </think> token though. It used to work fine but I cannot reproduce it now. Tested with the same model files using llama.cpp and it works fine.

bsakaguchi closed this issue

2026-10-29 14:33:35 +00:00

bsakaguchi commented

2026-10-29 14:33:36 +00:00

Author

Owner

@Atream commented on GitHub (Feb 11, 2025):

We have conducted tests. Although we did not see the token in the output, the model actually did perform thinking in its output. This is likely due to an incomplete template or the failure to successfully output the token. We are currently investigating this issue. Please wait a moment.

@Atream commented on GitHub (Feb 11, 2025): We have conducted tests. Although we did not see the <think> token in the output, the model actually did perform thinking in its output. This is likely due to an incomplete template or the failure to successfully output the <think> token. We are currently investigating this issue. Please wait a moment.

bsakaguchi commented

2026-10-29 14:33:36 +00:00

Author

Owner

@KMSorSMS commented on GitHub (Feb 11, 2025):

I've been testing the latest update and it seems like it's not outputting the initial token in the output. It does output the final token though.

It used to work fine but I cannot reproduce it now. Tested with the same model files using llama.cpp and it works fine.

The newest commit now supports an arg --force_think true to make sure the R1 thinking. The detail

from DeepSeek Official:

Additionally, we have observed that the DeepSeek-R1 series models tend to bypass thinking pattern (i.e., outputting "\n\n") when responding to certain queries, which can adversely affect the model's performance. To ensure that the model engages in thorough reasoning, we recommend enforcing the model to initiate its response with "\n" at the beginning of every output.

@KMSorSMS commented on GitHub (Feb 11, 2025): > I've been testing the latest update and it seems like it's not outputting the initial token in the output. It does output the final token though. > > It used to work fine but I cannot reproduce it now. Tested with the same model files using llama.cpp and it works fine. The newest commit now supports an arg `--force_think true` to make sure the R1 thinking. The [detail](https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/DeepseekR1_V3_tutorial.md#how-to-run) > from DeepSeek Official:<br> > Additionally, we have observed that the DeepSeek-R1 series models tend to bypass thinking pattern (i.e., outputting "<think>\n\n</think>") when responding to certain queries, which can adversely affect the model's performance. To ensure that the model engages in thorough reasoning, we recommend enforcing the model to initiate its response with "<think>\n" at the beginning of every output.

bsakaguchi commented

2026-10-29 14:33:36 +00:00

Author

Owner

@iammeizu commented on GitHub (Feb 13, 2025):

met the same problem，using sglang on 8*h20，any solution？

@iammeizu commented on GitHub (Feb 13, 2025): met the same problem，using sglang on 8*h20，any solution？

bsakaguchi commented

2026-10-29 14:33:36 +00:00

Author

Owner

@ArYuZzz commented on GitHub (Feb 13, 2025):

Is there any way to solve the problem under server mode?

@ArYuZzz commented on GitHub (Feb 13, 2025): Is there any way to solve the problem under server mode?

bsakaguchi commented

2026-10-29 14:33:37 +00:00

Author

Owner

@vibe-Chen commented on GitHub (Apr 13, 2025):

Is there any way to solve the problem under server mode?

If you meant to use .\server\main.py to initial a local server, just add the --force_think argument on startup.
answered with sha: github.com/kvcache-ai/ktransformers@6cbe044aae

@vibe-Chen commented on GitHub (Apr 13, 2025): > Is there any way to solve the problem under server mode? If you meant to use `.\server\main.py` to initial a local server, just add the `--force_think` argument on startup. answered with sha: https://github.com/kvcache-ai/ktransformers/commit/6cbe044aae0398cf2a27ae68b8a54390b4ad6bdd