[GH-ISSUE #146] 模型输出结果会忽略掉开头的<think> #80

New issue

Closed

opened 2026-10-29 14:33:45 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:33:45 +00:00

Owner

Originally created by @optimustian on GitHub (Feb 11, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/146

模型：DeepSeek R1-Q4-K-M
调用命令：python ktransformers/server/main.py --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu.yaml --model_path deepseek-ai/DeepSeek-R1 --gguf_path /data/soft/llm_models/DeepSeek-R1-Q4 --total_context 2048 --max_new_tokens 2048 --cpu_infer 16 --port 9999

终端打印的回复与API返回的回复都没有开头的<think>

Originally created by @optimustian on GitHub (Feb 11, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/146 模型：DeepSeek R1-Q4-K-M 调用命令：python ktransformers/server/main.py --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu.yaml --model_path deepseek-ai/DeepSeek-R1 --gguf_path /data/soft/llm_models/DeepSeek-R1-Q4 --total_context 2048 --max_new_tokens 2048 --cpu_infer 16 --port 9999 终端打印的回复与API返回的回复都没有开头的<think> ![Image](https://github.com/user-attachments/assets/a6c6e5cd-b294-4334-8725-25e8b2af2775)

bsakaguchi closed this issue

2026-10-29 14:33:45 +00:00

bsakaguchi referenced this issue

2026-10-29 14:51:36 +00:00

[PR #83] [MERGED] Use cond var to avoid busy loop #1346