[GH-ISSUE #322] 在同配置下，DeepSeek-R1-Distill-Qwen-14B推理速度比Qwen2.5-14B-Instruct慢了3倍左右 #251

New issue

Closed

opened 2026-10-26 21:17:53 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:17:53 +00:00

Owner

Originally created by @qiuyeqiang on GitHub (Feb 8, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/322

运行代码，来自于https://www.modelscope.cn/models/Qwen/Qwen2.5-14B-Instruct/summary ，Quickstart 部分

from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import snapshot_download

model_name = snapshot_download("deepseek-ai/DeepSeek-R1-Distill-Qwen-14B")

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

https://www.modelscope.cn/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/feedback/issueDetail/21819
这边也有人反馈相同的问题

Originally created by @qiuyeqiang on GitHub (Feb 8, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/322 运行代码，来自于[https://www.modelscope.cn/models/Qwen/Qwen2.5-14B-Instruct/summary ](https://www.modelscope.cn/models/Qwen/Qwen2.5-14B-Instruct/summary )，Quickstart 部分 ``` from modelscope import AutoModelForCausalLM, AutoTokenizer from modelscope import snapshot_download model_name = snapshot_download("deepseek-ai/DeepSeek-R1-Distill-Qwen-14B") model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained(model_name) prompt = "Give me a short introduction to large language model." messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( **model_inputs, max_new_tokens=512 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ] response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] print(response) ``` https://www.modelscope.cn/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/feedback/issueDetail/21819 这边也有人反馈相同的问题

bsakaguchi closed this issue

2026-10-26 21:17:54 +00:00

bsakaguchi commented

2026-10-26 21:17:55 +00:00

Author

Owner

@sz128 commented on GitHub (Feb 8, 2025):

R1蒸馏的模型推理生成tokens变多了，请求时延变长不是很正常嘛。

@sz128 commented on GitHub (Feb 8, 2025): R1蒸馏的模型推理生成tokens变多了，请求时延变长不是很正常嘛。