[GH-ISSUE #111] vllm输出结果质量低 #89

New issue

Open

opened 2026-10-26 21:11:46 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:11:46 +00:00

Owner

Originally created by @chenyzh28 on GitHub (Mar 7, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/111

我用官方的inference.py代码推理时，输出结果是比较正常的，比如问“图片中的角色是哪部动漫作品中的人物？”

输出是：

但当我使用vllm部署模型时，使用vllm提供的chat_template，同样的输入，输出结果却很奇怪。
vllm的启动命令：
CUDA_VISIBLE_DEVICES=7 vllm serve deepseek-vl2 --port 8102 --max-model-len 4096 --hf_overrides '{"architectures":["DeepseekVLV2ForCausalLM"]}' --gpu-memory-utilization 0.9 --chat_template ./template_deepseek_vl2.jinja
输出结果：

感觉精度明显下降了不少，不知道chat_template还是哪里的问题，求大佬们指点。

Originally created by @chenyzh28 on GitHub (Mar 7, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/111 我用官方的inference.py代码推理时，输出结果是比较正常的，比如问“图片中的角色是哪部动漫作品中的人物？” ![Image](https://github.com/user-attachments/assets/44dca395-f1dd-4b0c-87d6-d492df512976) 输出是： ![Image](https://github.com/user-attachments/assets/39332703-d613-403d-bcb0-e0cbb8e266d7) 但当我使用vllm部署模型时，使用vllm提供的[chat_template](https://github.com/vllm-project/vllm/blob/main/examples/template_deepseek_vl2.jinja)，同样的输入，输出结果却很奇怪。 vllm的启动命令： `CUDA_VISIBLE_DEVICES=7 vllm serve deepseek-vl2 --port 8102 --max-model-len 4096 --hf_overrides '{"architectures":["DeepseekVLV2ForCausalLM"]}' --gpu-memory-utilization 0.9 --chat_template ./template_deepseek_vl2.jinja` 输出结果： ![Image](https://github.com/user-attachments/assets/ea545f6e-fe01-4f67-a50a-65bbed41f86a) 感觉精度明显下降了不少，不知道chat_template还是哪里的问题，求大佬们指点。

bsakaguchi commented

2026-10-26 21:11:47 +00:00

Author

Owner

@ppllracing commented on GitHub (May 15, 2025):

I'd like to know which version of vllm you have installed? Is it necessary to modify the dependency of deepseek-vl2?

@ppllracing commented on GitHub (May 15, 2025): I'd like to know which version of vllm you have installed? Is it necessary to modify the dependency of deepseek-vl2?

bsakaguchi referenced this issue

2026-10-26 21:12:37 +00:00

[PR #89] [MERGED] change xformers import location #142