[GH-ISSUE #161] 用vllm加速推理框架推理速度还是很慢 #140

New issue

Open

opened 2026-10-26 21:04:02 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:04:02 +00:00

Owner

Originally created by @zhuzhiwei88 on GitHub (May 20, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/161

本地部署的deepseek-coder-33b，两块RTX A6000 48G的显卡
python -m vllm.entrypoints.openai.api_server --model /home/superadmin/coder33b --trust-remote-code --tensor-parallel-size=2 --served-model-name=deepseek-coder
启动后调用/v1/completions接口获取推理结果，但是效率极低，平均15tokens/s，如图

有人遇到过吗，如何解决有大佬指教吗

Originally created by @zhuzhiwei88 on GitHub (May 20, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/161 本地部署的deepseek-coder-33b，两块RTX A6000 48G的显卡 python -m vllm.entrypoints.openai.api_server --model /home/superadmin/coder33b --trust-remote-code --tensor-parallel-size=2 --served-model-name=deepseek-coder 启动后调用/v1/completions接口获取推理结果，但是效率极低，平均15tokens/s，如图 ![image](https://github.com/deepseek-ai/DeepSeek-Coder/assets/18474897/b37e19c0-66b1-4410-bd00-1a4ed6e17243) 有人遇到过吗，如何解决有大佬指教吗

bsakaguchi commented

2026-10-26 21:04:03 +00:00

Author

Owner

@zhusq20 commented on GitHub (Aug 16, 2024):

@zhusq20 commented on GitHub (Aug 16, 2024): +1