mirror of
https://github.com/deepseek-ai/DeepSeek-V2.git
synced 2026-11-05 00:47:23 +00:00
[GH-ISSUE #24] why i use vllm inference deepseek v2 ,speed is low #23
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-V2#23
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @ZzzybEric on GitHub (May 12, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/24
i use vllm to inference deepspeed, use flask to deploy model. When the problem enters the model, it always gets stuck for a long time in the processd prompt step，the code i use is your example code
@luofuli commented on GitHub (May 27, 2024):
https://huggingface.co/deepseek-ai/DeepSeek-V2/discussions/1
@ZzzybEric
@ran130683 commented on GitHub (Jun 20, 2024):
whats your gpu type？