mirror of
https://github.com/deepseek-ai/DeepSeek-VL2.git
synced 2026-11-05 01:07:23 +00:00
[GH-ISSUE #79] GPU memory usage issue #65
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-VL2#65
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @rodney-he on GitHub (Feb 13, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/79
Hi thanks for your excellent work. I'm facing a strange problem, I can use deepseek-vl-7b on my RTX 3090 GPU wtih 24GB vRAM, but can't load deepseek-vl2-small model which only has 2.8b parameters, the parameter number of deepseek-vl2 is smaller than deepseek-vl, why can't I use it in same GPU?
@HubHop commented on GitHub (Feb 13, 2025):
Hi @RodneyHe, please note that DeepSeek-VL2-Small is a MoE model, which has 16B parameters in total, with 2.8B activated parameters during inference.
@rodney-he commented on GitHub (Feb 13, 2025):
OK, 2.8B is the activated parameters, that makes sense, thanks for your explanation.
@trongldwork commented on GitHub (Mar 3, 2025):
if i only want to inference, Do I need 16GB Vram?
@rodney-he commented on GitHub (Mar 3, 2025):
For inference, 1B parameter require 4G+ VRAM(FP32) or 2G+ VRAM(FP16), and MoE model loads all 16B parameters, so I think 16G vRAM is not enough. At least requiring 32G vRAM.