mirror of
https://github.com/deepseek-ai/DeepSeek-V3.git
synced 2026-11-05 00:47:48 +00:00
[GH-ISSUE #403] Lite Version V3 weights #189
Labels
No labels
closed-as-stale
enhancement
question
spam
stale
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-V3#189
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Tortoise17 on GitHub (Jan 28, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/403
This is really an amazing work done.
I want to ask that is there any chance to have the lite version weights which can support the lower number of GPUs to run locally? Like minimal requirements which can be minimized to between 2 - 4 GPUs Nvidia A40 or so for inference?
Please guide me if maybe I am wrong and overseen any already implemented workflow.
@0xsaugat commented on GitHub (Jan 28, 2025):
Hey, I totally agree with you! Running DeepSeek-V3 locally with just 2-4 GPUs (like the Nvidia A40s) would make it much more accessible for users with limited hardware resources. It would be great if there could be a lite version of the model that can perform inference without needing such massive setups.
If anyone has any suggestions on how to optimize the current setup for fewer GPUs, I'd be really interested in hearing them! Maybe pruning certain layers or using more aggressive quantization could help?
@Tortoise17 commented on GitHub (Feb 8, 2025):
@mowentian if the weights are available. can you share link please?