deepseek-ai/DeepSeek-V2
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-V2.git synced 2026-11-05 00:47:23 +00:00
Code Issues 90 Projects Packages Wiki Activity

[GH-ISSUE #1] 请提供GGUF,并支持OLLAMA #3

New issue
Open
opened 2026-10-26 21:06:29 +00:00 by bsakaguchi · 6 comments
bsakaguchi commented 2026-10-26 21:06:29 +00:00
Owner
Copy link

Originally created by @taozhiyuai on GitHub (May 6, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/1

请提供GGUF,并支持OLLAMA

Originally created by @taozhiyuai on GitHub (May 6, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V2/issues/1 请提供GGUF,并支持OLLAMA
bsakaguchi commented 2026-10-26 21:06:30 +00:00
Author
Owner
Copy link

@fufay commented on GitHub (May 8, 2024):

附议

<!-- gh-comment-id:2099650857 --> @fufay commented on GitHub (May 8, 2024): 附议
bsakaguchi commented 2026-10-26 21:06:30 +00:00
Author
Owner
Copy link

@buildcharger commented on GitHub (May 9, 2024):

这么大的模型，用ollama 4bit可能都得120G内存以上。

有没有勇士尝试过？

<!-- gh-comment-id:2102161521 --> @buildcharger commented on GitHub (May 9, 2024): 这么大的模型，用ollama 4bit可能都得120G内存以上。 有没有勇士尝试过？
bsakaguchi commented 2026-10-26 21:06:30 +00:00
Author
Owner
Copy link

@wujohns commented on GitHub (May 9, 2024):

这么大的模型，用ollama 4bit可能都得120G内存以上。

有没有勇士尝试过？

看激活参数有多少吧，21b的激活参数的话，根据以往的经验，我的x99跑到3-5token/s都可以尝试摸一把的，内存是管饱的

<!-- gh-comment-id:2102196261 --> @wujohns commented on GitHub (May 9, 2024): > 这么大的模型，用ollama 4bit可能都得120G内存以上。 > > 有没有勇士尝试过？ 看激活参数有多少吧，21b的激活参数的话，根据以往的经验，我的x99跑到3-5token/s都可以尝试摸一把的，内存是管饱的
bsakaguchi commented 2026-10-26 21:06:30 +00:00
Author
Owner
Copy link

@taozhiyuai commented on GitHub (May 10, 2024):

这么大的模型，用ollama 4bit可能都得120G内存以上。

有没有勇士尝试过？

我的可以跑

<!-- gh-comment-id:2103880099 --> @taozhiyuai commented on GitHub (May 10, 2024): > 这么大的模型，用ollama 4bit可能都得120G内存以上。 > > 有没有勇士尝试过？ 我的可以跑
bsakaguchi commented 2026-10-26 21:06:30 +00:00
Author
Owner
Copy link

@haichuan1221 commented on GitHub (May 16, 2024):

有没有跑起来的呢? 是否可以分享一下gguf文件呢

<!-- gh-comment-id:2113823971 --> @haichuan1221 commented on GitHub (May 16, 2024): 有没有跑起来的呢? 是否可以分享一下gguf文件呢
bsakaguchi commented 2026-10-26 21:06:30 +00:00
Author
Owner
Copy link

@100ZZ commented on GitHub (May 29, 2024):

跑不起来；网上提供的GGUF都没法运行，下载原始模型通过最新llama.cpp可以转换成gguf，但是还是'error loading model architecture: unknown model architecture: 'deepseek2''"

<!-- gh-comment-id:2136396239 --> @100ZZ commented on GitHub (May 29, 2024): 跑不起来；网上提供的GGUF都没法运行，下载原始模型通过最新llama.cpp可以转换成gguf，但是还是'error loading model architecture: unknown model architecture: 'deepseek2''"
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-V2#3
No description provided.