[GH-ISSUE #32] 33b需要多少显存，怎么量化加载 #27

New issue

Closed

opened 2026-10-26 21:02:22 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:02:22 +00:00

Owner

Originally created by @xiaokai01 on GitHub (Nov 14, 2023).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/32

Originally created by @xiaokai01 on GitHub (Nov 14, 2023). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/32

bsakaguchi closed this issue

2026-10-26 21:02:22 +00:00

bsakaguchi commented

2026-10-26 21:02:23 +00:00

Author

Owner

@guoday commented on GitHub (Nov 22, 2023):

已有社区开源了量化后的模型：https://huggingface.co/TheBloke/deepseek-coder-33B-instruct-GPTQ
你可以参考一下

@guoday commented on GitHub (Nov 22, 2023): 已有社区开源了量化后的模型：https://huggingface.co/TheBloke/deepseek-coder-33B-instruct-GPTQ 你可以参考一下

bsakaguchi commented

2026-10-26 21:02:23 +00:00

Author

Owner

@soloice commented on GitHub (Nov 28, 2023):

原始版本：一个参数占两个字节，33B 就是 66 GB，再加上运行时 KV cache 占的空间，单卡的话得 80GB 的显卡才能跑；如果做流水并行/张量并行基本上可以线性减少。
量化版本：参考 the bloke 量化的结果。差不多量化到几个 bit 显存就对应节省多少。

@soloice commented on GitHub (Nov 28, 2023): 原始版本：一个参数占两个字节，33B 就是 66 GB，再加上运行时 KV cache 占的空间，单卡的话得 80GB 的显卡才能跑；如果做流水并行/张量并行基本上可以线性减少。量化版本：参考 the bloke 量化的结果。差不多量化到几个 bit 显存就对应节省多少。