[GH-ISSUE #452] Why is the original DeepSeek-V3 model required with quantized model? #312

New issue

Closed

opened 2026-10-29 14:37:42 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:37:42 +00:00

Owner

Originally created by @snailwei on GitHub (Feb 18, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/452

Hi Sir

I followed the docker deployment
https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/Docker.md

Buy why is the original huge model required with quantized model?

python -m ktransformers.local_chat  --gguf_path /models/path/to/gguf_path --model_path /models/path/to/model_path --cpu_infer 33

Originally created by @snailwei on GitHub (Feb 18, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/452 Hi Sir I followed the docker deployment https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/Docker.md Buy why is the original huge model required with quantized model? ``` python -m ktransformers.local_chat --gguf_path /models/path/to/gguf_path --model_path /models/path/to/model_path --cpu_infer 33 ```

bsakaguchi

2026-10-29 14:37:42 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:37:43 +00:00

Author

Owner

@snailwei commented on GitHub (Feb 19, 2025):

I solved by myself, it seems the --model_path just referes to original model's config, should do it like this

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/deepseek-ai/DeepSeek-V3

This should be documented as well, 官方文档喜欢用英文，但是 sucks !

@snailwei commented on GitHub (Feb 19, 2025): I solved by myself, it seems the `--model_path` just referes to original model's config, should do it like this ```sh GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/deepseek-ai/DeepSeek-V3 ``` This should be documented as well, 官方文档喜欢用英文，但是 `sucks `!

bsakaguchi commented

2026-10-29 14:37:43 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！