[GH-ISSUE #767] [Usage] Can we run the non-quantized version of Deepseek-V3 in multiple 4090s? #553

New issue

Closed

opened 2026-10-29 14:41:08 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:41:08 +00:00

Owner

Originally created by @CharlesRiggins on GitHub (Mar 3, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/767

I have an 8-GPU 4090 machine. The total GPU RAM is 192 GB. I want to run the full-scale, non-quantized FP8-weight Deepseek-V3 on it. Is it possible with the Ktransformers?

Originally created by @CharlesRiggins on GitHub (Mar 3, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/767 I have an 8-GPU 4090 machine. The total GPU RAM is 192 GB. I want to run the full-scale, non-quantized FP8-weight Deepseek-V3 on it. Is it possible with the Ktransformers?

bsakaguchi

2026-10-29 14:41:08 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:41:10 +00:00

Author

Owner

@chenht2022 commented on GitHub (Mar 3, 2025):

192GB HBM is not enough to accommodate non quantized Deepseek-V3 model. You can use KTransformers' GPU-CPU hybrid inference. CPU kernel supports Q8_0 quantization which has almost the same accuracy as the original model.

@chenht2022 commented on GitHub (Mar 3, 2025): 192GB HBM is not enough to accommodate non quantized Deepseek-V3 model. You can use KTransformers' GPU-CPU hybrid inference. CPU kernel supports Q8_0 quantization which has almost the same accuracy as the original model.

bsakaguchi commented

2026-10-29 14:41:10 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！