[GH-ISSUE #312] 讨论： Ktransformer目前对DeepSeek R1的多GPU适配 (Disscussion: Current Multi-GPU Support for Ktransformer with DeepSeek R1 Q4) #201

New issue

Closed

opened 2026-10-29 14:35:43 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 14:35:43 +00:00

Owner

Originally created by @kissint8 on GitHub (Feb 15, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/312

description: 尝试在H100(80G)上使用Ktransformer部署 DeepSeek R1 Q4_KM量化的模型，但是GPU显存占用只有17%，计算利用率只有15%,CPU负载6.8%(因为没开启CPU加速)，内存占用85GB。
此前对于DeepSeekV3的优化规则中有multi-gpu优化规则，请问运行R1模型时是否需要额外指定呢？
我的运行命令是：python3 -m ktransformers.local_chat --model_path <local_model_dir> --gguf_path <gguf_dir> --max_new_tokens 4096 --force_think true --web true --port <port>

Description:
I have deployed the DeepSeek R1 model with Q4_KM quantization using Ktransformer on an H100 (80GB) GPU. However, I obversed the following resource utilization:

GPU Memory Usage: 17%
Compute Utilization: 15%
CPU Load: 6.8% (since CPU acceleration is not enabled)
Memory Usage: 85GB

Questions:
In the previous optimization rules for DeepSeekV3, I found there was a multi-GPU optimization rule. Do I need to specify anything else when running the R1 model?

My Command is:
python3 -m ktransformers.local_chat --model_path <local_model_dir> --gguf_path <gguf_dir> --max_new_tokens 4096 --force_think true --web true --port <port>

Originally created by @kissint8 on GitHub (Feb 15, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/312 - description: 尝试在H100(80G)上使用Ktransformer部署 DeepSeek R1 Q4_KM量化的模型，但是GPU显存占用只有17%，计算利用率只有15%,CPU负载6.8%(因为没开启CPU加速)，内存占用85GB。 - 此前对于DeepSeekV3的优化规则中有multi-gpu优化规则，请问运行R1模型时是否需要额外指定呢？ - 我的运行命令是：`python3 -m ktransformers.local_chat --model_path <local_model_dir> --gguf_path <gguf_dir> --max_new_tokens 4096 --force_think true --web true --port <port>` Description: I have deployed the DeepSeek R1 model with Q4_KM quantization using Ktransformer on an H100 (80GB) GPU. However, I obversed the following resource utilization: GPU Memory Usage: 17% Compute Utilization: 15% CPU Load: 6.8% (since CPU acceleration is not enabled) Memory Usage: 85GB Questions: In the previous optimization rules for DeepSeekV3, I found there was a multi-GPU optimization rule. Do I need to specify anything else when running the R1 model? My Command is: `python3 -m ktransformers.local_chat --model_path <local_model_dir> --gguf_path <gguf_dir> --max_new_tokens 4096 --force_think true --web true --port <port>`

bsakaguchi

2026-10-29 14:35:43 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:35:44 +00:00

Author

Owner

@Atream commented on GitHub (Feb 15, 2025):

多GPU的支持正在进行中，需要自行修改yaml将更多experts放到GPU上，不过现在有些bug，我们再优化下

@Atream commented on GitHub (Feb 15, 2025): 多GPU的支持正在进行中，需要自行修改yaml将更多experts放到GPU上，不过现在有些bug，我们再优化下

bsakaguchi commented

2026-10-29 14:35:44 +00:00

Author

Owner

@Remondor commented on GitHub (Feb 16, 2025):

您好，我想问一下。
如果单卡没有16g。
用双卡组20g能否使用kt框架跑r1呢？
还有就是，q4以下量化的r1，比如q2 是不是就不用14g左右的显存了。仅需7g左右就能跑起来?

@Remondor commented on GitHub (Feb 16, 2025): 您好，我想问一下。如果单卡没有16g。用双卡组20g能否使用kt框架跑r1呢？还有就是，q4以下量化的r1，比如q2 是不是就不用14g左右的显存了。仅需7g左右就能跑起来?

bsakaguchi commented

2026-10-29 14:35:45 +00:00

Author

Owner

@zenwan commented on GitHub (Feb 16, 2025):

多GPU的支持正在进行中，需要自行修改yaml将更多experts放到GPU上，不过现在有些bug，我们再优化下

期待，希望能简化多GPU的配置

@zenwan commented on GitHub (Feb 16, 2025): > 多GPU的支持正在进行中，需要自行修改yaml将更多experts放到GPU上，不过现在有些bug，我们再优化下期待，希望能简化多GPU的配置

bsakaguchi commented

2026-10-29 14:35:45 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！