kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #397] 【GPU/CPU混合推理】deepseekR1使用了多GPU，但推理性能和单GPU一致，没有提升。 #267

New issue

Closed

opened 2026-10-29 14:36:55 +00:00 by bsakaguchi · 8 comments

bsakaguchi commented

2026-10-29 14:36:55 +00:00

Owner

Originally created by @Keithwwa on GitHub (Feb 17, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/397

使用了多GPU，每张卡上使用内存约为4G，推理性能和单GPU一致，没有提升。

Originally created by @Keithwwa on GitHub (Feb 17, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/397 使用了多GPU，每张卡上使用内存约为4G，推理性能和单GPU一致，没有提升。

bsakaguchi

2026-10-29 14:36:55 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@lld1995 commented on GitHub (Feb 17, 2025):

一样，跑起来巨慢。。。我看推文宣传说比llamacpp快好多倍。。。。。

@lld1995 commented on GitHub (Feb 17, 2025): 一样，跑起来巨慢。。。我看推文宣传说比llamacpp快好多倍。。。。。

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@TTThanos commented on GitHub (Feb 17, 2025):

https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/FAQ.md
这里的原文“Note: The ktransformers' multi-gpu stratigy is pipline, which is not able to speed up the model's inference. It's only for the model's weight distribution.” 多卡并不会加速推理，仅仅改变权重分布

@TTThanos commented on GitHub (Feb 17, 2025): https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/FAQ.md 这里的原文“Note: The ktransformers' multi-gpu stratigy is pipline, which is not able to speed up the model's inference. It's only for the model's weight distribution.” 多卡并不会加速推理，仅仅改变权重分布

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@lld1995 commented on GitHub (Feb 17, 2025):

https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/FAQ.md 这里的原文“Note: The ktransformers' multi-gpu stratigy is pipline, which is not able to speed up the model's inference. It's only for the model's weight distribution.” 多卡并不会加速推理，仅仅改变权重分布

这个是怎么跑出13Token/s的啊，我4卡3090，很久才崩一个词。CPU、内存频率对性能的影响更大？

@lld1995 commented on GitHub (Feb 17, 2025): > https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/FAQ.md 这里的原文“Note: The ktransformers' multi-gpu stratigy is pipline, which is not able to speed up the model's inference. It's only for the model's weight distribution.” 多卡并不会加速推理，仅仅改变权重分布 ![Image](https://github.com/user-attachments/assets/807af3bd-cf20-45bf-b380-e7f588fc2e86) 这个是怎么跑出13Token/s的啊，我4卡3090，很久才崩一个词。CPU、内存频率对性能的影响更大？

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@TTThanos commented on GitHub (Feb 17, 2025):

我下午刚跑通，我是6.7 Token/s, 单卡A10. 我下一步准备把cpu_infer再提高一点，之前用的是tutorial里的参数设置

@TTThanos commented on GitHub (Feb 17, 2025): 我下午刚跑通，我是6.7 Token/s, 单卡A10. 我下一步准备把cpu_infer再提高一点，之前用的是tutorial里的参数设置

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@TTThanos commented on GitHub (Feb 17, 2025):

你可以看一下你用的是KTransformer v0.2 还是 v0.3，这个结果好像是0.3的

@TTThanos commented on GitHub (Feb 17, 2025): 你可以看一下你用的是KTransformer v0.2 还是 v0.3，这个结果好像是0.3的

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@GavinH1984 commented on GitHub (Feb 17, 2025):

请问你们运行的参数是什么，怎么配置r1多卡运行的？

@GavinH1984 commented on GitHub (Feb 17, 2025): 请问你们运行的参数是什么，怎么配置r1多卡运行的？

bsakaguchi commented

2026-10-29 14:36:56 +00:00

Author

Owner

@yhfgyyf commented on GitHub (Feb 18, 2025):

速度主要受限于cpu和内存这边，受内存总带宽影响较大

@yhfgyyf commented on GitHub (Feb 18, 2025): 速度主要受限于cpu和内存这边，受内存总带宽影响较大

bsakaguchi commented

2026-10-29 14:36:57 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！