kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #319] 成功部署DeepSeek-R1-671B-4bit 性能问题 #202

New issue

Closed

opened 2026-10-29 14:35:46 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-29 14:35:46 +00:00

Owner

Originally created by @sicong-lin on GitHub (Feb 15, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/319

CPU：Intel(R) Xeon(R) Platinum 8352Y CPU @ 2.20GHz
GPU: 1*RTX 4090 24G
AVX-512
1000token输出只能达到6-7 token/s
cpu核心设置如下
--cpu_infer 65

readme里给出的数据32cores就可以达到8token/s

Originally created by @sicong-lin on GitHub (Feb 15, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/319 CPU：Intel(R) Xeon(R) Platinum 8352Y CPU @ 2.20GHz GPU: 1*RTX 4090 24G AVX-512 1000token输出只能达到6-7 token/s cpu核心设置如下 --cpu_infer 65 ![Image](https://github.com/user-attachments/assets/d3ec0201-dbb2-48e8-b2b3-1d17d8bac596) readme里给出的数据32cores就可以达到8token/s

bsakaguchi

2026-10-29 14:35:46 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:35:48 +00:00

Author

Owner

@guanghuizhao commented on GitHub (Feb 15, 2025):

关注

@guanghuizhao commented on GitHub (Feb 15, 2025): 关注

bsakaguchi commented

2026-10-29 14:35:48 +00:00

Author

Owner

@yorkane commented on GitHub (Feb 15, 2025):

4代志强才支持AMX，应该是这个原因

@yorkane commented on GitHub (Feb 15, 2025): 4代志强才支持AMX，应该是这个原因

bsakaguchi commented

2026-10-29 14:35:48 +00:00

Author

Owner

@Radohead commented on GitHub (Feb 15, 2025):

单路么？差不多吧，xeon 3代的ddr4带宽比ddr5 4800少了1/3

@Radohead commented on GitHub (Feb 15, 2025): 单路么？差不多吧，xeon 3代的ddr4带宽比ddr5 4800少了1/3

bsakaguchi commented

2026-10-29 14:35:48 +00:00

Author

Owner

@yydslc commented on GitHub (Feb 17, 2025):

借楼
CPU:epyc 7c132
GPU:4090
内存：ddr4 2933 64g8
设置56核，2000tokens
输出只有2.3 .单纯是内存条没插满吗？

@yydslc commented on GitHub (Feb 17, 2025): 借楼 CPU:epyc 7c13*2 GPU:4090 内存：ddr4 2933 64g*8 设置56核，2000tokens 输出只有2.3 .单纯是内存条没插满吗？

bsakaguchi commented

2026-10-29 14:35:48 +00:00

Author

Owner

@yansiyu550 commented on GitHub (Feb 18, 2025):

单路么？差不多吧，xeon 3代的ddr4带宽比ddr5 4800少了1/3

您好，想问一下，作者配置：Memory: standard DDR5-4800 server DRAM (1 TB), each socket with 8×DDR5-4800，内存带宽约为600G/s，性能只和内存带宽有关系吗？为啥不是看D2H带宽？

@yansiyu550 commented on GitHub (Feb 18, 2025): > 单路么？差不多吧，xeon 3代的ddr4带宽比ddr5 4800少了1/3 您好，想问一下，作者配置：Memory: standard DDR5-4800 server DRAM (1 TB), each socket with 8×DDR5-4800，内存带宽约为600G/s，性能只和内存带宽有关系吗？为啥不是看D2H带宽？

bsakaguchi commented

2026-10-29 14:35:48 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！