kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #1308] intel gpu推理速度 #878

New issue

Open

opened 2026-10-29 14:45:42 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-29 14:45:42 +00:00

Owner

Originally created by @zhaoyukoon on GitHub (May 15, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1308

想了解一下intel gpu配置对应的deepseek模型推理速度，作为参考。谢谢！

Originally created by @zhaoyukoon on GitHub (May 15, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1308 想了解一下intel gpu配置对应的deepseek模型推理速度，作为参考。谢谢！

bsakaguchi commented

2026-10-29 14:45:42 +00:00

Author

Owner

@Thoughtlesstime commented on GitHub (May 16, 2025):

性能好像跟cpu/mem的配置都有点关系，手头一台第五代至强+一张A770，Deepseek R1 Q4KM 在 local_chat（cpuinfer 53）模式下，差不多能跑到 7-8 tps，感觉也够用了

@Thoughtlesstime commented on GitHub (May 16, 2025): 性能好像跟cpu/mem的配置都有点关系，手头一台第五代至强+一张A770，Deepseek R1 Q4KM 在 local_chat（cpuinfer 53）模式下，差不多能跑到 7-8 tps，感觉也够用了

bsakaguchi commented

2026-10-29 14:45:42 +00:00

Author

Owner

@tsdcz commented on GitHub (May 16, 2025):

性能好像跟cpu/mem的配置都有点关系，手头一台第五代至强+一张A770，Deepseek R1 Q4KM 在 local_chat（cpuinfer 53）模式下，差不多能跑到 7-8 tps，感觉也够用了

能详细说一下机器的配置吗？

@tsdcz commented on GitHub (May 16, 2025): > 性能好像跟cpu/mem的配置都有点关系，手头一台第五代至强+一张A770，Deepseek R1 Q4KM 在 local_chat（cpuinfer 53）模式下，差不多能跑到 7-8 tps，感觉也够用了能详细说一下机器的配置吗？

bsakaguchi commented

2026-10-29 14:45:42 +00:00

Author

Owner

@drew-ye commented on GitHub (Jun 18, 2025):

我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样

@drew-ye commented on GitHub (Jun 18, 2025): 我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样

bsakaguchi commented

2026-10-29 14:45:42 +00:00

Author

Owner

@aubreyli commented on GitHub (Jun 18, 2025):

我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样

What's your OS distro and version? and your memory size and speed?
Different software component versions could cause different performance.
You can try xpu docker to see any difference.

@aubreyli commented on GitHub (Jun 18, 2025): > 我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样 What's your OS distro and version? and your memory size and speed? Different software component versions could cause different performance. You can try [xpu docker](https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/Docker_xpu.md) to see any difference.

bsakaguchi commented

2026-10-29 14:45:43 +00:00

Author

Owner

@icm-ai commented on GitHub (Jul 15, 2025):

我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样

我平均测下来速度比你还慢，只有 1.6token/s ...
我用docker部署跑的unsloth/DeepSeek-R1-0528-Q4_K_M_GGuf，只能用上单卡A770，你两张卡能都用上么？
cpu是 Intel(R) Xeon(R) w5-3525 32核，512g ddr5 4800

@icm-ai commented on GitHub (Jul 15, 2025): > 我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样我平均测下来速度比你还慢，只有 1.6token/s ... 我用docker部署跑的unsloth/DeepSeek-R1-0528-Q4_K_M_GGuf，只能用上单卡A770，你两张卡能都用上么？ cpu是 Intel(R) Xeon(R) w5-3525 32核，512g ddr5 4800

bsakaguchi commented

2026-10-29 14:45:43 +00:00

Author

Owner

@aubreyli commented on GitHub (Jul 15, 2025):

Please refer to issue #1329 , according to your hardware configuration, your decode speed should be around 5 to 6 tokens per second

@aubreyli commented on GitHub (Jul 15, 2025): Please refer to issue #1329 , according to your hardware configuration, your decode speed should be around 5 to 6 tokens per second

bsakaguchi commented

2026-10-29 14:45:43 +00:00

Author

Owner

@icm-ai commented on GitHub (Jul 23, 2025):

我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样

我平均测下来速度比你还慢，只有 1.6token/s ... 我用docker部署跑的unsloth/DeepSeek-R1-0528-Q4_K_M_GGuf，只能用上单卡A770，你两张卡能都用上么？ cpu是 Intel(R) Xeon(R) w5-3525 32核，512g ddr5 4800

更正一下，只有16核...速度确实很慢，Q4_K_M 极限也就2+

@icm-ai commented on GitHub (Jul 23, 2025): > > 我至强6代+2 intel Arc A770 跑满血版Q4量化速度是1.9~2.9 t/s 线程44 ，不知道大家是不是跟我一样 > > 我平均测下来速度比你还慢，只有 1.6token/s ... 我用docker部署跑的unsloth/DeepSeek-R1-0528-Q4_K_M_GGuf，只能用上单卡A770，你两张卡能都用上么？ cpu是 Intel(R) Xeon(R) w5-3525 32核，512g ddr5 4800 更正一下，只有16核...速度确实很慢，Q4_K_M 极限也就2+