[GH-ISSUE #1115] 【反馈】0.24post1 AMD线程撕裂者7975WX 性能反馈 #778

New issue

Open

opened 2026-10-29 14:44:22 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 14:44:22 +00:00

Owner

Originally created by @yeungtuzi on GitHub (Apr 10, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1115

硬件环境：
AMD线程撕裂者7975WX 32C/64T
8x DDR5-5200 64GB ECC
NVIDIA 4090D

软件环境：
Ubuntu 24.04.2 LTS kernel 6.11.0-21-generic
Driver Version: 570.124.06 CUDA Version: 12.8
approachingai/ktransformers:latest-FANCY (对应0.24post1-avx512)

运行命令：
ktransformers --model_path ~/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-R1/snapshots/56d4cbbb4d29f4355bab4b9a39ccb717a14ad5ad/ --gguf_path /model/DeepSeek-R1-Q4_K_M --port 10002 --max_new_tokens=32768 --cache_lens=131072 --model_name "DeepSeek-R1" --chunk_size 256 --max_batch_size 4 --backend_type balance_serve --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml --force_think --cpu_infer 60

性能测试：

单请求，客户端显示 [Request 0] Speed: 7.37 tokens/s
服务端显示 Performance(T/s): prefill 26.48842770198315, decode 10.484790031355752.

四路并发，客户端显示
[Request 2] Speed: 2.79 tokens/s
[Request 1] Speed: 2.69 tokens/s
[Request 3] Speed: 2.79 tokens/s
[Request 0] Speed: 6.76 tokens/s
服务端显示
Performance(T/s): prefill 2.6035733052505035, decode 9.986869241943065.

跟在Proxmox VE中运行ubuntu 24.04并passthrough显卡的性能基本一致，可见虚拟化平台基本对ktransformers没有性能冲击，追求系统管理方便的话完全可以使用

Originally created by @yeungtuzi on GitHub (Apr 10, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1115 <img width="999" alt="Image" src="https://github.com/user-attachments/assets/89fffdd9-f1c9-479c-aa21-5f24e97ad61a" /> 硬件环境： AMD线程撕裂者7975WX 32C/64T 8x DDR5-5200 64GB ECC NVIDIA 4090D 软件环境： Ubuntu 24.04.2 LTS kernel 6.11.0-21-generic Driver Version: 570.124.06 CUDA Version: 12.8 approachingai/ktransformers:latest-FANCY (对应0.24post1-avx512) 运行命令： ktransformers --model_path ~/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-R1/snapshots/56d4cbbb4d29f4355bab4b9a39ccb717a14ad5ad/ --gguf_path /model/DeepSeek-R1-Q4_K_M --port 10002 --max_new_tokens=32768 --cache_lens=131072 --model_name "DeepSeek-R1" --chunk_size 256 --max_batch_size 4 --backend_type balance_serve --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml --force_think --cpu_infer 60 性能测试：单请求，客户端显示 [Request 0] Speed: 7.37 tokens/s 服务端显示 Performance(T/s): prefill 26.48842770198315, decode 10.484790031355752. 四路并发，客户端显示 [Request 2] Speed: 2.79 tokens/s [Request 1] Speed: 2.69 tokens/s [Request 3] Speed: 2.79 tokens/s [Request 0] Speed: 6.76 tokens/s 服务端显示 Performance(T/s): prefill 2.6035733052505035, decode 9.986869241943065. **跟在Proxmox VE中运行ubuntu 24.04并passthrough显卡的性能基本一致**，可见虚拟化平台基本对ktransformers没有性能冲击，追求系统管理方便的话完全可以使用

bsakaguchi commented

2026-10-29 14:44:23 +00:00

Author

Owner

@magikRUKKOLA commented on GitHub (Apr 11, 2025):

Uh oh. What's going on here? 10.5 tps for DDR5? Doesn't make sense. I am having 8.5 tps for DDR4-3200 (overclocked though) with 64C CPU.

Have you tried to compile the code instead of using the pre-built binaries?

What are the timings config for your RAM? What's the IF clock?

@magikRUKKOLA commented on GitHub (Apr 11, 2025): Uh oh. What's going on here? 10.5 tps for DDR5? Doesn't make sense. I am having 8.5 tps for DDR4-3200 (overclocked though) with 64C CPU. Have you tried to compile the code instead of using the pre-built binaries? What are the timings config for your RAM? What's the IF clock?

bsakaguchi commented

2026-10-29 14:44:24 +00:00

Author

Owner

@chenhongxx commented on GitHub (Apr 11, 2025):

双路吗？有无出现iomouo与pci直通兼容问题？

@chenhongxx commented on GitHub (Apr 11, 2025): 双路吗？有无出现iomouo与pci直通兼容问题？

bsakaguchi commented

2026-10-29 14:44:24 +00:00

Author

Owner

@magikRUKKOLA commented on GitHub (Apr 11, 2025):

双路吗？

Nope, its single CPU system, as the author mentioned. AMD Threadripper PRO 7975WX

@magikRUKKOLA commented on GitHub (Apr 11, 2025): > 双路吗？ Nope, its single CPU system, as the author mentioned. AMD Threadripper PRO 7975WX

bsakaguchi commented

2026-10-29 14:44:24 +00:00

Author

Owner

@yeungtuzi commented on GitHub (Apr 11, 2025):

Uh oh. What's going on here? 10.5 tps for DDR5? Doesn't make sense. I am having 8.5 tps for DDR4-3200 (overclocked though) with 64C CPU.

Have you tried to compile the code instead of using the pre-built binaries?

What are the timings config for your RAM? What's the IF clock?

DDR5-5200，确实对比3200没有明显优势，可能是核心还是太少了，只有32核心
自己编译（裸机、docker内）和预编译（docker内）和虚拟化环境下都试过了，性能一致。
HP的工作站，其BIOS啥也没有，内存时序和频率均不可调整。

等我的9654双路（192C，24C DDR5-4800）上线之后再试试，期待能有较大提高。

@yeungtuzi commented on GitHub (Apr 11, 2025): > Uh oh. What's going on here? 10.5 tps for DDR5? Doesn't make sense. I am having 8.5 tps for DDR4-3200 (overclocked though) with 64C CPU. > > Have you tried to compile the code instead of using the pre-built binaries? > > What are the timings config for your RAM? What's the IF clock? DDR5-5200，确实对比3200没有明显优势，可能是核心还是太少了，只有32核心自己编译（裸机、docker内）和预编译（docker内）和虚拟化环境下都试过了，性能一致。 HP的工作站，其BIOS啥也没有，内存时序和频率均不可调整。等我的9654双路（192C，24C DDR5-4800）上线之后再试试，期待能有较大提高。

bsakaguchi referenced this issue

2026-10-29 14:52:47 +00:00

[PR #778] [MERGED] [Fix] Shape error in KLinearMarlin #1479