[GH-ISSUE #1533] [Bug] Why DUAL-CPU has no speed up? #1022

New issue

Closed

opened 2026-10-29 14:47:34 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:47:34 +00:00

Owner

Originally created by @mrgaolei on GitHub (Oct 28, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1533

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.
3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

DUAL-CPU's decode token speed is equal to Single-CPU.

Reproduction

When I bought a new dual-CPU server with AMX support, and install with libnuma-dev and USE_NUMA=1, but the speed is not up. always ≈ 13 token/s.

When it working, I monitor the RAM from 1-CPU's 400G up to 2-CPU's 800G, and both CPU cores working.
But the decod speed is equal than 1-CPU with 512G RAM.

Environment

CPU: 6416H * 2
RAM: 64G DDR5 * 16 = 1T
GPU: RTX 3090

KTransformers tag: v0.3.2

Originally created by @mrgaolei on GitHub (Oct 28, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1533 ### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug DUAL-CPU's decode token speed is equal to Single-CPU. ### Reproduction When I bought a new dual-CPU server with AMX support, and install with `libnuma-dev` and `USE_NUMA=1`, but the speed is not up. always ≈ 13 token/s. When it working, I monitor the RAM from 1-CPU's 400G up to 2-CPU's 800G, and both CPU cores working. But the decod speed is equal than 1-CPU with 512G RAM. ### Environment CPU: 6416H * 2 RAM: 64G DDR5 * 16 = 1T GPU: RTX 3090 KTransformers tag: v0.3.2

bsakaguchi closed this issue

2026-10-29 14:47:34 +00:00

bsakaguchi commented

2026-10-29 14:47:34 +00:00

Author

Owner

@Mrw33554432 commented on GitHub (Nov 4, 2025):

Is it possible that the bottleneck is the GPU now? 3090 is quite close to V100 in term of Flops.
Just a guess.

@Mrw33554432 commented on GitHub (Nov 4, 2025): Is it possible that the bottleneck is the GPU now? 3090 is quite close to V100 in term of Flops. Just a guess.

bsakaguchi commented

2026-10-29 14:47:35 +00:00

Author

Owner

@mrgaolei commented on GitHub (Nov 5, 2025):

Is it possible that the bottleneck is the GPU now? 3090 is quite close to V100 in term of Flops. Just a guess.

However, aren't ktransformers entirely computed by the CPU? The GPU only uses video memory. Because when I was executing decode, the CPU was at 100%, while the GPU was only at 30%, and the fan wasn't even spinning.

@mrgaolei commented on GitHub (Nov 5, 2025): > Is it possible that the bottleneck is the GPU now? 3090 is quite close to V100 in term of Flops. Just a guess. However, aren't ktransformers entirely computed by the CPU? The GPU only uses video memory. Because when I was executing decode, the CPU was at 100%, while the GPU was only at 30%, and the fan wasn't even spinning.