[GH-ISSUE #120] ktransformers_ext/bench #69

New issue

Closed

opened 2026-10-29 14:33:33 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:33:33 +00:00

Owner

Originally created by @wanwujie1 on GitHub (Jan 12, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/120

When running all scripts under the bench directory and finding that, when executed on the CPU, PyTorch's native fp16 and int8 are faster than ggml_type's fp16, q8_0, q4_k_m, and q2_k, and the performance differences between ggml_type's fp16, q8_0, q4_k_m, and q2_k are also minimal, is this normal?

Originally created by @wanwujie1 on GitHub (Jan 12, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/120 When running all scripts under the bench directory and finding that, when executed on the CPU, PyTorch's native fp16 and int8 are faster than ggml_type's fp16, q8_0, q4_k_m, and q2_k, and the performance differences between ggml_type's fp16, q8_0, q4_k_m, and q2_k are also minimal, is this normal?

bsakaguchi

2026-10-29 14:33:33 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:33:34 +00:00

Author

Owner

@chenht2022 commented on GitHub (Feb 10, 2025):

Have you ensured that the number of threads in cpuinfer does not exceed the number of physical CPU cores？You can set it in bench_xxx.py

@chenht2022 commented on GitHub (Feb 10, 2025): Have you ensured that the number of threads in cpuinfer does not exceed the number of physical CPU cores？You can set it in bench_xxx.py

bsakaguchi commented

2026-10-29 14:33:34 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！