[GH-ISSUE #398] --max_new_tokens速度测试期待优化 #269

New issue

Closed

opened 2026-10-29 14:36:57 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:36:57 +00:00

Owner

Originally created by @SongXiaoMao on GitHub (Feb 17, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/398

使用同一个模型同一个问题 DeepSeek-R1-Q4-K-M 测试速度如下（4*3090加载）

--max_new_tokens 16384 2.5token

--max_new_tokens 8192 3.2token

--max_new_tokens 4096 3.9token

--max_new_tokens 2048 4.1token

内存升级到512G 其他配置见图

Originally created by @SongXiaoMao on GitHub (Feb 17, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/398 使用同一个模型同一个问题 DeepSeek-R1-Q4-K-M 测试速度如下（4*3090加载） --max_new_tokens 16384 2.5token --max_new_tokens 8192 3.2token --max_new_tokens 4096 3.9token --max_new_tokens 2048 4.1token ![Image](https://github.com/user-attachments/assets/216e1c85-93aa-46d1-a113-e5454a63c56d) 内存升级到512G 其他配置见图

bsakaguchi

2026-10-29 14:36:57 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:36:59 +00:00

Author

Owner

@moneyisallyouneed commented on GitHub (Feb 17, 2025):

我的4090设置8192和81920的速度一样😂不知道什么情况。应该是还没到降速的时候就回答完了

@moneyisallyouneed commented on GitHub (Feb 17, 2025): 我的4090设置8192和81920的速度一样😂不知道什么情况。应该是还没到降速的时候就回答完了

bsakaguchi commented

2026-10-29 14:36:59 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！