[GH-ISSUE #486] 关于启动参数 repetition_penalty 的疑问 #345

New issue

Closed

opened 2026-10-29 14:38:15 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 14:38:15 +00:00

Owner

Originally created by @CallmeZhangChenchen on GitHub (Feb 19, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/486

好像启动服务的时候可以设置这个参数，然后通过openai接口调用的时候，也能设置这个参数repetition_penalty

repetition_penalty这个参数调到 -2.0，也没见它重复输出，输出偏短

另外观察到启动的时候占用15G显存，然后慢慢的就占满了，这时候最大输入200个token，就报OOM了

Originally created by @CallmeZhangChenchen on GitHub (Feb 19, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/486 好像启动服务的时候可以设置这个参数，然后通过openai接口调用的时候，也能设置这个参数repetition_penalty repetition_penalty这个参数调到 -2.0，也没见它重复输出，输出偏短另外观察到启动的时候占用15G显存，然后慢慢的就占满了，这时候最大输入200个token，就报OOM了

bsakaguchi

2026-10-29 14:38:15 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:38:15 +00:00

Author

Owner

@CallmeZhangChenchen commented on GitHub (Feb 19, 2025):

长度超过 --cache_lens 之后。服务就会崩溃

@CallmeZhangChenchen commented on GitHub (Feb 19, 2025): 长度超过 --cache_lens 之后。服务就会崩溃

bsakaguchi commented

2026-10-29 14:38:16 +00:00

Author

Owner

@CallmeZhangChenchen commented on GitHub (Feb 19, 2025):

The assistant's /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:94: operator(): block: [0,0,0],
thread: [0,0,0] Assertion -sizes[i] <= index && index < sizes[i] && "index out of bounds" failed.

输入加输出超过。max_new_tokens 之后就报了这个错误

@CallmeZhangChenchen commented on GitHub (Feb 19, 2025): The assistant's /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:94: operator(): block: [0,0,0], thread: [0,0,0] Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed. 输入加输出超过。max_new_tokens 之后就报了这个错误

bsakaguchi commented

2026-10-29 14:38:16 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！