[GH-ISSUE #113] RuntimeError CUDA error when running Infinite Bench #64

New issue

Closed

opened 2026-10-29 14:33:17 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:33:17 +00:00

Owner

Originally created by @Flitieter on GitHub (Nov 10, 2024).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/113

Hi expert, recently I ran the code in Infinite Bench (the GPU is RTX4090 24GB, and the cuda version is 12.1.105).

However, when the code ran the dataset about kv_retrieval, it threw an exception "RuntimeError CUDA error"(the specialed error case id is 281, and the code was ok in the dataset Passkey). I checked the pips' version and found they are all consistent with what has been written in the document "ktransformers/server/requirements.txt".

So I want to know whether there are still some problems in my environment.

Here are some pips' versions in my environment:

python==3.11.0
torch==2.3.0
transformers == 4.43.2
fastapi==0.115.4
langchain==0.3.7
blessed==1.20.0
accelerate==1.1.0
sentencepiece==0.2.0

And the complete error message is:

Originally created by @Flitieter on GitHub (Nov 10, 2024). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/113 Hi expert, recently I ran the code in Infinite Bench (the GPU is RTX4090 24GB, and the cuda version is 12.1.105). However, when the code ran the dataset about kv_retrieval, it threw an exception "RuntimeError CUDA error"(the specialed error case id is 281, and the code was ok in the dataset Passkey). I checked the pips' version and found they are all consistent with what has been written in the document "ktransformers/server/requirements.txt". So I want to know whether there are still some problems in my environment. Here are some pips' versions in my environment: ``` python==3.11.0 torch==2.3.0 transformers == 4.43.2 fastapi==0.115.4 langchain==0.3.7 blessed==1.20.0 accelerate==1.1.0 sentencepiece==0.2.0 ``` And the complete error message is: ![error](https://github.com/user-attachments/assets/0751346d-cdd0-4d67-ad0c-2a3b93513ef5)

bsakaguchi

2026-10-29 14:33:17 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:33:18 +00:00

Author

Owner

@KMSorSMS commented on GitHub (Feb 11, 2025):

What's your command in launch.json? You can try our newest version V0.2 with DeepSeek R1

@KMSorSMS commented on GitHub (Feb 11, 2025): What's your command in launch.json? You can try our newest version V0.2 with DeepSeek R1

bsakaguchi commented

2026-10-29 14:33:18 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！

bsakaguchi referenced this issue

2026-10-29 14:51:33 +00:00

[PR #64] [MERGED] docs: update long_context_introduction.md #1338