[GH-ISSUE #1070] [Bug] #753

New issue

Open

opened 2026-10-29 14:43:59 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:43:59 +00:00

Owner

Originally created by @chenhongxx on GitHub (Apr 7, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1070

检查清单

1. 我已经搜索过相关问题，但未能获得预期的帮助
2. 该问题在最新版本中尚未修复
3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述

(ktransformers) huadi@huadi:~/ktransformers$ python -m ktransformers.local_chat --model_path ./DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat-GGUF
2025-04-07 21:07:29,971 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/huadi/ktransformers/ktransformers/local_chat.py", line 26, in
from ktransformers.models.modeling_deepseek import DeepseekV2ForCausalLM
File "/home/huadi/ktransformers/ktransformers/models/modeling_deepseek.py", line 70, in
from flash_attn import flash_attn_func, flash_attn_varlen_func, flash_attn_with_kvcache
File "/home/huadi/anaconda3/envs/ktransformers/lib/python3.11/site-packages/flash_attn/init.py", line 3, in
from flash_attn.flash_attn_interface import (
File "/home/huadi/anaconda3/envs/ktransformers/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py", line 15, in
import flash_attn_2_cuda as flash_attn_gpu
ImportError: /home/huadi/anaconda3/envs/ktransformers/lib/python3.11/site-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationESs

复现步骤

python -m ktransformers.local_chat --model_path ./DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat-GGUF

环境信息

ubuntu22.04 4090 amd9754*2 1.1T ram cuda12.6

Originally created by @chenhongxx on GitHub (Apr 7, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1070 ### 检查清单 - [ ] 1. 我已经搜索过相关问题，但未能获得预期的帮助 - [ ] 2. 该问题在最新版本中尚未修复 - [ ] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性 - [ ] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [ ] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭 ### 问题描述 (ktransformers) huadi@huadi:~/ktransformers$ python -m ktransformers.local_chat --model_path ./DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat-GGUF 2025-04-07 21:07:29,971 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend found flashinfer Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/home/huadi/ktransformers/ktransformers/local_chat.py", line 26, in <module> from ktransformers.models.modeling_deepseek import DeepseekV2ForCausalLM File "/home/huadi/ktransformers/ktransformers/models/modeling_deepseek.py", line 70, in <module> from flash_attn import flash_attn_func, flash_attn_varlen_func, flash_attn_with_kvcache File "/home/huadi/anaconda3/envs/ktransformers/lib/python3.11/site-packages/flash_attn/__init__.py", line 3, in <module> from flash_attn.flash_attn_interface import ( File "/home/huadi/anaconda3/envs/ktransformers/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py", line 15, in <module> import flash_attn_2_cuda as flash_attn_gpu ImportError: /home/huadi/anaconda3/envs/ktransformers/lib/python3.11/site-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationESs ### 复现步骤 python -m ktransformers.local_chat --model_path ./DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat-GGUF ### 环境信息 ubuntu22.04 4090 amd9754*2 1.1T ram cuda12.6

bsakaguchi commented

2026-10-29 14:44:00 +00:00

Author

Owner

@qiyuxinlin commented on GitHub (Apr 7, 2025):

Hi @chenhongxx, I've encountered this bug before, and I was able to resolve it by switching the flash_attn version. You might want to give it a try.

@qiyuxinlin commented on GitHub (Apr 7, 2025): Hi @chenhongxx, I've encountered this bug before, and I was able to resolve it by switching the flash_attn version. You might want to give it a try.

bsakaguchi referenced this issue

2026-10-29 14:52:44 +00:00

[PR #753] [MERGED] Update local_chat.py #1475