mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1546] [Bug] torch_npu与torch_cuda环境冲突导致包导入失败 #1029
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#1029
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @GioGioBond on GitHub (Nov 4, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1546
检查清单
问题描述
显卡A800 80g
环境：
Python 3.11.14
ktransformers 0.3.2+cu126torch29fancy
torch 2.9.0+cu126
torchaudio 2.9.0+cu126
torchvision 0.24.0+cu126
分支是20251103 16：00左右拉取，先后两次拉取最新代码，仍然无法运行，报错没有setup_model_parallel，报错信息如下：
(ktransformers)_** user@user-R8488-G12:/data/wmh/ktransformers$ python -m ktransformers.local_chat --model_path /data/wmh/model/qwen14b --gguf_path ./qwen14b-GGUF
no balance_serve
2025-11-03 17:04:55,768 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/data/wmh/ktransformers/ktransformers/local_chat.py", line 258, in
fire.Fire(local_chat)
File "/home/user/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/data/wmh/ktransformers/ktransformers/local_chat.py", line 92, in local_chat
local_rank, world_size = setup_model_parallel(tp=tp)
^^^^^^^^^^^^^^^^^^^^
NameError: name 'setup_model_parallel' is not defined
经过排查，local_chat.py中，第16行import torch_npu报错，被异常捕捉跳过了setup_model_parallel导入，对于import torch_npu报错信息如下：
Traceback (most recent call last):
File "/data/wmh/ktransformers/ktransformers/local_chat.py", line 18, in
from ktransformers.util.ascend.ascend_utils import get_absort_weight, setup_model_parallel, get_tensor_parallel_group
File "/data/wmh/ktransformers/ktransformers/util/ascend/ascend_utils.py", line 5, in
import torch_npu
ModuleNotFoundError: No module named 'torch_npu'
对于torch_npu，搜索得知这个是华为显卡的torch，跟cuda版本torch是冲突的， 现在卡在这里了
复现步骤
python -m ktransformers.local_chat --model_path /data/wmh/model/qwen14b --gguf_path ./qwen14b-GGUF
用的模型是qwen2.5 14b
环境信息
==================== 系统信息 ====================
操作系统: Ubuntu 22.04.2 LTS
内核版本: 6.8.0-60-generic
架构: x86_64
==================== CPU 信息 ====================
Model name: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
Thread(s) per core: 2
Core(s) per socket: 32
Socket(s): 2
逻辑 CPU 核心数: 128
==================== GPU 信息 ====================
NVIDIA A800-SXM4-80GB, 560.35.05, 81920, 4, 16
@YHCyhc20010606 commented on GitHub (Nov 7, 2025):
我也遇到了这个问题 哥们 解决了吗？
@zzyuanyi commented on GitHub (Nov 12, 2025):
我也遇到了这个问题 哥们 解决了吗？
@GioGioBond commented on GitHub (Nov 12, 2025):
还没，导入torch_npu的地方太多了，等官方修复，或者自己把相关部分都注释掉
@GioGioBond commented on GitHub (Nov 12, 2025):
没呢，等官方修复呢，导入torch_npu的地方太多了
@zzyuanyi commented on GitHub (Nov 12, 2025):
感谢回复
------------------ 原始邮件 ------------------
发件人: "kvcache-ai/ktransformers" @.>;
发送时间: 2025年11月12日(星期三) 上午10:20
@.>;
@.@.>;
主题: Re: [kvcache-ai/ktransformers] [Bug] torch_npu与torch_cuda环境冲突导致包导入失败 (Issue #1546)
GioGioBond left a comment (kvcache-ai/ktransformers#1546)
我也遇到了这个问题 哥们 解决了吗？
还没
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>