mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #648] 出现了不兼容情况 #477
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#477
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @yuyangtian1000 on GitHub (Feb 24, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/648
Traceback (most recent call last):
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/local_chat.py", line 184, in
fire.Fire(local_chat)
File "/mnt/sda/suzhiqi/anaconda3/envs/ktransformer/lib/python3.12/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/anaconda3/envs/ktransformer/lib/python3.12/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/anaconda3/envs/ktransformer/lib/python3.12/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/local_chat.py", line 111, in local_chat
optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/optimize/optimize.py", line 131, in optimize_and_load_gguf
load_weights(module, gguf_loader)
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 86, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 88, in load_weights
module.load()
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/operators/base_operator.py", line 63, in load
utils.load_weights(child, self.gguf_loader, self.key+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 86, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 86, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 86, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 88, in load_weights
module.load()
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/operators/base_operator.py", line 63, in load
utils.load_weights(child, self.gguf_loader, self.key+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 86, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/utils.py", line 88, in load_weights
module.load()
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/operators/linear.py", line 459, in load
self.generate_linear.load(w=w)
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/operators/linear.py", line 218, in load
w = self.load_weight(device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/operators/linear.py", line 90, in load_weight
tensors = self.load_multi(key, ["weight"], device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/operators/linear.py", line 100, in load_multi
tensors[k] = self.gguf_loader.load_gguf_tensor(key + "." + k, device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/custom_gguf.py", line 344, in load_gguf_tensor
cur_values = GGML_DEQUANTIZE_GPU[ggml_name](data[blocks_beginblock_size : blocks_end*block_size], device, target_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sda/suzhiqi/ktransformers/./ktransformers/util/custom_gguf.py", line 732, in dequantize_q4_0_gpu
raise NotImplementedError()
NotImplementedError
模型是OPEA/DeepSeek-R1-int4-sym-gguf-q4-0-inc
有办法解决吗？还是说我的操作错误了？
@ubergarm commented on GitHub (Feb 25, 2025):
我从未尝试过那个特定的GGUF。建议你查看此指南以获取运行最新版ktransformers与已知可用的GGUF模型的简单示例。
I've never tried that specific GGUF. I suggest you look at this guide for easy examples of running the latest version of ktransformers with known working GGUF models.