mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #273] win11环境下，运行官网指定模型报错 #170
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#170
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @wuwenthink on GitHub (Feb 14, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/273
报错信息如下：运行的是internlm2_5-7b-chat-1m-Q4_K_M
(ktransformers) E:\AI_Chat_ktransformers>python -m ktransformers.local_chat --model_path ./_Model_info --gguf_path E:\AI_Chat_ktransformers_Model_GGUF\ --cpu_infer 4 --max_new_tokens 1000
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
please input the path of your rule file(yaml file containing optimize rules):E:\AI_Chat_ktransformers\ktransformers\optimize\optimize_rules\Internlm2_5-7b-Chat-1m.yaml
Injecting model as default
Injecting model.tok_embeddings as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.attention as default
Injecting model.layers.0.attention.wqkv as default
Injecting model.layers.0.attention.wo as default
Injecting model.layers.0.attention.rotary_emb as default
Injecting model.layers.0.feed_forward as default
Injecting model.layers.0.feed_forward.w1 as default
Injecting model.layers.0.feed_forward.w3 as default
Injecting model.layers.0.feed_forward.w2 as default
Injecting model.layers.0.feed_forward.act_fn as default
Injecting model.layers.0.attention_norm as default
Injecting model.layers.0.ffn_norm as default
Injecting model.layers.1 as default
Injecting model.layers.1.attention as default
Injecting model.layers.1.attention.wqkv as default
Injecting model.layers.1.attention.wo as default
Injecting model.layers.1.attention.rotary_emb as default
Injecting model.layers.1.feed_forward as default
Injecting model.layers.1.feed_forward.w1 as default
Injecting model.layers.1.feed_forward.w3 as default
Injecting model.layers.1.feed_forward.w2 as default
Injecting model.layers.1.feed_forward.act_fn as default
Injecting model.layers.1.attention_norm as default
Injecting model.layers.1.ffn_norm as default
Injecting model.layers.2 as default
Injecting model.layers.2.attention as default
Injecting model.layers.2.attention.wqkv as default
Injecting model.layers.2.attention.wo as default
Injecting model.layers.2.attention.rotary_emb as default
Injecting model.layers.2.feed_forward as default
Injecting model.layers.2.feed_forward.w1 as default
Injecting model.layers.2.feed_forward.w3 as default
Injecting model.layers.2.feed_forward.w2 as default
Injecting model.layers.2.feed_forward.act_fn as default
Injecting model.layers.2.attention_norm as default
Injecting model.layers.2.ffn_norm as default
Injecting model.layers.3 as default
Injecting model.layers.3.attention as default
Injecting model.layers.3.attention.wqkv as default
Injecting model.layers.3.attention.wo as default
Injecting model.layers.3.attention.rotary_emb as default
Injecting model.layers.3.feed_forward as default
Injecting model.layers.3.feed_forward.w1 as default
Injecting model.layers.3.feed_forward.w3 as default
Injecting model.layers.3.feed_forward.w2 as default
Injecting model.layers.3.feed_forward.act_fn as default
Injecting model.layers.3.attention_norm as default
Injecting model.layers.3.ffn_norm as default
Injecting model.layers.4 as default
Injecting model.layers.4.attention as default
Injecting model.layers.4.attention.wqkv as default
Injecting model.layers.4.attention.wo as default
Injecting model.layers.4.attention.rotary_emb as default
Injecting model.layers.4.feed_forward as default
Injecting model.layers.4.feed_forward.w1 as default
Injecting model.layers.4.feed_forward.w3 as default
Injecting model.layers.4.feed_forward.w2 as default
Injecting model.layers.4.feed_forward.act_fn as default
Injecting model.layers.4.attention_norm as default
Injecting model.layers.4.ffn_norm as default
Injecting model.layers.5 as default
Injecting model.layers.5.attention as default
Injecting model.layers.5.attention.wqkv as default
Injecting model.layers.5.attention.wo as default
Injecting model.layers.5.attention.rotary_emb as default
Injecting model.layers.5.feed_forward as default
Injecting model.layers.5.feed_forward.w1 as default
Injecting model.layers.5.feed_forward.w3 as default
Injecting model.layers.5.feed_forward.w2 as default
Injecting model.layers.5.feed_forward.act_fn as default
Injecting model.layers.5.attention_norm as default
Injecting model.layers.5.ffn_norm as default
Injecting model.layers.6 as default
Injecting model.layers.6.attention as default
Injecting model.layers.6.attention.wqkv as default
Injecting model.layers.6.attention.wo as default
Injecting model.layers.6.attention.rotary_emb as default
Injecting model.layers.6.feed_forward as default
Injecting model.layers.6.feed_forward.w1 as default
Injecting model.layers.6.feed_forward.w3 as default
Injecting model.layers.6.feed_forward.w2 as default
Injecting model.layers.6.feed_forward.act_fn as default
Injecting model.layers.6.attention_norm as default
Injecting model.layers.6.ffn_norm as default
Injecting model.layers.7 as default
Injecting model.layers.7.attention as default
Injecting model.layers.7.attention.wqkv as default
Injecting model.layers.7.attention.wo as default
Injecting model.layers.7.attention.rotary_emb as default
Injecting model.layers.7.feed_forward as default
Injecting model.layers.7.feed_forward.w1 as default
Injecting model.layers.7.feed_forward.w3 as default
Injecting model.layers.7.feed_forward.w2 as default
Injecting model.layers.7.feed_forward.act_fn as default
Injecting model.layers.7.attention_norm as default
Injecting model.layers.7.ffn_norm as default
Injecting model.layers.8 as default
Injecting model.layers.8.attention as default
Injecting model.layers.8.attention.wqkv as default
Injecting model.layers.8.attention.wo as default
Injecting model.layers.8.attention.rotary_emb as default
Injecting model.layers.8.feed_forward as default
Injecting model.layers.8.feed_forward.w1 as default
Injecting model.layers.8.feed_forward.w3 as default
Injecting model.layers.8.feed_forward.w2 as default
Injecting model.layers.8.feed_forward.act_fn as default
Injecting model.layers.8.attention_norm as default
Injecting model.layers.8.ffn_norm as default
Injecting model.layers.9 as default
Injecting model.layers.9.attention as default
Injecting model.layers.9.attention.wqkv as default
Injecting model.layers.9.attention.wo as default
Injecting model.layers.9.attention.rotary_emb as default
Injecting model.layers.9.feed_forward as default
Injecting model.layers.9.feed_forward.w1 as default
Injecting model.layers.9.feed_forward.w3 as default
Injecting model.layers.9.feed_forward.w2 as default
Injecting model.layers.9.feed_forward.act_fn as default
Injecting model.layers.9.attention_norm as default
Injecting model.layers.9.ffn_norm as default
Injecting model.layers.10 as default
Injecting model.layers.10.attention as default
Injecting model.layers.10.attention.wqkv as default
Injecting model.layers.10.attention.wo as default
Injecting model.layers.10.attention.rotary_emb as default
Injecting model.layers.10.feed_forward as default
Injecting model.layers.10.feed_forward.w1 as default
Injecting model.layers.10.feed_forward.w3 as default
Injecting model.layers.10.feed_forward.w2 as default
Injecting model.layers.10.feed_forward.act_fn as default
Injecting model.layers.10.attention_norm as default
Injecting model.layers.10.ffn_norm as default
Injecting model.layers.11 as default
Injecting model.layers.11.attention as default
Injecting model.layers.11.attention.wqkv as default
Injecting model.layers.11.attention.wo as default
Injecting model.layers.11.attention.rotary_emb as default
Injecting model.layers.11.feed_forward as default
Injecting model.layers.11.feed_forward.w1 as default
Injecting model.layers.11.feed_forward.w3 as default
Injecting model.layers.11.feed_forward.w2 as default
Injecting model.layers.11.feed_forward.act_fn as default
Injecting model.layers.11.attention_norm as default
Injecting model.layers.11.ffn_norm as default
Injecting model.layers.12 as default
Injecting model.layers.12.attention as default
Injecting model.layers.12.attention.wqkv as default
Injecting model.layers.12.attention.wo as default
Injecting model.layers.12.attention.rotary_emb as default
Injecting model.layers.12.feed_forward as default
Injecting model.layers.12.feed_forward.w1 as default
Injecting model.layers.12.feed_forward.w3 as default
Injecting model.layers.12.feed_forward.w2 as default
Injecting model.layers.12.feed_forward.act_fn as default
Injecting model.layers.12.attention_norm as default
Injecting model.layers.12.ffn_norm as default
Injecting model.layers.13 as default
Injecting model.layers.13.attention as default
Injecting model.layers.13.attention.wqkv as default
Injecting model.layers.13.attention.wo as default
Injecting model.layers.13.attention.rotary_emb as default
Injecting model.layers.13.feed_forward as default
Injecting model.layers.13.feed_forward.w1 as default
Injecting model.layers.13.feed_forward.w3 as default
Injecting model.layers.13.feed_forward.w2 as default
Injecting model.layers.13.feed_forward.act_fn as default
Injecting model.layers.13.attention_norm as default
Injecting model.layers.13.ffn_norm as default
Injecting model.layers.14 as default
Injecting model.layers.14.attention as default
Injecting model.layers.14.attention.wqkv as default
Injecting model.layers.14.attention.wo as default
Injecting model.layers.14.attention.rotary_emb as default
Injecting model.layers.14.feed_forward as default
Injecting model.layers.14.feed_forward.w1 as default
Injecting model.layers.14.feed_forward.w3 as default
Injecting model.layers.14.feed_forward.w2 as default
Injecting model.layers.14.feed_forward.act_fn as default
Injecting model.layers.14.attention_norm as default
Injecting model.layers.14.ffn_norm as default
Injecting model.layers.15 as default
Injecting model.layers.15.attention as default
Injecting model.layers.15.attention.wqkv as default
Injecting model.layers.15.attention.wo as default
Injecting model.layers.15.attention.rotary_emb as default
Injecting model.layers.15.feed_forward as default
Injecting model.layers.15.feed_forward.w1 as default
Injecting model.layers.15.feed_forward.w3 as default
Injecting model.layers.15.feed_forward.w2 as default
Injecting model.layers.15.feed_forward.act_fn as default
Injecting model.layers.15.attention_norm as default
Injecting model.layers.15.ffn_norm as default
Injecting model.layers.16 as default
Injecting model.layers.16.attention as default
Injecting model.layers.16.attention.wqkv as default
Injecting model.layers.16.attention.wo as default
Injecting model.layers.16.attention.rotary_emb as default
Injecting model.layers.16.feed_forward as default
Injecting model.layers.16.feed_forward.w1 as default
Injecting model.layers.16.feed_forward.w3 as default
Injecting model.layers.16.feed_forward.w2 as default
Injecting model.layers.16.feed_forward.act_fn as default
Injecting model.layers.16.attention_norm as default
Injecting model.layers.16.ffn_norm as default
Injecting model.layers.17 as default
Injecting model.layers.17.attention as default
Injecting model.layers.17.attention.wqkv as default
Injecting model.layers.17.attention.wo as default
Injecting model.layers.17.attention.rotary_emb as default
Injecting model.layers.17.feed_forward as default
Injecting model.layers.17.feed_forward.w1 as default
Injecting model.layers.17.feed_forward.w3 as default
Injecting model.layers.17.feed_forward.w2 as default
Injecting model.layers.17.feed_forward.act_fn as default
Injecting model.layers.17.attention_norm as default
Injecting model.layers.17.ffn_norm as default
Injecting model.layers.18 as default
Injecting model.layers.18.attention as default
Injecting model.layers.18.attention.wqkv as default
Injecting model.layers.18.attention.wo as default
Injecting model.layers.18.attention.rotary_emb as default
Injecting model.layers.18.feed_forward as default
Injecting model.layers.18.feed_forward.w1 as default
Injecting model.layers.18.feed_forward.w3 as default
Injecting model.layers.18.feed_forward.w2 as default
Injecting model.layers.18.feed_forward.act_fn as default
Injecting model.layers.18.attention_norm as default
Injecting model.layers.18.ffn_norm as default
Injecting model.layers.19 as default
Injecting model.layers.19.attention as default
Injecting model.layers.19.attention.wqkv as default
Injecting model.layers.19.attention.wo as default
Injecting model.layers.19.attention.rotary_emb as default
Injecting model.layers.19.feed_forward as default
Injecting model.layers.19.feed_forward.w1 as default
Injecting model.layers.19.feed_forward.w3 as default
Injecting model.layers.19.feed_forward.w2 as default
Injecting model.layers.19.feed_forward.act_fn as default
Injecting model.layers.19.attention_norm as default
Injecting model.layers.19.ffn_norm as default
Injecting model.layers.20 as default
Injecting model.layers.20.attention as default
Injecting model.layers.20.attention.wqkv as default
Injecting model.layers.20.attention.wo as default
Injecting model.layers.20.attention.rotary_emb as default
Injecting model.layers.20.feed_forward as default
Injecting model.layers.20.feed_forward.w1 as default
Injecting model.layers.20.feed_forward.w3 as default
Injecting model.layers.20.feed_forward.w2 as default
Injecting model.layers.20.feed_forward.act_fn as default
Injecting model.layers.20.attention_norm as default
Injecting model.layers.20.ffn_norm as default
Injecting model.layers.21 as default
Injecting model.layers.21.attention as default
Injecting model.layers.21.attention.wqkv as default
Injecting model.layers.21.attention.wo as default
Injecting model.layers.21.attention.rotary_emb as default
Injecting model.layers.21.feed_forward as default
Injecting model.layers.21.feed_forward.w1 as default
Injecting model.layers.21.feed_forward.w3 as default
Injecting model.layers.21.feed_forward.w2 as default
Injecting model.layers.21.feed_forward.act_fn as default
Injecting model.layers.21.attention_norm as default
Injecting model.layers.21.ffn_norm as default
Injecting model.layers.22 as default
Injecting model.layers.22.attention as default
Injecting model.layers.22.attention.wqkv as default
Injecting model.layers.22.attention.wo as default
Injecting model.layers.22.attention.rotary_emb as default
Injecting model.layers.22.feed_forward as default
Injecting model.layers.22.feed_forward.w1 as default
Injecting model.layers.22.feed_forward.w3 as default
Injecting model.layers.22.feed_forward.w2 as default
Injecting model.layers.22.feed_forward.act_fn as default
Injecting model.layers.22.attention_norm as default
Injecting model.layers.22.ffn_norm as default
Injecting model.layers.23 as default
Injecting model.layers.23.attention as default
Injecting model.layers.23.attention.wqkv as default
Injecting model.layers.23.attention.wo as default
Injecting model.layers.23.attention.rotary_emb as default
Injecting model.layers.23.feed_forward as default
Injecting model.layers.23.feed_forward.w1 as default
Injecting model.layers.23.feed_forward.w3 as default
Injecting model.layers.23.feed_forward.w2 as default
Injecting model.layers.23.feed_forward.act_fn as default
Injecting model.layers.23.attention_norm as default
Injecting model.layers.23.ffn_norm as default
Injecting model.layers.24 as default
Injecting model.layers.24.attention as default
Injecting model.layers.24.attention.wqkv as default
Injecting model.layers.24.attention.wo as default
Injecting model.layers.24.attention.rotary_emb as default
Injecting model.layers.24.feed_forward as default
Injecting model.layers.24.feed_forward.w1 as default
Injecting model.layers.24.feed_forward.w3 as default
Injecting model.layers.24.feed_forward.w2 as default
Injecting model.layers.24.feed_forward.act_fn as default
Injecting model.layers.24.attention_norm as default
Injecting model.layers.24.ffn_norm as default
Injecting model.layers.25 as default
Injecting model.layers.25.attention as default
Injecting model.layers.25.attention.wqkv as default
Injecting model.layers.25.attention.wo as default
Injecting model.layers.25.attention.rotary_emb as default
Injecting model.layers.25.feed_forward as default
Injecting model.layers.25.feed_forward.w1 as default
Injecting model.layers.25.feed_forward.w3 as default
Injecting model.layers.25.feed_forward.w2 as default
Injecting model.layers.25.feed_forward.act_fn as default
Injecting model.layers.25.attention_norm as default
Injecting model.layers.25.ffn_norm as default
Injecting model.layers.26 as default
Injecting model.layers.26.attention as default
Injecting model.layers.26.attention.wqkv as default
Injecting model.layers.26.attention.wo as default
Injecting model.layers.26.attention.rotary_emb as default
Injecting model.layers.26.feed_forward as default
Injecting model.layers.26.feed_forward.w1 as default
Injecting model.layers.26.feed_forward.w3 as default
Injecting model.layers.26.feed_forward.w2 as default
Injecting model.layers.26.feed_forward.act_fn as default
Injecting model.layers.26.attention_norm as default
Injecting model.layers.26.ffn_norm as default
Injecting model.layers.27 as default
Injecting model.layers.27.attention as default
Injecting model.layers.27.attention.wqkv as default
Injecting model.layers.27.attention.wo as default
Injecting model.layers.27.attention.rotary_emb as default
Injecting model.layers.27.feed_forward as default
Injecting model.layers.27.feed_forward.w1 as default
Injecting model.layers.27.feed_forward.w3 as default
Injecting model.layers.27.feed_forward.w2 as default
Injecting model.layers.27.feed_forward.act_fn as default
Injecting model.layers.27.attention_norm as default
Injecting model.layers.27.ffn_norm as default
Injecting model.layers.28 as default
Injecting model.layers.28.attention as default
Injecting model.layers.28.attention.wqkv as default
Injecting model.layers.28.attention.wo as default
Injecting model.layers.28.attention.rotary_emb as default
Injecting model.layers.28.feed_forward as default
Injecting model.layers.28.feed_forward.w1 as default
Injecting model.layers.28.feed_forward.w3 as default
Injecting model.layers.28.feed_forward.w2 as default
Injecting model.layers.28.feed_forward.act_fn as default
Injecting model.layers.28.attention_norm as default
Injecting model.layers.28.ffn_norm as default
Injecting model.layers.29 as default
Injecting model.layers.29.attention as default
Injecting model.layers.29.attention.wqkv as default
Injecting model.layers.29.attention.wo as default
Injecting model.layers.29.attention.rotary_emb as default
Injecting model.layers.29.feed_forward as default
Injecting model.layers.29.feed_forward.w1 as default
Injecting model.layers.29.feed_forward.w3 as default
Injecting model.layers.29.feed_forward.w2 as default
Injecting model.layers.29.feed_forward.act_fn as default
Injecting model.layers.29.attention_norm as default
Injecting model.layers.29.ffn_norm as default
Injecting model.layers.30 as default
Injecting model.layers.30.attention as default
Injecting model.layers.30.attention.wqkv as default
Injecting model.layers.30.attention.wo as default
Injecting model.layers.30.attention.rotary_emb as default
Injecting model.layers.30.feed_forward as default
Injecting model.layers.30.feed_forward.w1 as default
Injecting model.layers.30.feed_forward.w3 as default
Injecting model.layers.30.feed_forward.w2 as default
Injecting model.layers.30.feed_forward.act_fn as default
Injecting model.layers.30.attention_norm as default
Injecting model.layers.30.ffn_norm as default
Injecting model.layers.31 as default
Injecting model.layers.31.attention as default
Injecting model.layers.31.attention.wqkv as default
Injecting model.layers.31.attention.wo as default
Injecting model.layers.31.attention.rotary_emb as default
Injecting model.layers.31.feed_forward as default
Injecting model.layers.31.feed_forward.w1 as default
Injecting model.layers.31.feed_forward.w3 as default
Injecting model.layers.31.feed_forward.w2 as default
Injecting model.layers.31.feed_forward.act_fn as default
Injecting model.layers.31.attention_norm as default
Injecting model.layers.31.ffn_norm as default
Injecting model.norm as default
Injecting output as default
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "E:\AI_Chat_ktransformers\ktransformers\local_chat.py", line 179, in
fire.Fire(local_chat)
File "C:\Users\wuwenthink.conda\envs\ktransformers\Lib\site-packages\fire\core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\wuwenthink.conda\envs\ktransformers\Lib\site-packages\fire\core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\wuwenthink.conda\envs\ktransformers\Lib\site-packages\fire\core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "E:\AI_Chat_ktransformers\ktransformers\local_chat.py", line 110, in local_chat
optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
File "E:\AI_Chat_ktransformers\ktransformers\optimize\optimize.py", line 129, in optimize_and_load_gguf
load_weights(module, gguf_loader)
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 83, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 83, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 81, in load_weights
load_cur_state_dict(module, gguf_loader, prefix)
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 76, in load_cur_state_dict
raise Exception(f"can't find {translated_key} in GGUF file!")
Exception: can't find model.tok_embeddings.weight in GGUF file!
(ktransformers) E:\AI_Chat_ktransformers>python -m ktransformers.local_chat --model_path _Model_info --gguf_path _Model_GGUF --cpu_infer 4 --optimize_rule_path E:\AI_Chat_ktransformers\ktransformers\optimize\optimize_rules\Internlm2_5-7b-Chat-1m.yaml
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
Injecting model as default
Injecting model.tok_embeddings as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.attention as default
Injecting model.layers.0.attention.wqkv as default
Injecting model.layers.0.attention.wo as default
Injecting model.layers.0.attention.rotary_emb as default
Injecting model.layers.0.feed_forward as default
Injecting model.layers.0.feed_forward.w1 as default
Injecting model.layers.0.feed_forward.w3 as default
Injecting model.layers.0.feed_forward.w2 as default
Injecting model.layers.0.feed_forward.act_fn as default
Injecting model.layers.0.attention_norm as default
Injecting model.layers.0.ffn_norm as default
Injecting model.layers.1 as default
Injecting model.layers.1.attention as default
Injecting model.layers.1.attention.wqkv as default
Injecting model.layers.1.attention.wo as default
Injecting model.layers.1.attention.rotary_emb as default
Injecting model.layers.1.feed_forward as default
Injecting model.layers.1.feed_forward.w1 as default
Injecting model.layers.1.feed_forward.w3 as default
Injecting model.layers.1.feed_forward.w2 as default
Injecting model.layers.1.feed_forward.act_fn as default
Injecting model.layers.1.attention_norm as default
Injecting model.layers.1.ffn_norm as default
Injecting model.layers.2 as default
Injecting model.layers.2.attention as default
Injecting model.layers.2.attention.wqkv as default
Injecting model.layers.2.attention.wo as default
Injecting model.layers.2.attention.rotary_emb as default
Injecting model.layers.2.feed_forward as default
Injecting model.layers.2.feed_forward.w1 as default
Injecting model.layers.2.feed_forward.w3 as default
Injecting model.layers.2.feed_forward.w2 as default
Injecting model.layers.2.feed_forward.act_fn as default
Injecting model.layers.2.attention_norm as default
Injecting model.layers.2.ffn_norm as default
Injecting model.layers.3 as default
Injecting model.layers.3.attention as default
Injecting model.layers.3.attention.wqkv as default
Injecting model.layers.3.attention.wo as default
Injecting model.layers.3.attention.rotary_emb as default
Injecting model.layers.3.feed_forward as default
Injecting model.layers.3.feed_forward.w1 as default
Injecting model.layers.3.feed_forward.w3 as default
Injecting model.layers.3.feed_forward.w2 as default
Injecting model.layers.3.feed_forward.act_fn as default
Injecting model.layers.3.attention_norm as default
Injecting model.layers.3.ffn_norm as default
Injecting model.layers.4 as default
Injecting model.layers.4.attention as default
Injecting model.layers.4.attention.wqkv as default
Injecting model.layers.4.attention.wo as default
Injecting model.layers.4.attention.rotary_emb as default
Injecting model.layers.4.feed_forward as default
Injecting model.layers.4.feed_forward.w1 as default
Injecting model.layers.4.feed_forward.w3 as default
Injecting model.layers.4.feed_forward.w2 as default
Injecting model.layers.4.feed_forward.act_fn as default
Injecting model.layers.4.attention_norm as default
Injecting model.layers.4.ffn_norm as default
Injecting model.layers.5 as default
Injecting model.layers.5.attention as default
Injecting model.layers.5.attention.wqkv as default
Injecting model.layers.5.attention.wo as default
Injecting model.layers.5.attention.rotary_emb as default
Injecting model.layers.5.feed_forward as default
Injecting model.layers.5.feed_forward.w1 as default
Injecting model.layers.5.feed_forward.w3 as default
Injecting model.layers.5.feed_forward.w2 as default
Injecting model.layers.5.feed_forward.act_fn as default
Injecting model.layers.5.attention_norm as default
Injecting model.layers.5.ffn_norm as default
Injecting model.layers.6 as default
Injecting model.layers.6.attention as default
Injecting model.layers.6.attention.wqkv as default
Injecting model.layers.6.attention.wo as default
Injecting model.layers.6.attention.rotary_emb as default
Injecting model.layers.6.feed_forward as default
Injecting model.layers.6.feed_forward.w1 as default
Injecting model.layers.6.feed_forward.w3 as default
Injecting model.layers.6.feed_forward.w2 as default
Injecting model.layers.6.feed_forward.act_fn as default
Injecting model.layers.6.attention_norm as default
Injecting model.layers.6.ffn_norm as default
Injecting model.layers.7 as default
Injecting model.layers.7.attention as default
Injecting model.layers.7.attention.wqkv as default
Injecting model.layers.7.attention.wo as default
Injecting model.layers.7.attention.rotary_emb as default
Injecting model.layers.7.feed_forward as default
Injecting model.layers.7.feed_forward.w1 as default
Injecting model.layers.7.feed_forward.w3 as default
Injecting model.layers.7.feed_forward.w2 as default
Injecting model.layers.7.feed_forward.act_fn as default
Injecting model.layers.7.attention_norm as default
Injecting model.layers.7.ffn_norm as default
Injecting model.layers.8 as default
Injecting model.layers.8.attention as default
Injecting model.layers.8.attention.wqkv as default
Injecting model.layers.8.attention.wo as default
Injecting model.layers.8.attention.rotary_emb as default
Injecting model.layers.8.feed_forward as default
Injecting model.layers.8.feed_forward.w1 as default
Injecting model.layers.8.feed_forward.w3 as default
Injecting model.layers.8.feed_forward.w2 as default
Injecting model.layers.8.feed_forward.act_fn as default
Injecting model.layers.8.attention_norm as default
Injecting model.layers.8.ffn_norm as default
Injecting model.layers.9 as default
Injecting model.layers.9.attention as default
Injecting model.layers.9.attention.wqkv as default
Injecting model.layers.9.attention.wo as default
Injecting model.layers.9.attention.rotary_emb as default
Injecting model.layers.9.feed_forward as default
Injecting model.layers.9.feed_forward.w1 as default
Injecting model.layers.9.feed_forward.w3 as default
Injecting model.layers.9.feed_forward.w2 as default
Injecting model.layers.9.feed_forward.act_fn as default
Injecting model.layers.9.attention_norm as default
Injecting model.layers.9.ffn_norm as default
Injecting model.layers.10 as default
Injecting model.layers.10.attention as default
Injecting model.layers.10.attention.wqkv as default
Injecting model.layers.10.attention.wo as default
Injecting model.layers.10.attention.rotary_emb as default
Injecting model.layers.10.feed_forward as default
Injecting model.layers.10.feed_forward.w1 as default
Injecting model.layers.10.feed_forward.w3 as default
Injecting model.layers.10.feed_forward.w2 as default
Injecting model.layers.10.feed_forward.act_fn as default
Injecting model.layers.10.attention_norm as default
Injecting model.layers.10.ffn_norm as default
Injecting model.layers.11 as default
Injecting model.layers.11.attention as default
Injecting model.layers.11.attention.wqkv as default
Injecting model.layers.11.attention.wo as default
Injecting model.layers.11.attention.rotary_emb as default
Injecting model.layers.11.feed_forward as default
Injecting model.layers.11.feed_forward.w1 as default
Injecting model.layers.11.feed_forward.w3 as default
Injecting model.layers.11.feed_forward.w2 as default
Injecting model.layers.11.feed_forward.act_fn as default
Injecting model.layers.11.attention_norm as default
Injecting model.layers.11.ffn_norm as default
Injecting model.layers.12 as default
Injecting model.layers.12.attention as default
Injecting model.layers.12.attention.wqkv as default
Injecting model.layers.12.attention.wo as default
Injecting model.layers.12.attention.rotary_emb as default
Injecting model.layers.12.feed_forward as default
Injecting model.layers.12.feed_forward.w1 as default
Injecting model.layers.12.feed_forward.w3 as default
Injecting model.layers.12.feed_forward.w2 as default
Injecting model.layers.12.feed_forward.act_fn as default
Injecting model.layers.12.attention_norm as default
Injecting model.layers.12.ffn_norm as default
Injecting model.layers.13 as default
Injecting model.layers.13.attention as default
Injecting model.layers.13.attention.wqkv as default
Injecting model.layers.13.attention.wo as default
Injecting model.layers.13.attention.rotary_emb as default
Injecting model.layers.13.feed_forward as default
Injecting model.layers.13.feed_forward.w1 as default
Injecting model.layers.13.feed_forward.w3 as default
Injecting model.layers.13.feed_forward.w2 as default
Injecting model.layers.13.feed_forward.act_fn as default
Injecting model.layers.13.attention_norm as default
Injecting model.layers.13.ffn_norm as default
Injecting model.layers.14 as default
Injecting model.layers.14.attention as default
Injecting model.layers.14.attention.wqkv as default
Injecting model.layers.14.attention.wo as default
Injecting model.layers.14.attention.rotary_emb as default
Injecting model.layers.14.feed_forward as default
Injecting model.layers.14.feed_forward.w1 as default
Injecting model.layers.14.feed_forward.w3 as default
Injecting model.layers.14.feed_forward.w2 as default
Injecting model.layers.14.feed_forward.act_fn as default
Injecting model.layers.14.attention_norm as default
Injecting model.layers.14.ffn_norm as default
Injecting model.layers.15 as default
Injecting model.layers.15.attention as default
Injecting model.layers.15.attention.wqkv as default
Injecting model.layers.15.attention.wo as default
Injecting model.layers.15.attention.rotary_emb as default
Injecting model.layers.15.feed_forward as default
Injecting model.layers.15.feed_forward.w1 as default
Injecting model.layers.15.feed_forward.w3 as default
Injecting model.layers.15.feed_forward.w2 as default
Injecting model.layers.15.feed_forward.act_fn as default
Injecting model.layers.15.attention_norm as default
Injecting model.layers.15.ffn_norm as default
Injecting model.layers.16 as default
Injecting model.layers.16.attention as default
Injecting model.layers.16.attention.wqkv as default
Injecting model.layers.16.attention.wo as default
Injecting model.layers.16.attention.rotary_emb as default
Injecting model.layers.16.feed_forward as default
Injecting model.layers.16.feed_forward.w1 as default
Injecting model.layers.16.feed_forward.w3 as default
Injecting model.layers.16.feed_forward.w2 as default
Injecting model.layers.16.feed_forward.act_fn as default
Injecting model.layers.16.attention_norm as default
Injecting model.layers.16.ffn_norm as default
Injecting model.layers.17 as default
Injecting model.layers.17.attention as default
Injecting model.layers.17.attention.wqkv as default
Injecting model.layers.17.attention.wo as default
Injecting model.layers.17.attention.rotary_emb as default
Injecting model.layers.17.feed_forward as default
Injecting model.layers.17.feed_forward.w1 as default
Injecting model.layers.17.feed_forward.w3 as default
Injecting model.layers.17.feed_forward.w2 as default
Injecting model.layers.17.feed_forward.act_fn as default
Injecting model.layers.17.attention_norm as default
Injecting model.layers.17.ffn_norm as default
Injecting model.layers.18 as default
Injecting model.layers.18.attention as default
Injecting model.layers.18.attention.wqkv as default
Injecting model.layers.18.attention.wo as default
Injecting model.layers.18.attention.rotary_emb as default
Injecting model.layers.18.feed_forward as default
Injecting model.layers.18.feed_forward.w1 as default
Injecting model.layers.18.feed_forward.w3 as default
Injecting model.layers.18.feed_forward.w2 as default
Injecting model.layers.18.feed_forward.act_fn as default
Injecting model.layers.18.attention_norm as default
Injecting model.layers.18.ffn_norm as default
Injecting model.layers.19 as default
Injecting model.layers.19.attention as default
Injecting model.layers.19.attention.wqkv as default
Injecting model.layers.19.attention.wo as default
Injecting model.layers.19.attention.rotary_emb as default
Injecting model.layers.19.feed_forward as default
Injecting model.layers.19.feed_forward.w1 as default
Injecting model.layers.19.feed_forward.w3 as default
Injecting model.layers.19.feed_forward.w2 as default
Injecting model.layers.19.feed_forward.act_fn as default
Injecting model.layers.19.attention_norm as default
Injecting model.layers.19.ffn_norm as default
Injecting model.layers.20 as default
Injecting model.layers.20.attention as default
Injecting model.layers.20.attention.wqkv as default
Injecting model.layers.20.attention.wo as default
Injecting model.layers.20.attention.rotary_emb as default
Injecting model.layers.20.feed_forward as default
Injecting model.layers.20.feed_forward.w1 as default
Injecting model.layers.20.feed_forward.w3 as default
Injecting model.layers.20.feed_forward.w2 as default
Injecting model.layers.20.feed_forward.act_fn as default
Injecting model.layers.20.attention_norm as default
Injecting model.layers.20.ffn_norm as default
Injecting model.layers.21 as default
Injecting model.layers.21.attention as default
Injecting model.layers.21.attention.wqkv as default
Injecting model.layers.21.attention.wo as default
Injecting model.layers.21.attention.rotary_emb as default
Injecting model.layers.21.feed_forward as default
Injecting model.layers.21.feed_forward.w1 as default
Injecting model.layers.21.feed_forward.w3 as default
Injecting model.layers.21.feed_forward.w2 as default
Injecting model.layers.21.feed_forward.act_fn as default
Injecting model.layers.21.attention_norm as default
Injecting model.layers.21.ffn_norm as default
Injecting model.layers.22 as default
Injecting model.layers.22.attention as default
Injecting model.layers.22.attention.wqkv as default
Injecting model.layers.22.attention.wo as default
Injecting model.layers.22.attention.rotary_emb as default
Injecting model.layers.22.feed_forward as default
Injecting model.layers.22.feed_forward.w1 as default
Injecting model.layers.22.feed_forward.w3 as default
Injecting model.layers.22.feed_forward.w2 as default
Injecting model.layers.22.feed_forward.act_fn as default
Injecting model.layers.22.attention_norm as default
Injecting model.layers.22.ffn_norm as default
Injecting model.layers.23 as default
Injecting model.layers.23.attention as default
Injecting model.layers.23.attention.wqkv as default
Injecting model.layers.23.attention.wo as default
Injecting model.layers.23.attention.rotary_emb as default
Injecting model.layers.23.feed_forward as default
Injecting model.layers.23.feed_forward.w1 as default
Injecting model.layers.23.feed_forward.w3 as default
Injecting model.layers.23.feed_forward.w2 as default
Injecting model.layers.23.feed_forward.act_fn as default
Injecting model.layers.23.attention_norm as default
Injecting model.layers.23.ffn_norm as default
Injecting model.layers.24 as default
Injecting model.layers.24.attention as default
Injecting model.layers.24.attention.wqkv as default
Injecting model.layers.24.attention.wo as default
Injecting model.layers.24.attention.rotary_emb as default
Injecting model.layers.24.feed_forward as default
Injecting model.layers.24.feed_forward.w1 as default
Injecting model.layers.24.feed_forward.w3 as default
Injecting model.layers.24.feed_forward.w2 as default
Injecting model.layers.24.feed_forward.act_fn as default
Injecting model.layers.24.attention_norm as default
Injecting model.layers.24.ffn_norm as default
Injecting model.layers.25 as default
Injecting model.layers.25.attention as default
Injecting model.layers.25.attention.wqkv as default
Injecting model.layers.25.attention.wo as default
Injecting model.layers.25.attention.rotary_emb as default
Injecting model.layers.25.feed_forward as default
Injecting model.layers.25.feed_forward.w1 as default
Injecting model.layers.25.feed_forward.w3 as default
Injecting model.layers.25.feed_forward.w2 as default
Injecting model.layers.25.feed_forward.act_fn as default
Injecting model.layers.25.attention_norm as default
Injecting model.layers.25.ffn_norm as default
Injecting model.layers.26 as default
Injecting model.layers.26.attention as default
Injecting model.layers.26.attention.wqkv as default
Injecting model.layers.26.attention.wo as default
Injecting model.layers.26.attention.rotary_emb as default
Injecting model.layers.26.feed_forward as default
Injecting model.layers.26.feed_forward.w1 as default
Injecting model.layers.26.feed_forward.w3 as default
Injecting model.layers.26.feed_forward.w2 as default
Injecting model.layers.26.feed_forward.act_fn as default
Injecting model.layers.26.attention_norm as default
Injecting model.layers.26.ffn_norm as default
Injecting model.layers.27 as default
Injecting model.layers.27.attention as default
Injecting model.layers.27.attention.wqkv as default
Injecting model.layers.27.attention.wo as default
Injecting model.layers.27.attention.rotary_emb as default
Injecting model.layers.27.feed_forward as default
Injecting model.layers.27.feed_forward.w1 as default
Injecting model.layers.27.feed_forward.w3 as default
Injecting model.layers.27.feed_forward.w2 as default
Injecting model.layers.27.feed_forward.act_fn as default
Injecting model.layers.27.attention_norm as default
Injecting model.layers.27.ffn_norm as default
Injecting model.layers.28 as default
Injecting model.layers.28.attention as default
Injecting model.layers.28.attention.wqkv as default
Injecting model.layers.28.attention.wo as default
Injecting model.layers.28.attention.rotary_emb as default
Injecting model.layers.28.feed_forward as default
Injecting model.layers.28.feed_forward.w1 as default
Injecting model.layers.28.feed_forward.w3 as default
Injecting model.layers.28.feed_forward.w2 as default
Injecting model.layers.28.feed_forward.act_fn as default
Injecting model.layers.28.attention_norm as default
Injecting model.layers.28.ffn_norm as default
Injecting model.layers.29 as default
Injecting model.layers.29.attention as default
Injecting model.layers.29.attention.wqkv as default
Injecting model.layers.29.attention.wo as default
Injecting model.layers.29.attention.rotary_emb as default
Injecting model.layers.29.feed_forward as default
Injecting model.layers.29.feed_forward.w1 as default
Injecting model.layers.29.feed_forward.w3 as default
Injecting model.layers.29.feed_forward.w2 as default
Injecting model.layers.29.feed_forward.act_fn as default
Injecting model.layers.29.attention_norm as default
Injecting model.layers.29.ffn_norm as default
Injecting model.layers.30 as default
Injecting model.layers.30.attention as default
Injecting model.layers.30.attention.wqkv as default
Injecting model.layers.30.attention.wo as default
Injecting model.layers.30.attention.rotary_emb as default
Injecting model.layers.30.feed_forward as default
Injecting model.layers.30.feed_forward.w1 as default
Injecting model.layers.30.feed_forward.w3 as default
Injecting model.layers.30.feed_forward.w2 as default
Injecting model.layers.30.feed_forward.act_fn as default
Injecting model.layers.30.attention_norm as default
Injecting model.layers.30.ffn_norm as default
Injecting model.layers.31 as default
Injecting model.layers.31.attention as default
Injecting model.layers.31.attention.wqkv as default
Injecting model.layers.31.attention.wo as default
Injecting model.layers.31.attention.rotary_emb as default
Injecting model.layers.31.feed_forward as default
Injecting model.layers.31.feed_forward.w1 as default
Injecting model.layers.31.feed_forward.w3 as default
Injecting model.layers.31.feed_forward.w2 as default
Injecting model.layers.31.feed_forward.act_fn as default
Injecting model.layers.31.attention_norm as default
Injecting model.layers.31.ffn_norm as default
Injecting model.norm as default
Injecting output as default
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "E:\AI_Chat_ktransformers\ktransformers\local_chat.py", line 179, in
fire.Fire(local_chat)
File "C:\Users\wuwenthink.conda\envs\ktransformers\Lib\site-packages\fire\core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\wuwenthink.conda\envs\ktransformers\Lib\site-packages\fire\core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\wuwenthink.conda\envs\ktransformers\Lib\site-packages\fire\core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "E:\AI_Chat_ktransformers\ktransformers\local_chat.py", line 110, in local_chat
optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
File "E:\AI_Chat_ktransformers\ktransformers\optimize\optimize.py", line 129, in optimize_and_load_gguf
load_weights(module, gguf_loader)
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 83, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 83, in load_weights
load_weights(child, gguf_loader, prefix+name+".")
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 81, in load_weights
load_cur_state_dict(module, gguf_loader, prefix)
File "E:\AI_Chat_ktransformers\ktransformers\util\utils.py", line 76, in load_cur_state_dict
raise Exception(f"can't find {translated_key} in GGUF file!")
Exception: can't find model.tok_embeddings.weight in GGUF file!
@haojiubujian1985 commented on GitHub (Feb 18, 2025):
The same problem
@ayttop commented on GitHub (Jun 15, 2025):
same problem!!!!!!!!!
@wuwenthink commented on GitHub (Jun 15, 2025):
这是来自QQ邮箱的自动回复邮件。
您好，我可能无法马上回复您的邮件。但会尽快给您回复。