[GH-ISSUE #709] 运行报错 KeyError: 'Key output.weight not found in Safetensor files' #517

New issue

Open

opened 2026-10-29 14:40:45 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:40:45 +00:00

Owner

Originally created by @ryys1122 on GitHub (Feb 27, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/709

运行

python -m ktransformers.local_chat --model_path ./DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat.Q4_K_M.gguf

报错

Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/local_chat.py", line 183, in <module>
    fire.Fire(local_chat)
  File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/local_chat.py", line 110, in local_chat
    optimize_and_load_gguf(model, optimize_config_path, gguf_path, config)
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf
    load_weights(module.lm_head, gguf_loader, "lm_head.")
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/util/utils.py", line 111, in load_weights
    module.load()
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/linear.py", line 522, in load
    self.generate_linear.load(w=w)
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/linear.py", line 279, in load
    w = self.load_weight(device=device) 
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/linear.py", line 84, in load_weight
    tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight')
  File "/opt/conda/lib/python3.10/site-packages/ktransformers/util/custom_loader.py", line 61, in load_tensor
    raise KeyError(f"Key {key} not found in Safetensor files")
KeyError: 'Key output.weight not found in Safetensor files'

Originally created by @ryys1122 on GitHub (Feb 27, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/709 运行 ```bash python -m ktransformers.local_chat --model_path ./DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat.Q4_K_M.gguf ``` 报错 ```bash Traceback (most recent call last): File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/opt/conda/lib/python3.10/site-packages/ktransformers/local_chat.py", line 183, in <module> fire.Fire(local_chat) File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/opt/conda/lib/python3.10/site-packages/ktransformers/local_chat.py", line 110, in local_chat optimize_and_load_gguf(model, optimize_config_path, gguf_path, config) File "/opt/conda/lib/python3.10/site-packages/ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf load_weights(module.lm_head, gguf_loader, "lm_head.") File "/opt/conda/lib/python3.10/site-packages/ktransformers/util/utils.py", line 111, in load_weights module.load() File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/linear.py", line 522, in load self.generate_linear.load(w=w) File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/linear.py", line 279, in load w = self.load_weight(device=device) File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/linear.py", line 84, in load_weight tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight') File "/opt/conda/lib/python3.10/site-packages/ktransformers/util/custom_loader.py", line 61, in load_tensor raise KeyError(f"Key {key} not found in Safetensor files") KeyError: 'Key output.weight not found in Safetensor files' ```

bsakaguchi commented

2026-10-29 14:40:46 +00:00

Author

Owner

@qiyuxinlin commented on GitHub (Feb 27, 2025):

gguf 文件 safetensor 文件分开存放，给路径的时候 gguf_path 指定文件夹，不能指定 gguf 文件

@qiyuxinlin commented on GitHub (Feb 27, 2025): gguf 文件 safetensor 文件分开存放，给路径的时候 gguf_path 指定文件夹，不能指定 gguf 文件

bsakaguchi commented

2026-10-29 14:40:46 +00:00

Author

Owner

@fishingfly commented on GitHub (Mar 31, 2025):

same problem

$ python3 -X faulthandler ./ktransformers/local_chat.py --force_think true --use_cuda_graph false --cpu_infer 32 --model_path Qwen/Qwen2-57B-A14B --gguf_path /models/Qwen2-57B-A14B/

Injecting model.layers.27.mlp.shared_expert as default
Injecting model.layers.27.mlp.shared_expert.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.27.mlp.shared_expert.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.27.mlp.shared_expert.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.27.mlp.shared_expert.act_fn as default
Injecting model.layers.27.mlp.shared_expert_gate as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.27.input_layernorm as default
Injecting model.layers.27.post_attention_layernorm as default
Injecting model.norm as default
Injecting lm_head as ktransformers.operators.linear . KTransformersLinear
Traceback (most recent call last):
  File "/ws/./ktransformers/local_chat.py", line 182, in <module>
    fire.Fire(local_chat)
  File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/ws/./ktransformers/local_chat.py", line 110, in local_chat
    optimize_and_load_gguf(model, optimize_config_path, gguf_path, config)
  File "/ws/./ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf
    load_weights(module.lm_head, gguf_loader, "lm_head.")
  File "/ws/./ktransformers/util/utils.py", line 110, in load_weights
    module.load()
  File "/ws/./ktransformers/operators/linear.py", line 691, in load
    self.generate_linear.load(w=w)
  File "/ws/./ktransformers/operators/linear.py", line 447, in load
    w = self.load_weight(device=device)
  File "/ws/./ktransformers/operators/linear.py", line 86, in load_weight
    tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight')
  File "/ws/./ktransformers/util/custom_loader.py", line 61, in load_tensor
    raise KeyError(f"Key {key} not found in Safetensor files")
KeyError: 'Key output.weight not found in Safetensor files'

...
...

@fishingfly commented on GitHub (Mar 31, 2025): same problem ``` $ python3 -X faulthandler ./ktransformers/local_chat.py --force_think true --use_cuda_graph false --cpu_infer 32 --model_path Qwen/Qwen2-57B-A14B --gguf_path /models/Qwen2-57B-A14B/ Injecting model.layers.27.mlp.shared_expert as default Injecting model.layers.27.mlp.shared_expert.gate_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.27.mlp.shared_expert.up_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.27.mlp.shared_expert.down_proj as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.27.mlp.shared_expert.act_fn as default Injecting model.layers.27.mlp.shared_expert_gate as ktransformers.operators.linear . KTransformersLinear Injecting model.layers.27.input_layernorm as default Injecting model.layers.27.post_attention_layernorm as default Injecting model.norm as default Injecting lm_head as ktransformers.operators.linear . KTransformersLinear Traceback (most recent call last): File "/ws/./ktransformers/local_chat.py", line 182, in <module> fire.Fire(local_chat) File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/usr/local/lib/python3.10/dist-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/ws/./ktransformers/local_chat.py", line 110, in local_chat optimize_and_load_gguf(model, optimize_config_path, gguf_path, config) File "/ws/./ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf load_weights(module.lm_head, gguf_loader, "lm_head.") File "/ws/./ktransformers/util/utils.py", line 110, in load_weights module.load() File "/ws/./ktransformers/operators/linear.py", line 691, in load self.generate_linear.load(w=w) File "/ws/./ktransformers/operators/linear.py", line 447, in load w = self.load_weight(device=device) File "/ws/./ktransformers/operators/linear.py", line 86, in load_weight tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight') File "/ws/./ktransformers/util/custom_loader.py", line 61, in load_tensor raise KeyError(f"Key {key} not found in Safetensor files") KeyError: 'Key output.weight not found in Safetensor files' ... ... ```