mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1236] [Bug] KeyError: 'Key output.weight not found in Safetensor files' #839
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#839
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @344303947 on GitHub (Apr 29, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1236
检查清单
问题描述
Injecting model.layers.93.post_attention_layernorm as ktransformers.operators.layernorm . KQwen3MoeRMSNorm
Injecting model.norm as ktransformers.operators.layernorm . KQwen3MoeRMSNorm
Injecting model.rotary_emb as default
Injecting cache as default
Injecting lm_head as ktransformers.operators.linear . KTransformersLinear
Process SpawnProcess-1:
Traceback (most recent call last):
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 277, in run_engine
engine = Engine(args, token_queue, broadcast_endpoint, kvcache_event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 181, in init
optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config)
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 130, in optimize_and_load_gguf
load_weights(module.lm_head, gguf_loader, "lm_head.")
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/util/utils.py", line 127, in load_weights
module.load()
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/operators/linear.py", line 839, in load
self.generate_linear.load(w=w)
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/operators/linear.py", line 451, in load
w = self.load_weight(device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/operators/linear.py", line 88, in load_weight
tensor = self.gguf_loader.safetensor_loader.load_tensor(key+'.weight')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sean/miniconda3/envs/kt3/lib/python3.11/site-packages/ktransformers/util/custom_loader.py", line 61, in load_tensor
raise KeyError(f"Key {key} not found in Safetensor files")
KeyError: 'Key output.weight not found in Safetensor files'
复现步骤
python ktransformers/server/main.py --architectures Qwen3MoeForCausalLM
--port 6688
--model_path /media/mydisk/models/Qwen3-235B-A22B
--gguf_path /media/mydisk/models/Qwen3-235B-A22B
--model_name unsloth/Qwen3-235B-A22B
--optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml
--backend_type balance_serve
环境信息
v3.0最新版本主分支代码
@344303947 commented on GitHub (Apr 29, 2025):
模型没转成gguf格式不支持的问题。