mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1363] [Bug] KeyError: 'blk.24.ffn_up_exps.weight' #914
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#914
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @344303947 on GitHub (Jun 4, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1363
Originally assigned to: @Azure-Tang on GitHub.
检查清单
问题描述
loading model.layers.24.self_attn.kv_b_proj.weight to cuda:0
Process SpawnProcess-1:
Traceback (most recent call last):
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 277, in run_engine
engine = Engine(args, token_queue, broadcast_endpoint, kvcache_event)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 181, in init
optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/optimize/optimize.py", line 131, in optimize_and_load_gguf
load_weights(module, weights_loader, device=default_device)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 173, in load_weights
load_weights(child, gguf_loader, prefix+name+".", device=device)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 175, in load_weights
module.load()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 63, in load
utils.load_weights(child, self.gguf_loader, self.key+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 173, in load_weights
load_weights(child, gguf_loader, prefix+name+".", device=device)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 173, in load_weights
load_weights(child, gguf_loader, prefix+name+".", device=device)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 173, in load_weights
load_weights(child, gguf_loader, prefix+name+".", device=device)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 175, in load_weights
module.load()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 63, in load
utils.load_weights(child, self.gguf_loader, self.key+".")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 173, in load_weights
load_weights(child, gguf_loader, prefix+name+".", device=device)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/utils.py", line 175, in load_weights
module.load()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/experts.py", line 1268, in load
self.generate_experts.load(w, warmup=warmup)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/experts.py", line 163, in load
if w is None: w = self.load_weights()[self.key]
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/operators/experts.py", line 365, in load_weights
up = self.gguf_loader.get_mmap_tensor(key + ".ffn_up_exps.weight")
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/util/custom_loader.py", line 375, in get_mmap_tensor
t = self.tensor_info[name]
KeyError: 'blk.24.ffn_up_exps.weight'
^CTraceback (most recent call last):
File "/home/sean/ubuntu/ktransformers/ktransformers/server/main.py", line 122, in
main()
File "/home/sean/ubuntu/ktransformers/ktransformers/server/main.py", line 109, in main
create_interface(config=cfg, default_args=cfg)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface
GlobalInterface.interface = BackendInterface(default_args)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 319, in init
kvcache_event.wait()
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/synchronize.py", line 349, in wait
self._cond.wait(timeout)
File "/home/sean/miniconda3/envs/kt/lib/python3.10/multiprocessing/synchronize.py", line 261, in wait
return self._wait_semaphore.acquire(True, timeout)
KeyboardInterrupt
复现步骤
python3.10 ktransformers/server/main.py
--model_path "/media/mydisk/models/DeepSeek-R1-0528-config"
--gguf_path "/media/mydisk/models/unsloth/DeepSeek-R1-0528-GGUF/UD-Q4_K_XL"
--model_name "unsloth/DeepSeek-R1-671b"
--cpu_infer 28
--max_new_tokens 32000
--cache_lens 32000
--temperature 0.6
--top_p 0.95
--optimize_config_path "ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml"
--port 6699
--chunk_size 256
--max_batch_size 4
--backend_type balance_serve
--use_cuda_graph
--force_think
环境信息
v0.3.1
UD-Q4_K_XL报错
Q2没有问题
@Azure-Tang commented on GitHub (Jun 10, 2025):
检查一下sha256sum看一下是不是模型权重损坏？