mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #346] run DeepSeek-R1 throw KeyError: 'blk.0.attn_q_a.weight' #224
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#224
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @yangdaolin on GitHub (Feb 16, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/346
python local_chat.py --model_path=/opt/deepseek/DeepSeek-R1 --gguf_path=/opt/deepseek/DeepSeek-R1 --optimize_rule_path=./optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu.yaml
daolin@server238:~/ai/ktransformers/ktransformers$ sh run.sh
using custom modeling_xxx.py.
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.0.self_attn.q_a_proj as ktransformers.operators.linear . KTransformersLinear
Traceback (most recent call last):
File "/opt/ai/ktransformers/ktransformers/local_chat.py", line 179, in
fire.Fire(local_chat)
File "/home/daolin/.local/lib/python3.12/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/daolin/.local/lib/python3.12/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/home/daolin/.local/lib/python3.12/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/ai/ktransformers/ktransformers/local_chat.py", line 110, in local_chat
optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
File "/opt/ai/ktransformers/ktransformers/optimize/optimize.py", line 128, in optimize_and_load_gguf
inject(module, optimize_config, model_config, gguf_loader)
File "/opt/ai/ktransformers/ktransformers/optimize/optimize.py", line 42, in inject
inject(child, child_optimization_dict, model_config, gguf_loader, child_prefix)
File "/opt/ai/ktransformers/ktransformers/optimize/optimize.py", line 42, in inject
inject(child, child_optimization_dict, model_config, gguf_loader, child_prefix)
File "/opt/ai/ktransformers/ktransformers/optimize/optimize.py", line 42, in inject
inject(child, child_optimization_dict, model_config, gguf_loader, child_prefix)
[Previous line repeated 1 more time]
File "/opt/ai/ktransformers/ktransformers/optimize/optimize.py", line 33, in inject
inject_module=module_cls(key = inject_module_meta["key"], gguf_loader = gguf_loader, config = model_config, orig_module=child, **inject_module_meta["kwargs"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/ai/ktransformers/ktransformers/operators/linear.py", line 378, in init
KLinearBase.init(self, key, gguf_loader, config, orig_module, generate_device, **kwargs)
File "/opt/ai/ktransformers/ktransformers/operators/linear.py", line 61, in init
shape = self.gguf_loader.tensor_info[key + ".weight"]["shape"]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'blk.0.attn_q_a.weight'
@Azure-Tang commented on GitHub (Feb 16, 2025):
@Endiexxx commented on GitHub (Feb 16, 2025):
no work，i am the isue.redownload, and check it on Km studio work well
@yinghaodang commented on GitHub (Feb 17, 2025):
I have the same issue，model run on T4，flash_attn maybe not support.
@ErvinXie commented on GitHub (Dec 11, 2025):
This issue has been automatically closed due to inactivity for 60 days.
If you believe this issue is still relevant, please feel free to reopen it with additional information or context.
Thank you for your contribution to KTransformers!
此 issue 因超过 60 天无活动而被自动关闭。
如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。
感谢您对 KTransformers 的贡献！