mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #734] AttributeError: 'Linear' object has no attribute 'weight' #536
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#536
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @3keyallen3 on GitHub (Feb 28, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/734
烦请各位大佬帮忙看下，使用docker运行Ktransformers单卡可以运行，一旦加入多卡配置，加载后对话就报错。
approachingai/ktransformers:0.2.1
python -m ktransformers.local_chat --gguf_path /models/GGUF --model_path /models --cpu_infer 96 --port 10002 --web True --optimize_rule_path /models/DeepSeek-V3-Chat-multi-gpu-8.yaml --max_new_tokens 5000 --force_think ture
https://github.com/kvcache-ai/ktransformers/blob/main/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu-8.yaml
模型为DeepSeek-R1-Q8_0
CPU信息
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 52 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 384
On-line CPU(s) list: 0-383
Vendor ID: AuthenticAMD
Model name: AMD EPYC 9654 96-Core Processor
CPU family: 25
Model: 17
Thread(s) per core: 2
Core(s) per socket: 96
Socket(s): 2
Stepping: 1
Frequency boost: enabled
CPU(s) scaling MHz: 66%
CPU max MHz: 3707.8120
CPU min MHz: 1500.0000
BogoMIPS: 4792.89
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rap
l pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core p
erfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt
clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin cppc arat npt lbrv svm_lock nrip_save tsc_scale
vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid ov
erflow_recov succor smca fsrm flush_l1d debug_swap
Virtualization features:
Virtualization: AMD-V
Caches (sum of all):
L1d: 6 MiB (192 instances)
L1i: 6 MiB (192 instances)
L2: 192 MiB (192 instances)
L3: 768 MiB (24 instances)
NUMA:
NUMA node(s): 2
NUMA node0 CPU(s): 0-95,192-287
NUMA node1 CPU(s): 96-191,288-383
GPU信息：
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03 Driver Version: 560.35.03 CUDA Version: 12.6 |NVIDIA A100-SXM4-40GB*8
报错信息：
Chat: hello
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 39, in getattr
return object.getattribute(self, name) # if this attr belongs to BaseInjectedModule
AttributeError: 'KTransformersLinear' object has no attribute 'weight'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 44, in getattr
return nn.Module.getattr(self, "orig_module").getattr(name) # if this attr belongs to orig_module
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1709, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'Linear' object has no attribute 'weight'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.10/site-packages/ktransformers/local_chat.py", line 179, in
fire.Fire(local_chat)
File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/opt/conda/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/ktransformers/local_chat.py", line 173, in local_chat
generated = prefill_and_generate(
File "/opt/conda/lib/python3.10/site-packages/ktransformers/util/utils.py", line 154, in prefill_and_generate
logits = model(
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/ktransformers/models/modeling_deepseek_v3.py", line 1702, in forward
logits = self.lm_head(hidden_states.to(self.lm_head.weight.device))
File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 46, in getattr
return super(nn.Module, nn.Module.getattr(self, "orig_module")).getattribute(name) # if this attr belongs to orig_module but not in nn.Module.dict
AttributeError: 'Linear' object has no attribute 'weight'
File "/opt/conda/lib/python3.10/site-packages/ktransformers/models/modeling_deepseek_v3.py", line 1702, in forward
logits = self.lm_head(hidden_states.to(self.lm_head.weight.device))
File "/opt/conda/lib/python3.10/site-packages/ktransformers/operators/base_operator.py", line 46, in getattr
return super(nn.Module, nn.Module.getattr(self, "orig_module")).getattribute(name) # if this attr belongs to orig_module but not in nn.Module.dict
AttributeError: 'Linear' object has no attribute 'weight'
@ErvinXie commented on GitHub (Dec 11, 2025):
This issue has been automatically closed due to inactivity for 60 days.
If you believe this issue is still relevant, please feel free to reopen it with additional information or context.
Thank you for your contribution to KTransformers!
此 issue 因超过 60 天无活动而被自动关闭。
如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。
感谢您对 KTransformers 的贡献！