mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #356] Docker运行后出现 Illegal instruction (core dumped)报错 #232
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#232
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @LbuprofenL on GitHub (Feb 16, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/356
Docker pull 了最新的0.2.1版本镜像 运行一下代码后报 Illegal instruction (core dumped)错误
python -m ktransformers.local_chat --gguf_path /models/DeepSeek-R1-Q4_K_M/ --model_path /models --cpu_infer 33我的硬件支持AVX512指令集，还有什么原因会报这个错误呢？
@MasterJanus commented on GitHub (Feb 17, 2025):
Mark，same question
log:
using custom modeling_xxx.py.
using default_optimize_rule for DeepseekV2ForCausalLM
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.0.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.kv_a_layernorm as default
Injecting model.layers.0.self_attn.kv_b_proj as default
Injecting model.layers.0.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.0.mlp as default
Injecting model.layers.0.mlp.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.mlp.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.mlp.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.0.mlp.act_fn as default
Injecting model.layers.0.input_layernorm as default
Injecting model.layers.0.post_attention_layernorm as default
Injecting model.layers.1 as default
Injecting model.layers.1.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.1.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.self_attn.kv_a_layernorm as default
Injecting model.layers.1.self_attn.kv_b_proj as default
Injecting model.layers.1.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.1.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.1.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.1.mlp.gate as default
Injecting model.layers.1.mlp.shared_experts as default
Injecting model.layers.1.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.1.mlp.shared_experts.act_fn as default
Injecting model.layers.1.input_layernorm as default
Injecting model.layers.1.post_attention_layernorm as default
Injecting model.layers.2 as default
Injecting model.layers.2.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.2.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.2.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.2.self_attn.kv_a_layernorm as default
Injecting model.layers.2.self_attn.kv_b_proj as default
Injecting model.layers.2.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.2.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.2.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.2.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.2.mlp.gate as default
Injecting model.layers.2.mlp.shared_experts as default
Injecting model.layers.2.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.2.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.2.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.2.mlp.shared_experts.act_fn as default
Injecting model.layers.2.input_layernorm as default
Injecting model.layers.2.post_attention_layernorm as default
Injecting model.layers.3 as default
Injecting model.layers.3.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.3.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.3.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.3.self_attn.kv_a_layernorm as default
Injecting model.layers.3.self_attn.kv_b_proj as default
Injecting model.layers.3.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.3.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.3.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.3.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.3.mlp.gate as default
Injecting model.layers.3.mlp.shared_experts as default
Injecting model.layers.3.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.3.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.3.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.3.mlp.shared_experts.act_fn as default
Injecting model.layers.3.input_layernorm as default
Injecting model.layers.3.post_attention_layernorm as default
Injecting model.layers.4 as default
Injecting model.layers.4.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.4.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.4.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.4.self_attn.kv_a_layernorm as default
Injecting model.layers.4.self_attn.kv_b_proj as default
Injecting model.layers.4.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.4.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.4.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.4.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.4.mlp.gate as default
Injecting model.layers.4.mlp.shared_experts as default
Injecting model.layers.4.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.4.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.4.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.4.mlp.shared_experts.act_fn as default
Injecting model.layers.4.input_layernorm as default
Injecting model.layers.4.post_attention_layernorm as default
Injecting model.layers.5 as default
Injecting model.layers.5.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.5.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.5.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.5.self_attn.kv_a_layernorm as default
Injecting model.layers.5.self_attn.kv_b_proj as default
Injecting model.layers.5.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.5.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.5.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.5.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.5.mlp.gate as default
Injecting model.layers.5.mlp.shared_experts as default
Injecting model.layers.5.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.5.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.5.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.5.mlp.shared_experts.act_fn as default
Injecting model.layers.5.input_layernorm as default
Injecting model.layers.5.post_attention_layernorm as default
Injecting model.layers.6 as default
Injecting model.layers.6.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.6.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.6.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.6.self_attn.kv_a_layernorm as default
Injecting model.layers.6.self_attn.kv_b_proj as default
Injecting model.layers.6.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.6.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.6.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.6.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.6.mlp.gate as default
Injecting model.layers.6.mlp.shared_experts as default
Injecting model.layers.6.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.6.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.6.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.6.mlp.shared_experts.act_fn as default
Injecting model.layers.6.input_layernorm as default
Injecting model.layers.6.post_attention_layernorm as default
Injecting model.layers.7 as default
Injecting model.layers.7.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.7.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.7.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.7.self_attn.kv_a_layernorm as default
Injecting model.layers.7.self_attn.kv_b_proj as default
Injecting model.layers.7.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.7.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.7.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.7.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.7.mlp.gate as default
Injecting model.layers.7.mlp.shared_experts as default
Injecting model.layers.7.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.7.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.7.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.7.mlp.shared_experts.act_fn as default
Injecting model.layers.7.input_layernorm as default
Injecting model.layers.7.post_attention_layernorm as default
Injecting model.layers.8 as default
Injecting model.layers.8.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.8.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.8.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.8.self_attn.kv_a_layernorm as default
Injecting model.layers.8.self_attn.kv_b_proj as default
Injecting model.layers.8.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.8.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.8.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.8.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.8.mlp.gate as default
Injecting model.layers.8.mlp.shared_experts as default
Injecting model.layers.8.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.8.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.8.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.8.mlp.shared_experts.act_fn as default
Injecting model.layers.8.input_layernorm as default
Injecting model.layers.8.post_attention_layernorm as default
Injecting model.layers.9 as default
Injecting model.layers.9.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.9.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.9.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.9.self_attn.kv_a_layernorm as default
Injecting model.layers.9.self_attn.kv_b_proj as default
Injecting model.layers.9.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.9.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.9.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.9.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.9.mlp.gate as default
Injecting model.layers.9.mlp.shared_experts as default
Injecting model.layers.9.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.9.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.9.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.9.mlp.shared_experts.act_fn as default
Injecting model.layers.9.input_layernorm as default
Injecting model.layers.9.post_attention_layernorm as default
Injecting model.layers.10 as default
Injecting model.layers.10.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.10.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.10.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.10.self_attn.kv_a_layernorm as default
Injecting model.layers.10.self_attn.kv_b_proj as default
Injecting model.layers.10.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.10.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.10.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.10.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.10.mlp.gate as default
Injecting model.layers.10.mlp.shared_experts as default
Injecting model.layers.10.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.10.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.10.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.10.mlp.shared_experts.act_fn as default
Injecting model.layers.10.input_layernorm as default
Injecting model.layers.10.post_attention_layernorm as default
Injecting model.layers.11 as default
Injecting model.layers.11.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.11.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.11.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.11.self_attn.kv_a_layernorm as default
Injecting model.layers.11.self_attn.kv_b_proj as default
Injecting model.layers.11.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.11.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.11.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.11.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.11.mlp.gate as default
Injecting model.layers.11.mlp.shared_experts as default
Injecting model.layers.11.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.11.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.11.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.11.mlp.shared_experts.act_fn as default
Injecting model.layers.11.input_layernorm as default
Injecting model.layers.11.post_attention_layernorm as default
Injecting model.layers.12 as default
Injecting model.layers.12.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.12.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.12.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.12.self_attn.kv_a_layernorm as default
Injecting model.layers.12.self_attn.kv_b_proj as default
Injecting model.layers.12.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.12.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.12.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.12.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.12.mlp.gate as default
Injecting model.layers.12.mlp.shared_experts as default
Injecting model.layers.12.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.12.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.12.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.12.mlp.shared_experts.act_fn as default
Injecting model.layers.12.input_layernorm as default
Injecting model.layers.12.post_attention_layernorm as default
Injecting model.layers.13 as default
Injecting model.layers.13.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.13.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.13.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.13.self_attn.kv_a_layernorm as default
Injecting model.layers.13.self_attn.kv_b_proj as default
Injecting model.layers.13.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.13.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.13.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.13.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.13.mlp.gate as default
Injecting model.layers.13.mlp.shared_experts as default
Injecting model.layers.13.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.13.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.13.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.13.mlp.shared_experts.act_fn as default
Injecting model.layers.13.input_layernorm as default
Injecting model.layers.13.post_attention_layernorm as default
Injecting model.layers.14 as default
Injecting model.layers.14.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.14.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.14.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.14.self_attn.kv_a_layernorm as default
Injecting model.layers.14.self_attn.kv_b_proj as default
Injecting model.layers.14.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.14.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.14.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.14.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.14.mlp.gate as default
Injecting model.layers.14.mlp.shared_experts as default
Injecting model.layers.14.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.14.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.14.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.14.mlp.shared_experts.act_fn as default
Injecting model.layers.14.input_layernorm as default
Injecting model.layers.14.post_attention_layernorm as default
Injecting model.layers.15 as default
Injecting model.layers.15.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.15.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.15.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.15.self_attn.kv_a_layernorm as default
Injecting model.layers.15.self_attn.kv_b_proj as default
Injecting model.layers.15.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.15.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.15.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.15.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.15.mlp.gate as default
Injecting model.layers.15.mlp.shared_experts as default
Injecting model.layers.15.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.15.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.15.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.15.mlp.shared_experts.act_fn as default
Injecting model.layers.15.input_layernorm as default
Injecting model.layers.15.post_attention_layernorm as default
Injecting model.layers.16 as default
Injecting model.layers.16.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.16.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.16.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.16.self_attn.kv_a_layernorm as default
Injecting model.layers.16.self_attn.kv_b_proj as default
Injecting model.layers.16.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.16.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.16.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.16.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.16.mlp.gate as default
Injecting model.layers.16.mlp.shared_experts as default
Injecting model.layers.16.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.16.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.16.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.16.mlp.shared_experts.act_fn as default
Injecting model.layers.16.input_layernorm as default
Injecting model.layers.16.post_attention_layernorm as default
Injecting model.layers.17 as default
Injecting model.layers.17.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.17.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.17.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.17.self_attn.kv_a_layernorm as default
Injecting model.layers.17.self_attn.kv_b_proj as default
Injecting model.layers.17.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.17.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.17.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.17.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.17.mlp.gate as default
Injecting model.layers.17.mlp.shared_experts as default
Injecting model.layers.17.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.17.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.17.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.17.mlp.shared_experts.act_fn as default
Injecting model.layers.17.input_layernorm as default
Injecting model.layers.17.post_attention_layernorm as default
Injecting model.layers.18 as default
Injecting model.layers.18.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.18.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.18.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.18.self_attn.kv_a_layernorm as default
Injecting model.layers.18.self_attn.kv_b_proj as default
Injecting model.layers.18.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.18.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.18.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.18.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.18.mlp.gate as default
Injecting model.layers.18.mlp.shared_experts as default
Injecting model.layers.18.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.18.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.18.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.18.mlp.shared_experts.act_fn as default
Injecting model.layers.18.input_layernorm as default
Injecting model.layers.18.post_attention_layernorm as default
Injecting model.layers.19 as default
Injecting model.layers.19.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.19.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.19.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.19.self_attn.kv_a_layernorm as default
Injecting model.layers.19.self_attn.kv_b_proj as default
Injecting model.layers.19.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.19.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.19.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.19.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.19.mlp.gate as default
Injecting model.layers.19.mlp.shared_experts as default
Injecting model.layers.19.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.19.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.19.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.19.mlp.shared_experts.act_fn as default
Injecting model.layers.19.input_layernorm as default
Injecting model.layers.19.post_attention_layernorm as default
Injecting model.layers.20 as default
Injecting model.layers.20.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.20.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.20.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.20.self_attn.kv_a_layernorm as default
Injecting model.layers.20.self_attn.kv_b_proj as default
Injecting model.layers.20.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.20.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.20.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.20.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.20.mlp.gate as default
Injecting model.layers.20.mlp.shared_experts as default
Injecting model.layers.20.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.20.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.20.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.20.mlp.shared_experts.act_fn as default
Injecting model.layers.20.input_layernorm as default
Injecting model.layers.20.post_attention_layernorm as default
Injecting model.layers.21 as default
Injecting model.layers.21.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.21.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.21.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.21.self_attn.kv_a_layernorm as default
Injecting model.layers.21.self_attn.kv_b_proj as default
Injecting model.layers.21.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.21.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.21.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.21.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.21.mlp.gate as default
Injecting model.layers.21.mlp.shared_experts as default
Injecting model.layers.21.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.21.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.21.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.21.mlp.shared_experts.act_fn as default
Injecting model.layers.21.input_layernorm as default
Injecting model.layers.21.post_attention_layernorm as default
Injecting model.layers.22 as default
Injecting model.layers.22.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.22.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.22.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.22.self_attn.kv_a_layernorm as default
Injecting model.layers.22.self_attn.kv_b_proj as default
Injecting model.layers.22.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.22.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.22.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.22.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.22.mlp.gate as default
Injecting model.layers.22.mlp.shared_experts as default
Injecting model.layers.22.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.22.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.22.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.22.mlp.shared_experts.act_fn as default
Injecting model.layers.22.input_layernorm as default
Injecting model.layers.22.post_attention_layernorm as default
Injecting model.layers.23 as default
Injecting model.layers.23.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.23.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.23.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.23.self_attn.kv_a_layernorm as default
Injecting model.layers.23.self_attn.kv_b_proj as default
Injecting model.layers.23.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.23.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.23.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.23.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.23.mlp.gate as default
Injecting model.layers.23.mlp.shared_experts as default
Injecting model.layers.23.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.23.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.23.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.23.mlp.shared_experts.act_fn as default
Injecting model.layers.23.input_layernorm as default
Injecting model.layers.23.post_attention_layernorm as default
Injecting model.layers.24 as default
Injecting model.layers.24.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.24.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.24.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.24.self_attn.kv_a_layernorm as default
Injecting model.layers.24.self_attn.kv_b_proj as default
Injecting model.layers.24.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.24.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.24.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.24.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.24.mlp.gate as default
Injecting model.layers.24.mlp.shared_experts as default
Injecting model.layers.24.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.24.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.24.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.24.mlp.shared_experts.act_fn as default
Injecting model.layers.24.input_layernorm as default
Injecting model.layers.24.post_attention_layernorm as default
Injecting model.layers.25 as default
Injecting model.layers.25.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.25.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.25.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.25.self_attn.kv_a_layernorm as default
Injecting model.layers.25.self_attn.kv_b_proj as default
Injecting model.layers.25.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.25.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.25.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.25.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.25.mlp.gate as default
Injecting model.layers.25.mlp.shared_experts as default
Injecting model.layers.25.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.25.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.25.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.25.mlp.shared_experts.act_fn as default
Injecting model.layers.25.input_layernorm as default
Injecting model.layers.25.post_attention_layernorm as default
Injecting model.layers.26 as default
Injecting model.layers.26.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Injecting model.layers.26.self_attn.q_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.26.self_attn.kv_a_proj_with_mqa as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.26.self_attn.kv_a_layernorm as default
Injecting model.layers.26.self_attn.kv_b_proj as default
Injecting model.layers.26.self_attn.o_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.26.self_attn.rotary_emb as ktransformers.operators.RoPE . YarnRotaryEmbedding
Injecting model.layers.26.mlp as ktransformers.operators.experts . KDeepseekV2MoE
Injecting model.layers.26.mlp.experts as ktransformers.operators.experts . KTransformersExperts
Injecting model.layers.26.mlp.gate as default
Injecting model.layers.26.mlp.shared_experts as default
Injecting model.layers.26.mlp.shared_experts.gate_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.26.mlp.shared_experts.up_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.26.mlp.shared_experts.down_proj as ktransformers.operators.linear . KTransformersLinear
Injecting model.layers.26.mlp.shared_experts.act_fn as default
Injecting model.layers.26.input_layernorm as default
Injecting model.layers.26.post_attention_layernorm as default
Injecting model.norm as default
Injecting lm_head as default
loading token_embd.weight to cpu
/opt/conda/lib/python3.10/site-packages/ktransformers/util/custom_gguf.py:482: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1716905979055/work/torch/csrc/utils/tensor_numpy.cpp:206.)
data = torch.from_numpy(data)
loading blk.0.attn_kv_a_norm.weight to cuda:0
loading blk.0.attn_kv_b.weight to cuda:0
loading blk.0.attn_norm.weight to cuda:0
loading blk.0.ffn_norm.weight to cuda:0
loading blk.1.attn_kv_a_norm.weight to cuda:0
loading blk.1.attn_kv_b.weight to cuda:0
Illegal instruction (core dumped)
lscpu:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Vendor ID: GenuineIntel
Model name: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
CPU family: 6
Model: 106
Thread(s) per core: 2
Core(s) per socket: 32
Socket(s): 2
Stepping: 6
Frequency boost: enabled
CPU max MHz: 2601.0000
CPU min MHz: 800.0000
BogoMIPS: 5200.00
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mm
x fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon peb
s bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 ds_cpl vmx
smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_de
adline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 inv
pcid_single ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ep
t_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed
adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xs
avec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local wbnoinvd dtherm ida a
rat pln pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_
bitalg tme avx512_vpopcntdq rdpid md_clear pconfig flush_l1d arch_capabilities
Virtualization features:
Virtualization: VT-x
Caches (sum of all):
L1d: 3 MiB (64 instances)
L1i: 2 MiB (64 instances)
L2: 80 MiB (64 instances)
L3: 96 MiB (2 instances)
NUMA:
NUMA node(s): 2
NUMA node0 CPU(s): 0-31,64-95
NUMA node1 CPU(s): 32-63,96-127
Vulnerabilities:
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Srbds: Not affected
Tsx async abort: Not affected
@zhudongwork commented on GitHub (Feb 18, 2025):
一样的问题
@yansiyu550 commented on GitHub (Feb 18, 2025):
same problem，https://github.com/kvcache-ai/ktransformers/issues/320，It works for me.
@yileld commented on GitHub (Feb 18, 2025):
你也是只支持avx512但不支持amx的机器吗？
@LbuprofenL commented on GitHub (Feb 19, 2025):
yes
@tianwaifeidie commented on GitHub (Feb 20, 2025):
在自己机器的docker里面重新编译一下ks，bash install.sh
@zwbsxwt commented on GitHub (Feb 27, 2025):
解决了吗? docker中重新编译（bash install.sh） 以及 宿主机conda方式都试过了
跑Qwen2-57B-A14B-Instruct可以
跑DeepSeek-R1-Q4_K_M 就会Segmentation fault (core dumped)
目前重新下载模型文件，排除文件损坏的可能性.
硬件信息
CPU Intel Xeon Platinum 8336C * 2
RAM 1TB DDR4 Recc
Gpu RTX 4090 * 4
@zwbsxwt commented on GitHub (Feb 28, 2025):
问题解决, 我的是模型文件不完整导致的,计算sha256
重新下载不一样的文件就可以了
@ErvinXie commented on GitHub (Dec 11, 2025):
This issue has been automatically closed due to inactivity for 60 days.
If you believe this issue is still relevant, please feel free to reopen it with additional information or context.
Thank you for your contribution to KTransformers!
此 issue 因超过 60 天无活动而被自动关闭。
如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。
感谢您对 KTransformers 的贡献！