[GH-ISSUE #484] Docker run local chat raise error, please confirm whether AMD CPU can run ktransformers #342

New issue

Closed

opened 2026-10-29 14:38:09 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 14:38:09 +00:00

Owner

Originally created by @DongZhaoXiong on GitHub (Feb 19, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/484

reproduce step:
#1 docker pull approachingai/ktransformers:0.2.1
#2 docker run --gpus all -v /home/ec2-user/SageMaker/DeepSeek-R1-Q4_K_M:/models --name ktransformers -itd approachingai/ktransformers:0.2.1
#3 docker exec -it ktransformers /bin/bash
#4 python -m ktransformers.local_chat --gguf_path /models --model_path /models

root@dd814097d888:/workspace# python -m ktransformers.local_chat --gguf_path /models --model_path /models using custom modeling_xxx.py. using default_optimize_rule for DeepseekV3ForCausalLM Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Illegal instruction (core dumped)

environment:

AWS EC2 g5.12xlarge.
CPU: AMD EPYC 7R32
MEM： 192
GPU： 4*NVIDIA A10G

Originally created by @DongZhaoXiong on GitHub (Feb 19, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/484 reproduce step: #1 docker pull approachingai/ktransformers:0.2.1 #2 docker run --gpus all -v /home/ec2-user/SageMaker/DeepSeek-R1-Q4_K_M:/models --name ktransformers -itd approachingai/ktransformers:0.2.1 #3 docker exec -it ktransformers /bin/bash #4 python -m ktransformers.local_chat --gguf_path /models --model_path /models `root@dd814097d888:/workspace# python -m ktransformers.local_chat --gguf_path /models --model_path /models using custom modeling_xxx.py. using default_optimize_rule for DeepseekV3ForCausalLM Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Illegal instruction (core dumped)` environment: AWS EC2 g5.12xlarge. CPU: AMD EPYC 7R32 MEM： 192 GPU： 4*NVIDIA A10G

bsakaguchi

2026-10-29 14:38:09 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:38:09 +00:00

Author

Owner

@snailwei commented on GitHub (Feb 19, 2025):

这是CPU不支持AMX指令，有重复的问题了。可能要尝试本地编译。

@snailwei commented on GitHub (Feb 19, 2025): 这是CPU不支持AMX指令，有重复的问题了。可能要尝试本地编译。

bsakaguchi commented

2026-10-29 14:38:09 +00:00

Author

Owner

@txg1550759 commented on GitHub (Feb 20, 2025):

重新编译dockerfile打镜像，不要用官方的docker镜像，他的镜像带了amx指令，是至强第五代cpu才能用那个镜像

@txg1550759 commented on GitHub (Feb 20, 2025): 重新编译dockerfile打镜像，不要用官方的docker镜像，他的镜像带了amx指令，是至强第五代cpu才能用那个镜像

bsakaguchi commented

2026-10-29 14:38:10 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！