kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #495] ktransformers V0.3报错Illegal instruction (core dumped) #352

New issue

Closed

opened 2026-10-29 14:38:21 +00:00 by bsakaguchi · 8 comments

bsakaguchi commented

2026-10-29 14:38:21 +00:00

Owner

Originally created by @GodXuxilie on GitHub (Feb 19, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/495

在配置完ktransformers 0.3后，

pip install ~/ktransformers-0.3.0rc0+cu126torch26fancy-cp311-cp311-linux_x86_64.whl

运行如下命令后，

python -m ktransformers.local_chat --model_path ./DeepSeek-R1 --gguf_path ./DeepSeek-R1-GGUF  --cpu_infer 65 --max_new_tokens 1000 --force_think true

出现以下报错

using custom modeling_xxx.py.
using default_optimize_rule for DeepseekV3ForCausalLM
__AVX512F__
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Illegal instruction (core dumped)

之前有成功使用上ktransformers 0.2。不知道为什么用不了ktransformers 0.3？硬件1*A100, AMD EPYC 9654 (96 cpu cores).
有大神能帮下忙吗？

Originally created by @GodXuxilie on GitHub (Feb 19, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/495 在配置完ktransformers 0.3后， ``` pip install ~/ktransformers-0.3.0rc0+cu126torch26fancy-cp311-cp311-linux_x86_64.whl ``` 运行如下命令后， ``` python -m ktransformers.local_chat --model_path ./DeepSeek-R1 --gguf_path ./DeepSeek-R1-GGUF --cpu_infer 65 --max_new_tokens 1000 --force_think true ``` 出现以下报错 ``` using custom modeling_xxx.py. using default_optimize_rule for DeepseekV3ForCausalLM __AVX512F__ Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Illegal instruction (core dumped) ``` 之前有成功使用上ktransformers 0.2。不知道为什么用不了ktransformers 0.3？硬件1*A100, AMD EPYC 9654 (96 cpu cores). 有大神能帮下忙吗？

bsakaguchi

2026-10-29 14:38:21 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:38:22 +00:00

Author

Owner

@minhe1 commented on GitHub (Feb 19, 2025):

0.3看说明是用了AMX指令，估计必须用新的Intel CPU才行，我没有试过，瞎猜的。

@minhe1 commented on GitHub (Feb 19, 2025): 0.3看说明是用了AMX指令，估计必须用新的Intel CPU才行，我没有试过，瞎猜的。

bsakaguchi commented

2026-10-29 14:38:22 +00:00

Author

Owner

@loverara commented on GitHub (Feb 19, 2025):

我也这样，但是我用的cpu是Intel Xeon Gold 6240，有人知道如何解决吗？

@loverara commented on GitHub (Feb 19, 2025): 我也这样，但是我用的cpu是Intel Xeon Gold 6240，有人知道如何解决吗？

bsakaguchi commented

2026-10-29 14:38:22 +00:00

Author

Owner

@yansiyu550 commented on GitHub (Feb 19, 2025):

CPU需要支持AMX，支持AMX出现Illegal instruction (core dumped)问题可以试试这个：#320

gcc -shared -fPIC -o preload_amx.so preload_amx.c
LD_PRELOAD=./preload_amx.so python -m ktransformers.local_chat --model_path $MODEL_PATH --gguf_path $GGUF_PATH --prompt_file prompt.file --cpu_infer 178 --max_new_tokens 1000

//preload_amx.c
#include <stdio.h>
#include <sys/syscall.h>
#include <unistd.h>

#define ARCH_REQ_XCOMP_PERM     0x1023
#define XFEATURE_XTILEDATA      18

__attribute__((constructor)) void init() {
 if (syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_PERM, XFEATURE_XTILEDATA))
 {
    printf("\n Fail to do XFEATURE_XTILEDATA \n\n");
 }
 else
 {
    printf("\n TILE DATA USE SET - OK \n\n");
 }
}

@yansiyu550 commented on GitHub (Feb 19, 2025): CPU需要支持AMX，支持AMX出现Illegal instruction (core dumped)问题可以试试这个：[#320](https://github.com/kvcache-ai/ktransformers/issues/320) ``` gcc -shared -fPIC -o preload_amx.so preload_amx.c LD_PRELOAD=./preload_amx.so python -m ktransformers.local_chat --model_path $MODEL_PATH --gguf_path $GGUF_PATH --prompt_file prompt.file --cpu_infer 178 --max_new_tokens 1000 ``` ``` //preload_amx.c #include <stdio.h> #include <sys/syscall.h> #include <unistd.h> #define ARCH_REQ_XCOMP_PERM 0x1023 #define XFEATURE_XTILEDATA 18 __attribute__((constructor)) void init() { if (syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_PERM, XFEATURE_XTILEDATA)) { printf("\n Fail to do XFEATURE_XTILEDATA \n\n"); } else { printf("\n TILE DATA USE SET - OK \n\n"); } } ```

bsakaguchi commented

2026-10-29 14:38:23 +00:00

Author

Owner

@txg1550759 commented on GitHub (Feb 22, 2025):

CPU需要支持AMX，支持AMX出现Illegal instruction (core dumped)问题可以试试这个：#320

gcc -shared -fPIC -o preload_amx.so preload_amx.c
LD_PRELOAD=./preload_amx.so python -m ktransformers.local_chat --model_path $MODEL_PATH --gguf_path $GGUF_PATH --prompt_file prompt.file --cpu_infer 178 --max_new_tokens 1000

//preload_amx.c
#include <stdio.h>
#include <sys/syscall.h>
#include <unistd.h>

#define ARCH_REQ_XCOMP_PERM     0x1023
#define XFEATURE_XTILEDATA      18

__attribute__((constructor)) void init() {
 if (syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_PERM, XFEATURE_XTILEDATA))
 {
    printf("\n Fail to do XFEATURE_XTILEDATA \n\n");
 }
 else
 {
    printf("\n TILE DATA USE SET - OK \n\n");
 }
}

我跑0.3 cpu不支持 amx 怎么弄？
能不能用这种方法，改造下可用？

@txg1550759 commented on GitHub (Feb 22, 2025): > CPU需要支持AMX，支持AMX出现Illegal instruction (core dumped)问题可以试试这个：[#320](https://github.com/kvcache-ai/ktransformers/issues/320) > > ``` > gcc -shared -fPIC -o preload_amx.so preload_amx.c > LD_PRELOAD=./preload_amx.so python -m ktransformers.local_chat --model_path $MODEL_PATH --gguf_path $GGUF_PATH --prompt_file prompt.file --cpu_infer 178 --max_new_tokens 1000 > ``` > > ``` > //preload_amx.c > #include <stdio.h> > #include <sys/syscall.h> > #include <unistd.h> > > #define ARCH_REQ_XCOMP_PERM 0x1023 > #define XFEATURE_XTILEDATA 18 > > __attribute__((constructor)) void init() { > if (syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_PERM, XFEATURE_XTILEDATA)) > { > printf("\n Fail to do XFEATURE_XTILEDATA \n\n"); > } > else > { > printf("\n TILE DATA USE SET - OK \n\n"); > } > } > ``` 我跑0.3 cpu不支持 amx 怎么弄？能不能用这种方法，改造下可用？

bsakaguchi commented

2026-10-29 14:38:23 +00:00

Author

Owner

@txg1550759 commented on GitHub (Feb 22, 2025):

cpu Intel(R) Xeon(R) Gold 5320 CPU
a800 80g
跑0.21本正常，跑0.3版本，也报这个，有人知道如何解决吗：
flashinfer not found, use triton for linux
using custom modeling_xxx.py.
AVX512F
Injecting model as ktransformers.operators.models . KDeepseekV2Model
Injecting model.embed_tokens as default
Injecting model.layers as default
Injecting model.layers.0 as default
Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention
Illegal instruction (core dumped)

@txg1550759 commented on GitHub (Feb 22, 2025): cpu Intel(R) Xeon(R) Gold 5320 CPU a800 80g 跑0.21本正常，跑0.3版本，也报这个，有人知道如何解决吗： flashinfer not found, use triton for linux using custom modeling_xxx.py. __AVX512F__ Injecting model as ktransformers.operators.models . KDeepseekV2Model Injecting model.embed_tokens as default Injecting model.layers as default Injecting model.layers.0 as default Injecting model.layers.0.self_attn as ktransformers.operators.attention . KDeepseekV2Attention Illegal instruction (core dumped)

bsakaguchi commented

2026-10-29 14:38:23 +00:00

Author

Owner

@ArSrNa commented on GitHub (Feb 23, 2025):

先使用cat /proc/cpuinfo，查看支持的指令集是avx2还是avx512，到release下载对应的安装包，比如avx512，python3.10，torch在2.4以上的就下载[ktransformers-0.2.1.post1+cu121torch24avx512-cp310-cp310-linux_x86_64.whl](https://github.com/kvcache-ai/ktransformers/releases/download/v0.2.1.post1/ktransformers-0.2.1.post1+cu121torch24avx512-cp310-cp310-linux_x86_64.whl)

@ArSrNa commented on GitHub (Feb 23, 2025): 先使用`cat /proc/cpuinfo`，查看支持的指令集是avx2还是avx512，到[release](https://github.com/kvcache-ai/ktransformers/releases/)下载对应的安装包，比如avx512，python3.10，torch在2.4以上的就下载`[ktransformers-0.2.1.post1+cu121torch24avx512-cp310-cp310-linux_x86_64.whl](https://github.com/kvcache-ai/ktransformers/releases/download/v0.2.1.post1/ktransformers-0.2.1.post1+cu121torch24avx512-cp310-cp310-linux_x86_64.whl)` ![Image](https://github.com/user-attachments/assets/ad922c69-cb12-4e9a-88d3-ea9d255317b5)

bsakaguchi commented

2026-10-29 14:38:23 +00:00

Author

Owner

@loverara commented on GitHub (Feb 23, 2025):

cpu Intel(R) Xeon(R) Gold 5320 CPU

0.3需要支持AMX你的cpu不支持哦

@loverara commented on GitHub (Feb 23, 2025): > cpu Intel(R) Xeon(R) Gold 5320 CPU 0.3需要支持AMX你的cpu不支持哦 <img width="946" alt="Image" src="https://github.com/user-attachments/assets/4967d5dd-a111-41ae-ae18-70780a6a2241" />

bsakaguchi commented

2026-10-29 14:38:23 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！