mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1863] epyc 7402 CPU上推理Qwen3-235B-A22B出现illegal instruction #1159
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#1159
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @108851027 on GitHub (Feb 23, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1863
Reminder
System Info
KTransformers 环境诊断
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Check ┃ Status ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Python 版本 │ 正常 │ 3.12.12 │
│ CUDA 可用性 │ 正常 │ 12.6 │
│ GPU 检测 │ 正常 │ 发现 2 个 GPU: NVIDIA GeForce RTX 3090, NVIDIA GeForce RTX 3090 │
│ CPU │ 正常 │ AMD EPYC 7402 24-Core Processor (24 核心 / 48 线程) │
│ CPU 指令集 │ 警告 │ AVX2, AVX, FMA, SSE4.2, BMI1, BMI2, F16C, SSE (+3 more) │
│ NUMA 拓扑 │ 正常 │ 1 个节点 │
│ kt-kernel │ 警告 │ v0.5.1 (AVX2) │
│ 系统内存 │ 正常 │ 247.1GB 可用 / 251.6GB 总计 │
│ 磁盘空间 │ 正常 │ /mnt/chunk/models 有 4462.3GB 可用空间 │
│ SGLang Source │ 正常 │ Source (GitHub: kvcache-ai/sglang, branch: main) │
│ SGLang kt-kernel │ 正常 │ SGLang kt-kernel 支持已验证 │
│ Environment Managers │ 正常 │ conda 24.1.2, venv builtin │
└──────────────────────┴──────────┴─────────────────────────────────────────────────────────────────┘
✓ 所有检查通过！您的环境已就绪。
Reproduction
启动命令如下：
python -m sglang.launch_server --host 0.0.0.0 --port 8000 --model /mnt/chunk/models/Qwen3-235B-A22B --trust-remote-code --mem-fraction-static 0.92 --chunked-prefill-size 4096 --served-model-name Qwen3-235B-A22B --enable-mixed-chunk --kt-method LLAMAFILE --kt-weight-path /mnt/chunk/models/Qwen3-235B-A22B-GGUF/Q4_K_M/ --kt-cpuinfer 32 --kt-threadpool-count 1 --kt-num-gpu-experts 2 --kt-max-deferred-experts-per-token 2 --tp 2
报错如下：
Loading safetensors checkpoint shards: 100% Completed | 118/118 [00:08<00:00, 13.13it/s]
Loading safetensors checkpoint shards: 100% Completed | 118/118 [00:08<00:00, 14.11it/s]
Fatal Python error: Illegal instruction
Thread 0x00007ed20dfff640 (most recent call first):
File "/root/miniconda3/envs/ktransformers/lib/python3.12/threading.py", line 359 in wait
File "/root/miniconda3/envs/ktransformers/lib/python3.12/threading.py", line 655 in wait
File "/root/miniconda3/envs/ktransformers/lib/python3.12/site-packages/tqdm/_monitor.py", line 60 in run
File "/root/miniconda3/envs/ktransformers/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/root/miniconda3/envs/ktransformers/lib/python3.12/threading.py", line 1032 in _bootstrap
Current thread 0x00007f11f36cd440 (most recent call first):
File "/root/miniconda3/envs/ktransformers/lib/python3.12/site-packages/kt_kernel/utils/llamafile.py", line 190 in load_weights
File "/home/david/ktransformers/sglang/python/sglang/srt/layers/moe/kt_ep_wrapper.py", line 2019 in process_weights_after_loading
File "/home/david/ktransformers/sglang/python/sglang/srt/model_loader/loader.py", line 692 in load_weights_and_postprocess
File "/home/david/ktransformers/sglang/python/sglang/srt/model_loader/loader.py", line 668 in load_model
File "/home/david/ktransformers/sglang/python/sglang/srt/model_executor/model_runner.py", line 961 in load_model
File "/home/david/ktransformers/sglang/python/sglang/srt/model_executor/model_runner.py", line 483 in initialize
File "/home/david/ktransformers/sglang/python/sglang/srt/model_executor/model_runner.py", line 403 in init
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/tp_worker.py", line 325 in _init_model_runner
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/tp_worker.py", line 242 in init
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 503 in init_tp_model_worker
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 541 in init_model_worker
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 352 in init
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 3039 in run_scheduler_process
File "/root/miniconda3/envs/ktransformers/lib/python3.12/multiprocessing/process.py", line 108 in run
File "/root/miniconda3/envs/ktransformers/lib/python3.12/multiprocessing/process.py", line 314 in _bootstrap
File "/root/miniconda3/envs/ktransformers/lib/python3.12/multiprocessing/spawn.py", line 135 in _main
File "/root/miniconda3/envs/ktransformers/lib/python3.12/multiprocessing/spawn.py", line 122 in spawn_main
File "", line 1 in
Extension modules: numpy._core._multiarray_umath, numpy.linalg._umath_linalg, pybase64._pybase64, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, psutil._psutil_linux, zmq.backend.cython._zmq, PIL._imaging, sentencepiece._sentencepiece, yaml._yaml, regex._regex, markupsafe._speedups, cuda_utils, PIL._imagingft, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._pcg64, numpy.random._generator, numpy.random._mt19937, numpy.random._philox, numpy.random._sfc64, numpy.random.mtrand, _cffi_backend, _cyutility, scipy._cyutility, scipy._lib._ccallback_c, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._batched_linalg, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_schur_sqrtm, scipy.linalg._matfuncs_expm, scipy.linalg._linalg_pythran, scipy.linalg.cython_blas, scipy.linalg._decomp_update, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpacklib, scipy.sparse.linalg._propack, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._slsqplib, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy._lib._uarray._uarray, scipy.special._ufuncs_cxx, scipy.special._ellip_harm_2, scipy.special._special_ufuncs, scipy.special._gufuncs, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.linalg._decomp_interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rotation_cy, scipy.spatial.transform._rigid_transform_cy, scipy.optimize._direct, setproctitle._setproctitle, cuda.bindings._bindings.cydriver, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda.bindings.nvrtc, msgspec._core, cuda.cudart, cuda.nvrtc (total: 109)
[rank1]:[E223 19:12:49.114652203 ProcessGroupGloo.cpp:71] Rank 1 successfully reached monitoredBarrier, but received errors while waiting for send/recv from rank 0. Please check rank 0 logs for faulty rank.
[2026-02-23 19:12:49] Rank 0 scheduler is dead. Please check if there are relevant logs.
[2026-02-23 19:12:49 TP1] Scheduler hit an exception: Traceback (most recent call last):
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 3039, in run_scheduler_process
scheduler = Scheduler(
^^^^^^^^^^
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 352, in init
self.init_model_worker()
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 541, in init_model_worker
self.init_tp_model_worker()
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/scheduler.py", line 503, in init_tp_model_worker
self.tp_worker = TpModelWorker(
^^^^^^^^^^^^^^
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/tp_worker.py", line 242, in init
self._init_model_runner()
File "/home/david/ktransformers/sglang/python/sglang/srt/managers/tp_worker.py", line 325, in _init_model_runner
self._model_runner = ModelRunner(
^^^^^^^^^^^^
File "/home/david/ktransformers/sglang/python/sglang/srt/model_executor/model_runner.py", line 403, in init
self.initialize(min_per_gpu_memory)
File "/home/david/ktransformers/sglang/python/sglang/srt/model_executor/model_runner.py", line 483, in initialize
self.load_model()
File "/home/david/ktransformers/sglang/python/sglang/srt/model_executor/model_runner.py", line 1061, in load_model
raise ValueError(
ValueError: TP rank 1 could finish the model loading, but there are other ranks that didn't finish loading. It is likely due to unexpected failures (e.g., OOM) or a slow node.
[2026-02-23 19:12:49] Received sigquit from a child process. It usually means the child failed.
Others
No response
@ErvinXie commented on GitHub (Feb 28, 2026):
EPYC 7402 是 Zen 2 架构，仅支持 AVX2，不支持 AVX-512。KTransformers 的 LLAMAFILE 后端的部分 kernel 需要 AVX-512 指令集支持，在不支持的 CPU 上会触发
Illegal instruction。请确认：
lscpu | grep avx512确认 CPU 是否支持 AVX-512另外，我们的 kt-kernel 正在开发 AVX2 的原生支持，届时将可以在 Zen 2 等仅支持 AVX2 的 CPU 上运行。请关注后续版本更新。
This reply was generated by an AI agent and reviewed by a human.
@sonicz921 commented on GitHub (Mar 1, 2026):
AVX2 你可以手动编译
Maximum compatibility (works on any CPU from 2013+)
export CPUINFER_CPU_INSTRUCT=AVX2
export CPUINFER_ENABLE_AMX=OFF
./install.sh build --manual