[GH-ISSUE #1690] 【ERROR】Install Ktransformers for AscendNPU #1081

New issue

Open

opened 2026-10-29 14:48:18 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:48:18 +00:00

Owner

Originally created by @Mr-Xiao2021 on GitHub (Dec 9, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1690

Originally assigned to: @RICHARDNAN on GitHub.

Reminder

I have read the above rules and searched the existing issues.

System Info

我参考如下链接进行安装：
ktransformers/doc/zh/DeepseekR1_V3_tutorial_zh_for_Ascend_NPU.md at main · kvcache-ai/ktransformers](https://github.com/kvcache-ai/ktransformers/blob/main/doc/zh/DeepseekR1_V3_tutorial_zh_for_Ascend_NPU.md#ktransformers部署)

我的系统信息是：

服务器型号：Atlas A2 A800T服务器
NPU： A2 A800T (910B3)
CPU: HUAWEI Kunpeng 920 5250
内存: DDR5服务器内存（1TB）

HDK: 24.1.rc3

CANN: CANN 8.3.RC1.alpha003，配套ToolKit，Kernel和NNAL均已安装好

OS: openEuler 22.03 LTS

操作系统内核为：5.10.0-60.18.0.50.oe2203.aarch64

GCC: 10.3.1

按照教程链接顺利安装好了conda环境和python部署，并安装好了指定版本torch_npu 2.5.1

Reproduction

将项目文件部署到机器上：

初始化third_party。由于此过程耗时较多，且容易受网络影响导致仓库克隆失败，建议初始化一次后，将相关文件进行打包，以便后续直接解压使用。

# 拉取最新版本
git clone https://github.com/kvcache-ai/ktransformers.git
cd archive
git submodule update --init --recursive

发现没有成功初始化，根据 archive/.gitmodules 文件内容，手动将这些子模块添加到主仓库中

# 在主项目根目录 (ktransformers/) 下执行以下所有命令

# 1. 添加 archive/ 下的第三方库
git submodule add https://github.com/ggerganov/llama.cpp.git archive/third_party/llama.cpp
git submodule add https://github.com/pybind/pybind11.git archive/third_party/pybind11
git submodule add https://github.com/gabime/spdlog.git archive/third_party/spdlog
git submodule add https://github.com/kvcache-ai/custom_flashinfer.git archive/third_party/custom_flashinfer
git submodule add https://github.com/Cyan4973/xxHash.git archive/third_party/xxHash
git submodule add https://github.com/jupp0r/prometheus-cpp archive/third_party/prometheus-cpp
git submodule add https://github.com/alibaba/PhotonLibOS.git archive/third_party/PhotonLibOS

# 2. 添加 kt-kernel/ 下的第三方库
git submodule add https://github.com/ggerganov/llama.cpp.git kt-kernel/third_party/llama.cpp
git submodule add https://github.com/pybind/pybind11.git kt-kernel/third_party/pybind11

# 切换分支
cd archive/third_party/custom_flashinfer
git checkout fix-precision-mla-merge-main
cd ../../..

# 初始化并更新所有子模块（包括嵌套的子模块）
git submodule update --init --recursive

对于arm平台，注释掉

./third_party/llamafile/iqk_mul_mat_arm82.cpp

中的

#define iqk_mul_mat iqk_mul_mat_arm82
#define iqk_mul_mat_moe iqk_mul_mat_moe_arm82

执行source /usr/local/Ascend/ascend-toolkit/set_env.sh（以实际CANN-TOOLKIT安装路径为准）。
执行apt install cmake libhwloc-dev pkg-config安装依赖。
修改项目目录下 /ktransformers/config/config.yaml 中attn部分的page_size: 128 chunk_size: 16384
执行USE_BALANCE_SERVE=1 USE_NUMA=1 bash ./install.sh，等待安装完成。

首次编译，发现手动添加的模块/ktransformers/archive/third_party/llama.cpp结构不符，

报错：

  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O3 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O3 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp
  In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul.inc:23,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp:8:
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory
     50 | #include "llama.cpp/ggml-impl.h"
        |          ^~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm.inc:23,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm.inc:50,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp:8:
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory
     50 | #include "llama.cpp/ggml-impl.h"
        |          ^~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:244: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o] Error 1
  gmake[2]: *** Waiting for unfinished jobs....
  gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:342: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o] Error 1
  In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul.inc:23,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp:8:
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory
     50 | #include "llama.cpp/ggml-impl.h"
        |          ^~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm.inc:23,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm.inc:50,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp:9:
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory
     50 | #include "llama.cpp/ggml-impl.h"
        |          ^~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory
     27 | #include "llama.cpp/ggml-impl.h"
        |          ^~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:230: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp.o] Error 1
  gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:356: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o] Error 1
  gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:118: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp.o] Error 1
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  gmake[1]: *** [CMakeFiles/Makefile2:216: CMakeFiles/llamafile.dir/all] Error 2
  gmake[1]: *** Waiting for unfinished jobs....
  [ 14%] Linking CXX shared library ../../../../bin/libggml-base.so
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/ggml/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/ggml-base.dir/link.txt --verbose=1
  /usr/bin/c++ -fPIC  -O3 -ffast-math -fopenmp -O2 -DNDEBUG -shared -Wl,-soname,libggml-base.so.0 -o ../../../../bin/libggml-base.so.0.9.4 CMakeFiles/ggml-base.dir/ggml.c.o CMakeFiles/ggml-base.dir/ggml.cpp.o CMakeFiles/ggml-base.dir/ggml-alloc.c.o CMakeFiles/ggml-base.dir/ggml-backend.cpp.o CMakeFiles/ggml-base.dir/ggml-opt.cpp.o CMakeFiles/ggml-base.dir/ggml-threading.cpp.o CMakeFiles/ggml-base.dir/ggml-quants.c.o CMakeFiles/ggml-base.dir/gguf.cpp.o  -lm
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/ggml/src && /usr/bin/cmake -E cmake_symlink_library ../../../../bin/libggml-base.so.0.9.4 ../../../../bin/libggml-base.so.0 ../../../../bin/libggml-base.so
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  [ 14%] Built target ggml-base
  gmake[1]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  gmake: *** [Makefile:136: all] Error 2
  CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0']
  CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.4.1+npu2.5.1.post3torch25aarch64']
  build_temp: /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build

将仓库/kransformers/third_party下的llama.cpp模块覆盖掉/ktransformers/archive/third_party的llama.cpp模块，就能找到llama.cpp/ggml-impl.h头文件，再次编译，头文件可以找到，但是出现了新的问题：

[ 36%] Building CXX object CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/ggml.dir/sgemm.cpp.o -MF CMakeFiles/ggml.dir/sgemm.cpp.o.d -o CMakeFiles/ggml.dir/sgemm.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/sgemm.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp
  /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  [ 37%] Built target build_info
  In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
   4135 | IQK_ALWAYS_INLINE void compute_8_blocks(const int8x16_t * qx, const Q8& q8,
        |                        ^~~~~~~~~~~~~~~~
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4112:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
 ....ommit...
   4953 | IQK_ALWAYS_INLINE int32x4_t prepare_scales_8(const uint32x4_t& v1) {
        |                             ^~~~~~~~~~~~~~~~
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4938:31: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
   4938 | IQK_ALWAYS_INLINE int32x4x2_t prepare_scales_8(const uint32x4_t& v1, const uint32x4_t& v2) {
        |                               ^~~~~~~~~~~~~~~~
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4664:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
   4664 | IQK_ALWAYS_INLINE void fusion_mul_mat_qX_K_q8_K_T_y1_d4k(
        |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4476:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
   4476 | IQK_ALWAYS_INLINE void fusion_mul_mat_qX_K_q8_K_T_y1_d6k(
        |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  In function ‘int32x4_t vdotq_laneq_s32(int32x4_t, int8x16_t, int8x16_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 1]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:3986:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate
  33614 |   return __builtin_aarch64_sdot_laneqv16qi (__r, __a, __b, __index);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate
  In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 1]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 1]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  In function ‘int32x4_t vdotq_laneq_s32(int32x4_t, int8x16_t, int8x16_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 2]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:3986:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate
 .... omit....
  33614 |   return __builtin_aarch64_sdot_laneqv16qi (__r, __a, __b, __index);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate
  In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 3]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 3]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  In function ‘int32x4_t vdotq_laneq_s32(int32x4_t, int8x16_t, int8x16_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ6K>; int BN = 3]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:3986:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate
  33614 |   return __builtin_aarch64_sdot_laneqv16qi (__r, __a, __b, __index);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  [ 37%] Built target ggml
  /usr/bin/gmake  -f third_party/llama.cpp/CMakeFiles/llama.dir/build.make third_party/llama.cpp/CMakeFiles/llama.dir/depend
  /usr/bin/gmake  -f third_party/llama.cpp/CMakeFiles/ggml_static.dir/build.make third_party/llama.cpp/CMakeFiles/ggml_static.dir/depend
  gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext /home/user3/workspace/ktransformers/archive/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/CMakeFiles/ggml_static.dir/DependInfo.cmake --color=
  gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext /home/user3/workspace/ktransformers/archive/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/CMakeFiles/llama.dir/DependInfo.cmake --color=
  In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34,
                   from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9:
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ3K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ3K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  /usr/bin/gmake  -f third_party/llama.cpp/CMakeFiles/llama.dir/build.make third_party/llama.cpp/CMakeFiles/llama.dir/build
  /usr/bin/gmake  -f third_party/llama.cpp/CMakeFiles/ggml_static.dir/build.make third_party/llama.cpp/CMakeFiles/ggml_static.dir/build
  gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o
  [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode-data.cpp.o
  [ 42%] Linking CXX static library libggml_static.a
  [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -P CMakeFiles/ggml_static.dir/cmake_clean_target.cmake
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o -MF CMakeFiles/llama.dir/llama.cpp.o.d -o CMakeFiles/llama.dir/llama.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/llama.cpp
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o -MF CMakeFiles/llama.dir/unicode.cpp.o.d -o CMakeFiles/llama.dir/unicode.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/unicode.cpp
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/llama.dir/unicode-data.cpp.o -MF CMakeFiles/llama.dir/unicode-data.cpp.o.d -o CMakeFiles/llama.dir/unicode-data.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/unicode-data.cpp
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/ggml_static.dir/link.txt --verbose=1
  /usr/bin/ar qc libggml_static.a CMakeFiles/ggml.dir/ggml.c.o CMakeFiles/ggml.dir/ggml-alloc.c.o CMakeFiles/ggml.dir/ggml-backend.c.o CMakeFiles/ggml.dir/ggml-quants.c.o CMakeFiles/ggml.dir/sgemm.cpp.o
  /usr/bin/ranlib libggml_static.a
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  [ 42%] Built target ggml_static
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ5K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ5K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ4K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’,
      inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ4K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64:
  /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate
   7749 |   return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c);
        |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:118: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp.o] Error 1
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  gmake[1]: *** [CMakeFiles/Makefile2:170: CMakeFiles/llamafile.dir/all] Error 2
  gmake[1]: *** Waiting for unfinished jobs....
  [ 44%] Linking CXX static library libllama.a
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -P CMakeFiles/llama.dir/cmake_clean_target.cmake
  cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/llama.dir/link.txt --verbose=1
  /usr/bin/ar qc libllama.a CMakeFiles/llama.dir/llama.cpp.o CMakeFiles/llama.dir/unicode.cpp.o CMakeFiles/llama.dir/unicode-data.cpp.o CMakeFiles/ggml.dir/ggml.c.o CMakeFiles/ggml.dir/ggml-alloc.c.o CMakeFiles/ggml.dir/ggml-backend.c.o CMakeFiles/ggml.dir/ggml-quants.c.o CMakeFiles/ggml.dir/sgemm.cpp.o
  /usr/bin/ranlib libllama.a
  gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  [ 44%] Built target llama
  gmake[1]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build'
  gmake: *** [Makefile:136: all] Error 2
  CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0']
  CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.4.1+npu2.5.1.post3torch25aarch64']
  build_temp: /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build
  Traceback (most recent call last):
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
      main()
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
      json_out["return_val"] = hook(**hook_input["kwargs"])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
      return _build_backend().build_wheel(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 435, in build_wheel
      return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 423, in _build
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
      self.run_setup()
    File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup
      exec(code, locals())

Others

这是什么问题，我的ARM CPU经过验证支持 armv8.2-a + fp16 + fp16fml + dotprod，也正确开启了-DKTRANSFORMERS_USE_NPU=1，是什么原因呢，是否是GCC编译器版本太低的问题。我想了解这个部署Ascend NPU官方文档更详细的硬件配置和编译器配置。

Originally created by @Mr-Xiao2021 on GitHub (Dec 9, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1690 Originally assigned to: @RICHARDNAN on GitHub. ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info 我参考如下链接进行安装： [ktransformers/doc/zh/DeepseekR1_V3_tutorial_zh_for_Ascend_NPU.md at main · kvcache-ai/ktransformers](https://github.com/kvcache-ai/ktransformers/blob/main/doc/zh/DeepseekR1_V3_tutorial_zh_for_Ascend_NPU.md#ktransformers%E9%83%A8%E7%BD%B2)](https://github.com/kvcache-ai/ktransformers/blob/main/doc/zh/DeepseekR1_V3_tutorial_zh_for_Ascend_NPU.md#ktransformers部署) 我的系统信息是： - 服务器型号：Atlas A2 A800T服务器 - NPU： A2 A800T (910B3) - CPU: HUAWEI Kunpeng 920 5250 - 内存: DDR5服务器内存（1TB） HDK: 24.1.rc3 CANN: CANN 8.3.RC1.alpha003，配套ToolKit，Kernel和NNAL均已安装好 OS: openEuler 22.03 LTS 操作系统内核为：5.10.0-60.18.0.50.oe2203.aarch64 GCC: 10.3.1 按照教程链接顺利安装好了conda环境和python部署，并安装好了指定版本torch_npu 2.5.1 ### Reproduction 将项目文件部署到机器上： - 初始化third_party。由于此过程耗时较多，且容易受网络影响导致仓库克隆失败，建议初始化一次后，将相关文件进行打包，以便后续直接解压使用。 ```bash # 拉取最新版本 git clone https://github.com/kvcache-ai/ktransformers.git cd archive git submodule update --init --recursive ``` 发现没有成功初始化，根据 `archive/.gitmodules` 文件内容，**手动**将这些子模块添加到主仓库中 ```bash # 在主项目根目录 (ktransformers/) 下执行以下所有命令 # 1. 添加 archive/ 下的第三方库 git submodule add https://github.com/ggerganov/llama.cpp.git archive/third_party/llama.cpp git submodule add https://github.com/pybind/pybind11.git archive/third_party/pybind11 git submodule add https://github.com/gabime/spdlog.git archive/third_party/spdlog git submodule add https://github.com/kvcache-ai/custom_flashinfer.git archive/third_party/custom_flashinfer git submodule add https://github.com/Cyan4973/xxHash.git archive/third_party/xxHash git submodule add https://github.com/jupp0r/prometheus-cpp archive/third_party/prometheus-cpp git submodule add https://github.com/alibaba/PhotonLibOS.git archive/third_party/PhotonLibOS # 2. 添加 kt-kernel/ 下的第三方库 git submodule add https://github.com/ggerganov/llama.cpp.git kt-kernel/third_party/llama.cpp git submodule add https://github.com/pybind/pybind11.git kt-kernel/third_party/pybind11 # 切换分支 cd archive/third_party/custom_flashinfer git checkout fix-precision-mla-merge-main cd ../../.. # 初始化并更新所有子模块（包括嵌套的子模块） git submodule update --init --recursive ``` - 对于arm平台，注释掉 ```bash ./third_party/llamafile/iqk_mul_mat_arm82.cpp ``` 中的 ```bash #define iqk_mul_mat iqk_mul_mat_arm82 #define iqk_mul_mat_moe iqk_mul_mat_moe_arm82 ``` - 执行`source /usr/local/Ascend/ascend-toolkit/set_env.sh`（以实际CANN-TOOLKIT安装路径为准）。 - 执行`apt install cmake libhwloc-dev pkg-config`安装依赖。 - 修改项目目录下 /ktransformers/config/config.yaml 中attn部分的page_size: 128 chunk_size: 16384 - 执行`USE_BALANCE_SERVE=1 USE_NUMA=1 bash ./install.sh`，等待安装完成。首次编译，发现手动添加的模块/ktransformers/archive/third_party/llama.cpp结构不符，报错： ```bash /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O3 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O3 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul.inc:23, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp:8: /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory 50 | #include "llama.cpp/ggml-impl.h" | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm.inc:23, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm.inc:50, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp:8: /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory 50 | #include "llama.cpp/ggml-impl.h" | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:244: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o] Error 1 gmake[2]: *** Waiting for unfinished jobs.... gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:342: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o] Error 1 In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul.inc:23, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp:8: /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory 50 | #include "llama.cpp/ggml-impl.h" | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm.inc:23, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm.inc:50, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp:9: /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu.h:50:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory 50 | #include "llama.cpp/ggml-impl.h" | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27:10: fatal error: llama.cpp/ggml-impl.h: No such file or directory 27 | #include "llama.cpp/ggml-impl.h" | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:230: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp.o] Error 1 gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:356: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o] Error 1 gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:118: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp.o] Error 1 gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' gmake[1]: *** [CMakeFiles/Makefile2:216: CMakeFiles/llamafile.dir/all] Error 2 gmake[1]: *** Waiting for unfinished jobs.... [ 14%] Linking CXX shared library ../../../../bin/libggml-base.so cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/ggml/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/ggml-base.dir/link.txt --verbose=1 /usr/bin/c++ -fPIC -O3 -ffast-math -fopenmp -O2 -DNDEBUG -shared -Wl,-soname,libggml-base.so.0 -o ../../../../bin/libggml-base.so.0.9.4 CMakeFiles/ggml-base.dir/ggml.c.o CMakeFiles/ggml-base.dir/ggml.cpp.o CMakeFiles/ggml-base.dir/ggml-alloc.c.o CMakeFiles/ggml-base.dir/ggml-backend.cpp.o CMakeFiles/ggml-base.dir/ggml-opt.cpp.o CMakeFiles/ggml-base.dir/ggml-threading.cpp.o CMakeFiles/ggml-base.dir/ggml-quants.c.o CMakeFiles/ggml-base.dir/gguf.cpp.o -lm cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/ggml/src && /usr/bin/cmake -E cmake_symlink_library ../../../../bin/libggml-base.so.0.9.4 ../../../../bin/libggml-base.so.0 ../../../../bin/libggml-base.so gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' [ 14%] Built target ggml-base gmake[1]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' gmake: *** [Makefile:136: all] Error 2 CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0'] CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.4.1+npu2.5.1.post3torch25aarch64'] build_temp: /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build ``` 将仓库/kransformers/third_party下的llama.cpp模块覆盖掉/ktransformers/archive/third_party的llama.cpp模块，就能找到llama.cpp/ggml-impl.h头文件，再次编译，头文件可以找到，但是出现了新的问题： ```bash [ 36%] Building CXX object CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/ggml.dir/sgemm.cpp.o -MF CMakeFiles/ggml.dir/sgemm.cpp.o.d -o CMakeFiles/ggml.dir/sgemm.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/sgemm.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp /usr/bin/c++ -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -I/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -std=gnu++17 -MD -MT CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o -MF CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o.d -o CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llamafile/tinyblas_cpu_unsupported.cpp gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' [ 37%] Built target build_info In file included from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] 4135 | IQK_ALWAYS_INLINE void compute_8_blocks(const int8x16_t * qx, const Q8& q8, | ^~~~~~~~~~~~~~~~ /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4135:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4112:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] ....ommit... 4953 | IQK_ALWAYS_INLINE int32x4_t prepare_scales_8(const uint32x4_t& v1) { | ^~~~~~~~~~~~~~~~ /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4938:31: warning: ‘always_inline’ function might not be inlinable [-Wattributes] 4938 | IQK_ALWAYS_INLINE int32x4x2_t prepare_scales_8(const uint32x4_t& v1, const uint32x4_t& v2) { | ^~~~~~~~~~~~~~~~ /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4664:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] 4664 | IQK_ALWAYS_INLINE void fusion_mul_mat_qX_K_q8_K_T_y1_d4k( | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4476:24: warning: ‘always_inline’ function might not be inlinable [-Wattributes] 4476 | IQK_ALWAYS_INLINE void fusion_mul_mat_qX_K_q8_K_T_y1_d6k( | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: In function ‘int32x4_t vdotq_laneq_s32(int32x4_t, int8x16_t, int8x16_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 1]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:3986:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate 33614 | return __builtin_aarch64_sdot_laneqv16qi (__r, __a, __b, __index); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~ /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 1]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 1]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: In function ‘int32x4_t vdotq_laneq_s32(int32x4_t, int8x16_t, int8x16_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 2]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:3986:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate .... omit.... 33614 | return __builtin_aarch64_sdot_laneqv16qi (__r, __a, __b, __index); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~ /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 3]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 3]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: In function ‘int32x4_t vdotq_laneq_s32(int32x4_t, int8x16_t, int8x16_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ6K>; int BN = 3]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:3986:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate 33614 | return __builtin_aarch64_sdot_laneqv16qi (__r, __a, __b, __index); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~ /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:33614:44: error: argument 4 must be a constant immediate gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' [ 37%] Built target ggml /usr/bin/gmake -f third_party/llama.cpp/CMakeFiles/llama.dir/build.make third_party/llama.cpp/CMakeFiles/llama.dir/depend /usr/bin/gmake -f third_party/llama.cpp/CMakeFiles/ggml_static.dir/build.make third_party/llama.cpp/CMakeFiles/ggml_static.dir/depend gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext /home/user3/workspace/ktransformers/archive/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/CMakeFiles/ggml_static.dir/DependInfo.cmake --color= gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext /home/user3/workspace/ktransformers/archive/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp/CMakeFiles/llama.dir/DependInfo.cmake --color= In file included from /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/../../third_party/llama.cpp/ggml-impl.h:160, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:27, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat.inc:34, from /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp:9: In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ3K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ3K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' /usr/bin/gmake -f third_party/llama.cpp/CMakeFiles/llama.dir/build.make third_party/llama.cpp/CMakeFiles/llama.dir/build /usr/bin/gmake -f third_party/llama.cpp/CMakeFiles/ggml_static.dir/build.make third_party/llama.cpp/CMakeFiles/ggml_static.dir/build gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' gmake[2]: Entering directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode-data.cpp.o [ 42%] Linking CXX static library libggml_static.a [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -P CMakeFiles/ggml_static.dir/cmake_clean_target.cmake cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o -MF CMakeFiles/llama.dir/llama.cpp.o.d -o CMakeFiles/llama.dir/llama.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/llama.cpp cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o -MF CMakeFiles/llama.dir/unicode.cpp.o.d -o CMakeFiles/llama.dir/unicode.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/unicode.cpp cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/c++ -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_LLAMAFILE -DGGML_USE_OPENMP -DKTRANSFORMERS_USE_NPU=1 -D_GLIBCXX_USE_CXX11_ABI=0 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -I/home/user3/workspace/ktransformers/archive/third_party/llama.cpp/. -O1 -ffast-math -fopenmp -O2 -DNDEBUG -fPIC -march=armv8.2-a+fp16+fp16fml+dotprod -lnuma -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-array-bounds -Wno-format-truncation -Wextra-semi -fopenmp -std=gnu++11 -MD -MT third_party/llama.cpp/CMakeFiles/llama.dir/unicode-data.cpp.o -MF CMakeFiles/llama.dir/unicode-data.cpp.o.d -o CMakeFiles/llama.dir/unicode-data.cpp.o -c /home/user3/workspace/ktransformers/archive/third_party/llama.cpp/unicode-data.cpp cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/ggml_static.dir/link.txt --verbose=1 /usr/bin/ar qc libggml_static.a CMakeFiles/ggml.dir/ggml.c.o CMakeFiles/ggml.dir/ggml-alloc.c.o CMakeFiles/ggml.dir/ggml-backend.c.o CMakeFiles/ggml.dir/ggml-quants.c.o CMakeFiles/ggml.dir/sgemm.cpp.o /usr/bin/ranlib libggml_static.a gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ2K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ [ 42%] Built target ggml_static In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerIQ4XS>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ5K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ5K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ4K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4037:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ In function ‘int32x4_t vmlal_laneq_s16(int32x4_t, int16x4_t, int16x8_t, int)’, inlined from ‘void {anonymous}::matmul_v2_kernel(const Dequantizer*, const block_q8_K**, const {anonymous}::DataInfo&, int, int) [with Dequantizer = {anonymous}::BlockQxK<{anonymous}::DequantizerQ4K>; int BN = 4]’ at /home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm.inc:4039:64: /usr/lib/gcc/aarch64-linux-gnu/10.3.1/include/arm_neon.h:7749:49: error: argument 4 must be a constant immediate 7749 | return __builtin_aarch64_vec_smlal_laneq_v4hi (__acc, __a, __b, __c); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ gmake[2]: *** [CMakeFiles/llamafile.dir/build.make:118: CMakeFiles/llamafile.dir/home/user3/workspace/ktransformers/archive/third_party/llamafile/iqk_mul_mat_arm82.cpp.o] Error 1 gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' gmake[1]: *** [CMakeFiles/Makefile2:170: CMakeFiles/llamafile.dir/all] Error 2 gmake[1]: *** Waiting for unfinished jobs.... [ 44%] Linking CXX static library libllama.a cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -P CMakeFiles/llama.dir/cmake_clean_target.cmake cd /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build/third_party/llama.cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/llama.dir/link.txt --verbose=1 /usr/bin/ar qc libllama.a CMakeFiles/llama.dir/llama.cpp.o CMakeFiles/llama.dir/unicode.cpp.o CMakeFiles/llama.dir/unicode-data.cpp.o CMakeFiles/ggml.dir/ggml.c.o CMakeFiles/ggml.dir/ggml-alloc.c.o CMakeFiles/ggml.dir/ggml-backend.c.o CMakeFiles/ggml.dir/ggml-quants.c.o CMakeFiles/ggml.dir/sgemm.cpp.o /usr/bin/ranlib libllama.a gmake[2]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' [ 44%] Built target llama gmake[1]: Leaving directory '/home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build' gmake: *** [Makefile:136: all] Error 2 CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0'] CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/user3/workspace/ktransformers/archive/build/lib.linux-aarch64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/user3/miniconda3/envs/py311/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_NPU=ON', '-DKTRANSFORMERS_USE_CUDA=OFF', '-D_GLIBCXX_USE_CXX11_ABI=0', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.4.1+npu2.5.1.post3torch25aarch64'] build_temp: /home/user3/workspace/ktransformers/archive/csrc/ktransformers_ext/build Traceback (most recent call last): File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module> main() File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main json_out["return_val"] = hook(**hook_input["kwargs"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel return _build_backend().build_wheel( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 435, in build_wheel return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 423, in _build return self._build_with_temp_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir self.run_setup() File "/home/user3/miniconda3/envs/py311/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup exec(code, locals()) ``` ### Others 这是什么问题，我的ARM CPU经过验证支持 armv8.2-a + fp16 + fp16fml + dotprod，也正确开启了-DKTRANSFORMERS_USE_NPU=1，是什么原因呢，是否是GCC编译器版本太低的问题。我想了解这个部署Ascend NPU官方文档更详细的硬件配置和编译器配置。

bsakaguchi added the

bug

label

2026-10-29 14:48:18 +00:00

bsakaguchi commented

2026-10-29 14:48:19 +00:00

Author

Owner

@RICHARDNAN commented on GitHub (Dec 9, 2025):

您好，由于conda环境不可控因素较多，可以尝试参考这个pr中在MindIE镜像中尝试进行编译：https://github.com/kvcache-ai/ktransformers/pull/1691/files

@RICHARDNAN commented on GitHub (Dec 9, 2025): 您好，由于conda环境不可控因素较多，可以尝试参考这个pr中在MindIE镜像中尝试进行编译：https://github.com/kvcache-ai/ktransformers/pull/1691/files