kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1667] [kt-kernel] Error: AMD CPU 编译成功但提示 undefined symbol: aocl_gemm_s8s8s32os32 #1073

New issue
Closed
opened 2026-10-29 14:48:13 +00:00 by bsakaguchi · 5 comments
bsakaguchi commented 2026-10-29 14:48:13 +00:00
Owner
Copy link

Originally created by @xzy103 on GitHub (Dec 4, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1667

Reminder

  • I have read the above rules and searched the existing issues.

System Info

系统：WSL Ubuntu 24.04
硬件：AMD CPU

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: AuthenticAMD
Model name: AMD Ryzen 7 5800H with Radeon Graphics
CPU family: 25
Model: 80
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
Stepping: 0
BogoMIPS: 6387.84
Virtualization features:
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
Caches (sum of all):
L1d: 256 KiB (8 instances)
L1i: 256 KiB (8 instances)
L2: 4 MiB (8 instances)
L3: 16 MiB (1 instance)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-15

Reproduction

kt-kernel，可以编译成功，但在最终验证时提示异常：

# 验证安装
python -c "from kt_kernel import KTMoEWrapper; print('✓ kt-kernel installed successfully')" 

# 提示 undefined symbol: aocl_gemm_s8s8s32os32
Traceback (most recent call last): 
  File "<string>", line 1, in <module>
  File "/root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/__init__.py", line 27, in <module>
    from .experts import KTMoEWrapper
  File "/root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/experts.py", line 17, in <module>
    from .experts_base import BaseMoEWrapper, KExpertsCPUBuffer
  File "/root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/experts_base.py", line 18, in <module>
    import kt_kernel_ext
ImportError: /root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel_ext.cpython-311-x86_64-linux-gnu.so: undefined symbol: aocl_gemm_s8s8s32os32

我查找到该符号函数是存在的：

cd /opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib
> nm -D *.so | grep aocl_gemm_s8s8s32os32
# 输出
0000000000c652c0 T aocl_gemm_s8s8s32os32
0000000000c05f40 T aocl_gemm_s8s8s32os32

通过手动设置环境变量解决了问题：

export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis-mt.so
export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so

不过这种方法治标不治本。

问了下ai，说是CMakeLists.txt中有关if(KTRANSFORMERS_CPU_MOE_AMD)那段：

这里只是“找到 BLIS 并把路径存在变量里”，但还没把这个库真正 link 到 kt_kernel_ext 上。
按注释，本来应该在后面 pybind11_add_module(kt_kernel_ext ...) 之后，再调用 target_include_directories/target_link_libraries 去用 _KT_BLIS_INCLUDE_DIR 和 _KT_BLIS_LIBRARY，但现在这一步显然缺了，所以你才需要用 LD_PRELOAD 去补救。

我看不太懂，求大神补救~

Others

No response

Originally created by @xzy103 on GitHub (Dec 4, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1667 ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info 系统：WSL Ubuntu 24.04 硬件：AMD CPU Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: AuthenticAMD Model name: AMD Ryzen 7 5800H with Radeon Graphics CPU family: 25 Model: 80 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 1 Stepping: 0 BogoMIPS: 6387.84 Virtualization features: Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full Caches (sum of all): L1d: 256 KiB (8 instances) L1i: 256 KiB (8 instances) L2: 4 MiB (8 instances) L3: 16 MiB (1 instance) NUMA: NUMA node(s): 1 NUMA node0 CPU(s): 0-15 ### Reproduction kt-kernel，可以编译成功，但在最终验证时提示异常： ```bash # 验证安装 python -c "from kt_kernel import KTMoEWrapper; print('✓ kt-kernel installed successfully')" # 提示 undefined symbol: aocl_gemm_s8s8s32os32 Traceback (most recent call last): File "<string>", line 1, in <module> File "/root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/__init__.py", line 27, in <module> from .experts import KTMoEWrapper File "/root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/experts.py", line 17, in <module> from .experts_base import BaseMoEWrapper, KExpertsCPUBuffer File "/root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel/experts_base.py", line 18, in <module> import kt_kernel_ext ImportError: /root/data/miniconda3/envs/kt-kernel/lib/python3.11/site-packages/kt_kernel_ext.cpython-311-x86_64-linux-gnu.so: undefined symbol: aocl_gemm_s8s8s32os32 ``` 我查找到该符号函数是存在的： ```bash cd /opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib > nm -D *.so | grep aocl_gemm_s8s8s32os32 # 输出 0000000000c652c0 T aocl_gemm_s8s8s32os32 0000000000c05f40 T aocl_gemm_s8s8s32os32 ``` 通过手动设置环境变量解决了问题： ```bash export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis-mt.so export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so ``` 不过这种方法治标不治本。 问了下ai，说是CMakeLists.txt中有关`if(KTRANSFORMERS_CPU_MOE_AMD)`那段： ```txt 这里只是“找到 BLIS 并把路径存在变量里”，但还没把这个库真正 link 到 kt_kernel_ext 上。 按注释，本来应该在后面 pybind11_add_module(kt_kernel_ext ...) 之后，再调用 target_include_directories/target_link_libraries 去用 _KT_BLIS_INCLUDE_DIR 和 _KT_BLIS_LIBRARY，但现在这一步显然缺了，所以你才需要用 LD_PRELOAD 去补救。 ``` 我看不太懂，求大神补救~ ### Others _No response_
bsakaguchi 2026-10-29 14:48:13 +00:00
  • closed this issue
  • added the
    bug
         label
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@0xhaggis commented on GitHub (Dec 5, 2025):

Same error when compiling from source.

  • Python 3.11 venv on linux
  • Pull latest ktransformers from git
  • Update submodules
  • Change into kt-kernel dir
  • uv pip install .
> uv pip install .
Using Python 3.11.13 environment at: /home/0xhaggis/ktransformers/.venv
Resolved 54 packages in 202ms
  × Failed to build `kt-kernel @ file:///home/0xhaggis/ktransformers/kt-kernel`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1)

      [stdout]
      running bdist_wheel
      running build
      running build_py
      copying python/__init__.py -> build/lib.linux-x86_64-cpython-311/kt_kernel
      copying python/experts.py -> build/lib.linux-x86_64-cpython-311/kt_kernel
      copying python/experts_base.py -> build/lib.linux-x86_64-cpython-311/kt_kernel
      copying python/utils/moe_kernel.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils
      copying python/utils/__init__.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils
      copying python/utils/amx.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils
      copying python/utils/loader.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils
      copying python/utils/llamafile.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils
      running egg_info
      writing kt_kernel.egg-info/PKG-INFO
      writing dependency_links to kt_kernel.egg-info/dependency_links.txt
      writing requirements to kt_kernel.egg-info/requires.txt
      writing top-level names to kt_kernel.egg-info/top_level.txt
      reading manifest file 'kt_kernel.egg-info/SOURCES.txt'
      writing manifest file 'kt_kernel.egg-info/SOURCES.txt'
      running build_ext
      -- The C compiler identification is GNU 13.3.0
      -- The CXX compiler identification is GNU 13.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Detected git worktree at /home/0xhaggis/ktransformers; installing hooks from kt-kernel/.githooks
      -- Found OpenMP_C: -fopenmp (found version "4.5")
      -- Found OpenMP_CXX: -fopenmp (found version "4.5")
      -- Found OpenMP: TRUE (found version "4.5")
      -- CMAKE_CXX_FLAGS:  -O3 -ffast-math
      -- CMAKE_SYSTEM_PROCESSOR: x86_64
      -- x86 detected
      -- Found BLIS include at /usr/include/x86_64-linux-gnu
      -- Found BLIS library /usr/lib/x86_64-linux-gnu/libblis.so
      -- ARCH_FLAGS: -mf16c;-mfma;-mavx;-mfma;-msse3;-mf16c;-mavx2;-mfma;-msse3;-mf16c;-mavx512f;-mavx512bw;-mavx512dq;-mfma;-mf16c;-msse3;-mfma;-mf16c;-mavx512bf16;-mavx512vnni
      -- pybind11 v2.14.0 dev1
      -- Found PythonInterp: /home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/bin/python (found suitable version "3.11.13", minimum required is "3.7")
      -- Found PythonLibs: /home/0xhaggis/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/libpython3.11.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Found Git: /usr/bin/git (found version "2.43.0")
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Found OpenMP_C: -fopenmp (found version "4.5")
      -- Found OpenMP_CXX: -fopenmp (found version "4.5")
      -- OpenMP found
      -- ccache found, compilation results will be cached. Disable with LLAMA_CCACHE=OFF.
      -- CMAKE_SYSTEM_PROCESSOR: x86_64
      -- x86 detected
      -- CUDA detected
      -- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include;/usr/local/cuda/targets/x86_64-linux/include/cccl (found version "13.0.88")
      -- enabling CUDA
      -- The CUDA compiler identification is NVIDIA 13.0.88 with host compiler GNU 13.3.0
      -- Detecting CUDA compiler ABI info
      -- Detecting CUDA compiler ABI info - done
      -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
      -- Detecting CUDA compile features
      -- Detecting CUDA compile features - done
      -- SOURCE_DIR7: /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/la/mat_kernel.cpp;/home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp
      -- Using clang-format 18.1.3 at /usr/bin/clang-format-18
      -- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.1")
      -- Checking for one of the modules 'hwloc'
      -- LTO: disabled
      -- NUMA library found: /usr/lib/x86_64-linux-gnu/libnuma.so - enabling NUMA support
      -- Configuring done (3.3s)
      -- Generating done (0.0s)
      -- Build files have been written to: /home/0xhaggis/ktransformers/kt-kernel/build/temp.linux-x86_64-cpython-311/kt_kernel_ext_Release
      [  1%] Generating build details from Git
      -- Found Git: /usr/bin/git (found version "2.43.0")
      [  2%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml.c.o
      [  3%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml-backend.c.o
      [  5%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.o
      [  6%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml-quants.c.o
      [  7%] Building CXX object third_party/llama.cpp/CMakeFiles/ggml.dir/sgemm.cpp.o
      [  8%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_avx2.cpp.o
      [ 10%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/flags.cpp.o
      [ 11%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx.cpp.o
      [ 12%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_zen4.cpp.o
      [ 14%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_fma.cpp.o
      [ 16%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx2.cpp.o
      [ 16%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_arm82.cpp.o
      [ 17%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx512f.cpp.o
      [ 19%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/sgemm.cpp.o
      [ 20%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avxvnni.cpp.o
      [ 21%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_zen4.cpp.o
      [ 23%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o
      [ 24%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp.o
      [ 24%] Built target ggml
      [ 25%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o
      [ 26%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o
      [ 28%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o
      [ 29%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o
      [ 30%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o
      [ 32%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o
      [ 33%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o
      [ 34%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o
      [ 35%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o
      [ 37%] Linking CXX static library libggml_static.a
      [ 38%] Linking CXX static library libllamafile.a
      [ 39%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o
      [ 41%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o
      [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode-data.cpp.o
      [ 43%] Linking CXX static library libllama.a
      [ 44%] Building CXX object third_party/llama.cpp/common/CMakeFiles/build_info.dir/build-info.cpp.o
      [ 44%] Built target build_info
      [ 44%] Built target ggml_static
      [ 44%] Built target llamafile
      [ 44%] Built target llama
      [ 46%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/common.cpp.o
      [ 47%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/console.cpp.o
      [ 48%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/sampling.cpp.o
      [ 51%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/grammar-parser.cpp.o
      [ 51%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o
      [ 52%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/train.cpp.o
      [ 53%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/ngram-cache.cpp.o
      [ 55%] Linking CXX static library libcommon.a
      [ 56%] Building CXX object CMakeFiles/kt_kernel_ext.dir/cpu_backend/shared_mem_buffer.cpp.o
      [ 57%] Building CXX object CMakeFiles/kt_kernel_ext.dir/ext_bindings.cpp.o
      [ 58%] Building CXX object CMakeFiles/kt_kernel_ext.dir/cpu_backend/task_queue.cpp.o
      [ 60%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/llamafile/mlp.cpp.o
      [ 61%] Building CXX object CMakeFiles/kt_kernel_ext.dir/cpu_backend/worker_pool.cpp.o
      [ 62%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/llamafile/linear.cpp.o
      [ 64%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/flags.cpp.o
      [ 65%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_avx2.cpp.o
      [ 66%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_zen4.cpp.o
      [ 69%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/sgemm.cpp.o
      [ 69%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_arm82.cpp.o
      [ 70%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx.cpp.o
      [ 71%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx2.cpp.o
      [ 73%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx512f.cpp.o
      [ 74%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_zen4.cpp.o
      [ 75%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avxvnni.cpp.o
      [ 76%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp.o
      [ 78%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_fma.cpp.o
      [ 79%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o
      [ 80%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o
      [ 82%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o
      [ 83%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o
      [ 84%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o
      [ 85%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o
      [ 87%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o
      [ 88%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o
      [ 89%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o
      [ 91%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o
      [ 92%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_attn.cpp.o
      [ 93%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_load_dump.cpp.o
      [ 94%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_read_write.cpp.o
      [ 96%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_utils.cpp.o
      [ 97%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/moe_kernel/la/mat_kernel.cpp.o
      [ 98%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp.o
      [ 98%] Built target common
      -- CPUINFER_USE_CUDA not set; auto-detected CUDA toolkit: YES
      Detected CPU info: {'vendor': 'amd', 'arch': 'x86_64', 'features': {'AVX2', 'AVX512'}, 'raw': {'flags': {'bmi2', '3dnowprefetch', 'ibpb', 'fsrm', 'hw_pstate', 'vgif', 'skinit',
      'avx512_vp2intersect', 'extd_apicid', 'svm_lock', 'vmcb_clean', 'vme', 'amd_lbr_v2', 'bmi1', 'aperfmperf', 'irperf', 'cqm_llc', 'avx2', 'nopl', 'flush_l1d', 'smap', 'avx512ifma',
      'rapl', 'avx512dq', 'cppc', 'cpuid', 'avx512cd', 'pae', 'ibs', 'pge', 'mce', 'rdt_a', 'avic', 'ibrs', 'cr8_legacy', 'x2avic', 'apic', 'fpu', 'avx512bw', 'fma', 'osvw', 'cx8',
      'cqm_occup_llc', 'vaes', 'wdt', 'ssse3', 'sse', 'rdpru', 'erms', 'rdrand', 'pni', 'vnmi', 'xsave', 'sha_ni', 'aes', 'ht', 'mba', 'tsc_scale', 'avx512_vnni', 'vpclmulqdq',
      'mmxext', 'bus_lock_detect', 'mmx', 'xsaves', 'rdpid', 'perfctr_llc', 'pausefilter', 'fxsr', 'decodeassists', 'pat', 'syscall', 'cqm_mbm_total', 'pse36', 'pclmulqdq', 'vmmcall',
      'adx', 'rep_good', 'msr', 'clflush', 'ssbd', 'cqm', 'nonstop_tsc', 'abm', 'amd_ppin', 'fsgsbase', 'xsaveopt', 'lbrv', 'movdiri', 'tce', 'smep', 'user_shstk', 'overflow_recov',
      'avx512vbmi', 'topoext', 'wbnoinvd', 'tsc', 'constant_tsc', 'mca', 'pse', 'popcnt', 'avx512_vbmi2', 'movdir64b', 'nrip_save', 'f16c', 'sse4_2', 'cat_l3', 'arat', 'xsavec',
      'sep', 'tsc_adjust', 'ospke', 'ibrs_enhanced', 'smca', 'cpb', 'invpcid', 'rdseed', 'lahf_lm', 'sse4_1', 'lm', 'avx512_bf16', 'avx512_bitalg', 'sse4a', 'movbe', 'xsaveerptr',
      'clzero', 'fxsr_opt', 'v_spec_ctrl', 'sse2', 'xgetbv1', 'extapic', 'pdpe1gb', 'avx', 'v_vmsave_vmload', 'flushbyasid', 'cmp_legacy', 'misalignsse', 'perfctr_core', 'debug_swap',
      'umip', 'perfctr_nb', 'cqm_mbm_local', 'stibp', 'avx512f', 'succor', 'la57', 'gfni', 'avx_vnni', 'monitor', 'cmov', 'nx', 'clwb', 'pku', 'clflushopt', 'mtrr', 'perfmon_v2', 'de',
      'avx512vl', 'amd_ibpb_ret', 'svm', 'pcid', 'x2apic', 'mwaitx', 'bpext', 'avx512_vpopcntdq', 'npt', 'pfthreshold', 'cdp_l3', 'cx16', 'rdtscp'}}}
      -- Detected AMD CPU; enabling AMD MoE kernel (-DKTRANSFORMERS_CPU_MOE_AMD=ON)
      -- Forward CPUINFER_ENABLE_AMX -> -DKTRANSFORMERS_CPU_USE_AMX=OFF
      -- Enabling AMX/AVX512 umbrella (-DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON)
      -- Auto-enabling MOE kernel (-DKTRANSFORMERS_CPU_MOE_KERNEL=ON) because CPUINFER_ENABLE_AMD or CPUINFER_ENABLE_KML is ON
      -- CPU detection: vendor=amd arch=x86_64 features=['AVX2', 'AVX512']
      -- Enabling CUDA backend (-DKTRANSFORMERS_USE_CUDA=ON)
      -- Auto-detected nvcc: /usr/local/cuda/bin/nvcc (adding -DCMAKE_CUDA_COMPILER)
      -- CMake configure args:
          -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/0xhaggis/ktransformers/kt-kernel/build/lib.linux-x86_64-cpython-311/
          -DPYTHON_EXECUTABLE=/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/bin/python
          -DCMAKE_BUILD_TYPE=Release
          -DLLAMA_NATIVE=OFF
          -DLLAMA_FMA=ON
          -DLLAMA_F16C=ON
          -DLLAMA_AVX=ON
          -DLLAMA_AVX2=ON
          -DLLAMA_AVX512=ON
          -DKTRANSFORMERS_CPU_MOE_AMD=ON
          -DKTRANSFORMERS_CPU_USE_AMX=OFF
          -DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON
          -DKTRANSFORMERS_CPU_MOE_KERNEL=ON
          -DKTRANSFORMERS_USE_CUDA=ON
          -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
      -- CMake build args: --build . --config Release --parallel 256

      [stderr]
      /home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsWarning: `license` overwritten by `pyproject.toml`
        corresp(dist, value, root_dir)
      CMake Warning at CMakeLists.txt:265 (message):
        pure AVX is not supported at least avx2


      CMake Deprecation Warning at /home/0xhaggis/ktransformers/third_party/pybind11/CMakeLists.txt:13 (cmake_minimum_required):
        Compatibility with CMake < 3.10 will be removed from a future version of
        CMake.

        Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
        to tell CMake that the project requires at least <min> but has been updated
        to work with policies introduced by <max> or earlier.


      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘void decode_cblas_gemm_s8s8s32(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE,
      KERNEL_CBLAS_TRANSPOSE, KERNEL_CBLAS_OFFSET, size_t, size_t, size_t, float, const void*, size_t, BLASINT8, const void*, size_t, BLASINT8, float, int32_t*, size_t, const int32_t*)’:
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:46:3: error: ‘aocl_gemm_s8s8s32os32’ was not declared in this scope
         46 |   aocl_gemm_s8s8s32os32(order, op_a, op_b, static_cast<dim_t>(m), static_cast<dim_t>(n), static_cast<dim_t>(k),
            |   ^~~~~~~~~~~~~~~~~~~~~
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘void prefill_cblas_gemm_s8s8s32(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE,
      KERNEL_CBLAS_TRANSPOSE, KERNEL_CBLAS_OFFSET, size_t, size_t, size_t, float, const void*, size_t, BLASINT8, const void*, size_t, BLASINT8, float, int32_t*, size_t, const int32_t*)’:
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:61:3: error: ‘aocl_gemm_s8s8s32os32’ was not declared in this scope
         61 |   aocl_gemm_s8s8s32os32(order, op_a, op_b, static_cast<dim_t>(m), static_cast<dim_t>(n), static_cast<dim_t>(k),
            |   ^~~~~~~~~~~~~~~~~~~~~
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘void reorder_B_gemm(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE, size_t,
      size_t, size_t, const void*, void*)’:
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:89:3: error: ‘aocl_reorder_s8s8s32os32’ was not declared in this scope
         89 |   aocl_reorder_s8s8s32os32(order, op_b, 'B', static_cast<const int8_t*>(b), static_cast<int8_t*>(b_reordered), k, n,
            |   ^~~~~~~~~~~~~~~~~~~~~~~~
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘size_t get_reorder_B_size(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE,
      size_t, size_t)’:
      /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:95:10: error: ‘aocl_get_reorder_buf_size_s8s8s32os32’ was not declared in this scope
         95 |   return aocl_get_reorder_buf_size_s8s8s32os32(ToAoclOrder(layout), ToAoclTranspose(transb), 'B', k, n);
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      gmake[2]: *** [CMakeFiles/kt_kernel_ext.dir/build.make:541: CMakeFiles/kt_kernel_ext.dir/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp.o] Error 1
      gmake[1]: *** [CMakeFiles/Makefile2:265: CMakeFiles/kt_kernel_ext.dir/all] Error 2
      gmake: *** [Makefile:136: all] Error 2
      Traceback (most recent call last):
        File "<string>", line 11, in <module>
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 432, in build_wheel
          return _build(['bdist_wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 423, in _build
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
          self.run_setup()
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 436, in <module>
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/__init__.py", line 115, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 186, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
          dist.run_commands()
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
          self.run_command(cmd)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
          super().run_command(command)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
          self.run_command("build")
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
          self.distribution.run_command(command)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
          super().run_command(command)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
          self.distribution.run_command(command)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
          super().run_command(command)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "<string>", line 145, in run
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 96, in run
          _build_ext.run(self)
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
          self.build_extensions()
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 484, in build_extensions
          self._build_extensions_serial()
        File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 510, in _build_extensions_serial
          self.build_extension(ext)
        File "<string>", line 412, in build_extension
        File "/home/0xhaggis/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/subprocess.py", line 571, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--parallel', '256']' returned non-zero exit status 2.

      hint: This usually indicates a problem with the package or the build environment.
<!-- gh-comment-id:3614890989 --> @0xhaggis commented on GitHub (Dec 5, 2025): Same error when compiling from source. - Python 3.11 venv on linux - Pull latest ktransformers from git - Update submodules - Change into kt-kernel dir - uv pip install . ``` > uv pip install . Using Python 3.11.13 environment at: /home/0xhaggis/ktransformers/.venv Resolved 54 packages in 202ms × Failed to build `kt-kernel @ file:///home/0xhaggis/ktransformers/kt-kernel` ├─▶ The build backend returned an error ╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1) [stdout] running bdist_wheel running build running build_py copying python/__init__.py -> build/lib.linux-x86_64-cpython-311/kt_kernel copying python/experts.py -> build/lib.linux-x86_64-cpython-311/kt_kernel copying python/experts_base.py -> build/lib.linux-x86_64-cpython-311/kt_kernel copying python/utils/moe_kernel.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils copying python/utils/__init__.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils copying python/utils/amx.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils copying python/utils/loader.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils copying python/utils/llamafile.py -> build/lib.linux-x86_64-cpython-311/kt_kernel/utils running egg_info writing kt_kernel.egg-info/PKG-INFO writing dependency_links to kt_kernel.egg-info/dependency_links.txt writing requirements to kt_kernel.egg-info/requires.txt writing top-level names to kt_kernel.egg-info/top_level.txt reading manifest file 'kt_kernel.egg-info/SOURCES.txt' writing manifest file 'kt_kernel.egg-info/SOURCES.txt' running build_ext -- The C compiler identification is GNU 13.3.0 -- The CXX compiler identification is GNU 13.3.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Detected git worktree at /home/0xhaggis/ktransformers; installing hooks from kt-kernel/.githooks -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- Found OpenMP: TRUE (found version "4.5") -- CMAKE_CXX_FLAGS: -O3 -ffast-math -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- x86 detected -- Found BLIS include at /usr/include/x86_64-linux-gnu -- Found BLIS library /usr/lib/x86_64-linux-gnu/libblis.so -- ARCH_FLAGS: -mf16c;-mfma;-mavx;-mfma;-msse3;-mf16c;-mavx2;-mfma;-msse3;-mf16c;-mavx512f;-mavx512bw;-mavx512dq;-mfma;-mf16c;-msse3;-mfma;-mf16c;-mavx512bf16;-mavx512vnni -- pybind11 v2.14.0 dev1 -- Found PythonInterp: /home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/bin/python (found suitable version "3.11.13", minimum required is "3.7") -- Found PythonLibs: /home/0xhaggis/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/libpython3.11.so -- Performing Test HAS_FLTO -- Performing Test HAS_FLTO - Success -- Found Git: /usr/bin/git (found version "2.43.0") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- OpenMP found -- ccache found, compilation results will be cached. Disable with LLAMA_CCACHE=OFF. -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- x86 detected -- CUDA detected -- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include;/usr/local/cuda/targets/x86_64-linux/include/cccl (found version "13.0.88") -- enabling CUDA -- The CUDA compiler identification is NVIDIA 13.0.88 with host compiler GNU 13.3.0 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- SOURCE_DIR7: /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/la/mat_kernel.cpp;/home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp -- Using clang-format 18.1.3 at /usr/bin/clang-format-18 -- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.1") -- Checking for one of the modules 'hwloc' -- LTO: disabled -- NUMA library found: /usr/lib/x86_64-linux-gnu/libnuma.so - enabling NUMA support -- Configuring done (3.3s) -- Generating done (0.0s) -- Build files have been written to: /home/0xhaggis/ktransformers/kt-kernel/build/temp.linux-x86_64-cpython-311/kt_kernel_ext_Release [ 1%] Generating build details from Git -- Found Git: /usr/bin/git (found version "2.43.0") [ 2%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml.c.o [ 3%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml-backend.c.o [ 5%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.o [ 6%] Building C object third_party/llama.cpp/CMakeFiles/ggml.dir/ggml-quants.c.o [ 7%] Building CXX object third_party/llama.cpp/CMakeFiles/ggml.dir/sgemm.cpp.o [ 8%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_avx2.cpp.o [ 10%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/flags.cpp.o [ 11%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx.cpp.o [ 12%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_zen4.cpp.o [ 14%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_fma.cpp.o [ 16%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx2.cpp.o [ 16%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_arm82.cpp.o [ 17%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx512f.cpp.o [ 19%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/sgemm.cpp.o [ 20%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avxvnni.cpp.o [ 21%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_zen4.cpp.o [ 23%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o [ 24%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp.o [ 24%] Built target ggml [ 25%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o [ 26%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o [ 28%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o [ 29%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o [ 30%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o [ 32%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o [ 33%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o [ 34%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o [ 35%] Building CXX object CMakeFiles/llamafile.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o [ 37%] Linking CXX static library libggml_static.a [ 38%] Linking CXX static library libllamafile.a [ 39%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o [ 41%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o [ 42%] Building CXX object third_party/llama.cpp/CMakeFiles/llama.dir/unicode-data.cpp.o [ 43%] Linking CXX static library libllama.a [ 44%] Building CXX object third_party/llama.cpp/common/CMakeFiles/build_info.dir/build-info.cpp.o [ 44%] Built target build_info [ 44%] Built target ggml_static [ 44%] Built target llamafile [ 44%] Built target llama [ 46%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/common.cpp.o [ 47%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/console.cpp.o [ 48%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/sampling.cpp.o [ 51%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/grammar-parser.cpp.o [ 51%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o [ 52%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/train.cpp.o [ 53%] Building CXX object third_party/llama.cpp/common/CMakeFiles/common.dir/ngram-cache.cpp.o [ 55%] Linking CXX static library libcommon.a [ 56%] Building CXX object CMakeFiles/kt_kernel_ext.dir/cpu_backend/shared_mem_buffer.cpp.o [ 57%] Building CXX object CMakeFiles/kt_kernel_ext.dir/ext_bindings.cpp.o [ 58%] Building CXX object CMakeFiles/kt_kernel_ext.dir/cpu_backend/task_queue.cpp.o [ 60%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/llamafile/mlp.cpp.o [ 61%] Building CXX object CMakeFiles/kt_kernel_ext.dir/cpu_backend/worker_pool.cpp.o [ 62%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/llamafile/linear.cpp.o [ 64%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/flags.cpp.o [ 65%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_avx2.cpp.o [ 66%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_amd_zen4.cpp.o [ 69%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/sgemm.cpp.o [ 69%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/iqk_mul_mat_arm82.cpp.o [ 70%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx.cpp.o [ 71%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx2.cpp.o [ 73%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avx512f.cpp.o [ 74%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_zen4.cpp.o [ 75%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_avxvnni.cpp.o [ 76%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm80.cpp.o [ 78%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_amd_fma.cpp.o [ 79%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_mixmul_arm82.cpp.o [ 80%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx.cpp.o [ 82%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx2.cpp.o [ 83%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avx512f.cpp.o [ 84%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_fma.cpp.o [ 85%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm82.cpp.o [ 87%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_avxvnni.cpp.o [ 88%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_arm80.cpp.o [ 89%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_sgemm_amd_zen4.cpp.o [ 91%] Building CXX object CMakeFiles/kt_kernel_ext.dir/home/0xhaggis/ktransformers/third_party/llamafile/tinyblas_cpu_unsupported.cpp.o [ 92%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_attn.cpp.o [ 93%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_load_dump.cpp.o [ 94%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_read_write.cpp.o [ 96%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/kvcache/kvcache_utils.cpp.o [ 97%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/moe_kernel/la/mat_kernel.cpp.o [ 98%] Building CXX object CMakeFiles/kt_kernel_ext.dir/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp.o [ 98%] Built target common -- CPUINFER_USE_CUDA not set; auto-detected CUDA toolkit: YES Detected CPU info: {'vendor': 'amd', 'arch': 'x86_64', 'features': {'AVX2', 'AVX512'}, 'raw': {'flags': {'bmi2', '3dnowprefetch', 'ibpb', 'fsrm', 'hw_pstate', 'vgif', 'skinit', 'avx512_vp2intersect', 'extd_apicid', 'svm_lock', 'vmcb_clean', 'vme', 'amd_lbr_v2', 'bmi1', 'aperfmperf', 'irperf', 'cqm_llc', 'avx2', 'nopl', 'flush_l1d', 'smap', 'avx512ifma', 'rapl', 'avx512dq', 'cppc', 'cpuid', 'avx512cd', 'pae', 'ibs', 'pge', 'mce', 'rdt_a', 'avic', 'ibrs', 'cr8_legacy', 'x2avic', 'apic', 'fpu', 'avx512bw', 'fma', 'osvw', 'cx8', 'cqm_occup_llc', 'vaes', 'wdt', 'ssse3', 'sse', 'rdpru', 'erms', 'rdrand', 'pni', 'vnmi', 'xsave', 'sha_ni', 'aes', 'ht', 'mba', 'tsc_scale', 'avx512_vnni', 'vpclmulqdq', 'mmxext', 'bus_lock_detect', 'mmx', 'xsaves', 'rdpid', 'perfctr_llc', 'pausefilter', 'fxsr', 'decodeassists', 'pat', 'syscall', 'cqm_mbm_total', 'pse36', 'pclmulqdq', 'vmmcall', 'adx', 'rep_good', 'msr', 'clflush', 'ssbd', 'cqm', 'nonstop_tsc', 'abm', 'amd_ppin', 'fsgsbase', 'xsaveopt', 'lbrv', 'movdiri', 'tce', 'smep', 'user_shstk', 'overflow_recov', 'avx512vbmi', 'topoext', 'wbnoinvd', 'tsc', 'constant_tsc', 'mca', 'pse', 'popcnt', 'avx512_vbmi2', 'movdir64b', 'nrip_save', 'f16c', 'sse4_2', 'cat_l3', 'arat', 'xsavec', 'sep', 'tsc_adjust', 'ospke', 'ibrs_enhanced', 'smca', 'cpb', 'invpcid', 'rdseed', 'lahf_lm', 'sse4_1', 'lm', 'avx512_bf16', 'avx512_bitalg', 'sse4a', 'movbe', 'xsaveerptr', 'clzero', 'fxsr_opt', 'v_spec_ctrl', 'sse2', 'xgetbv1', 'extapic', 'pdpe1gb', 'avx', 'v_vmsave_vmload', 'flushbyasid', 'cmp_legacy', 'misalignsse', 'perfctr_core', 'debug_swap', 'umip', 'perfctr_nb', 'cqm_mbm_local', 'stibp', 'avx512f', 'succor', 'la57', 'gfni', 'avx_vnni', 'monitor', 'cmov', 'nx', 'clwb', 'pku', 'clflushopt', 'mtrr', 'perfmon_v2', 'de', 'avx512vl', 'amd_ibpb_ret', 'svm', 'pcid', 'x2apic', 'mwaitx', 'bpext', 'avx512_vpopcntdq', 'npt', 'pfthreshold', 'cdp_l3', 'cx16', 'rdtscp'}}} -- Detected AMD CPU; enabling AMD MoE kernel (-DKTRANSFORMERS_CPU_MOE_AMD=ON) -- Forward CPUINFER_ENABLE_AMX -> -DKTRANSFORMERS_CPU_USE_AMX=OFF -- Enabling AMX/AVX512 umbrella (-DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON) -- Auto-enabling MOE kernel (-DKTRANSFORMERS_CPU_MOE_KERNEL=ON) because CPUINFER_ENABLE_AMD or CPUINFER_ENABLE_KML is ON -- CPU detection: vendor=amd arch=x86_64 features=['AVX2', 'AVX512'] -- Enabling CUDA backend (-DKTRANSFORMERS_USE_CUDA=ON) -- Auto-detected nvcc: /usr/local/cuda/bin/nvcc (adding -DCMAKE_CUDA_COMPILER) -- CMake configure args: -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/0xhaggis/ktransformers/kt-kernel/build/lib.linux-x86_64-cpython-311/ -DPYTHON_EXECUTABLE=/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/bin/python -DCMAKE_BUILD_TYPE=Release -DLLAMA_NATIVE=OFF -DLLAMA_FMA=ON -DLLAMA_F16C=ON -DLLAMA_AVX=ON -DLLAMA_AVX2=ON -DLLAMA_AVX512=ON -DKTRANSFORMERS_CPU_MOE_AMD=ON -DKTRANSFORMERS_CPU_USE_AMX=OFF -DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON -DKTRANSFORMERS_CPU_MOE_KERNEL=ON -DKTRANSFORMERS_USE_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc -- CMake build args: --build . --config Release --parallel 256 [stderr] /home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsWarning: `license` overwritten by `pyproject.toml` corresp(dist, value, root_dir) CMake Warning at CMakeLists.txt:265 (message): pure AVX is not supported at least avx2 CMake Deprecation Warning at /home/0xhaggis/ktransformers/third_party/pybind11/CMakeLists.txt:13 (cmake_minimum_required): Compatibility with CMake < 3.10 will be removed from a future version of CMake. Update the VERSION argument <min> value. Or, use the <min>...<max> syntax to tell CMake that the project requires at least <min> but has been updated to work with policies introduced by <max> or earlier. /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘void decode_cblas_gemm_s8s8s32(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE, KERNEL_CBLAS_TRANSPOSE, KERNEL_CBLAS_OFFSET, size_t, size_t, size_t, float, const void*, size_t, BLASINT8, const void*, size_t, BLASINT8, float, int32_t*, size_t, const int32_t*)’: /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:46:3: error: ‘aocl_gemm_s8s8s32os32’ was not declared in this scope 46 | aocl_gemm_s8s8s32os32(order, op_a, op_b, static_cast<dim_t>(m), static_cast<dim_t>(n), static_cast<dim_t>(k), | ^~~~~~~~~~~~~~~~~~~~~ /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘void prefill_cblas_gemm_s8s8s32(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE, KERNEL_CBLAS_TRANSPOSE, KERNEL_CBLAS_OFFSET, size_t, size_t, size_t, float, const void*, size_t, BLASINT8, const void*, size_t, BLASINT8, float, int32_t*, size_t, const int32_t*)’: /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:61:3: error: ‘aocl_gemm_s8s8s32os32’ was not declared in this scope 61 | aocl_gemm_s8s8s32os32(order, op_a, op_b, static_cast<dim_t>(m), static_cast<dim_t>(n), static_cast<dim_t>(k), | ^~~~~~~~~~~~~~~~~~~~~ /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘void reorder_B_gemm(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE, size_t, size_t, size_t, const void*, void*)’: /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:89:3: error: ‘aocl_reorder_s8s8s32os32’ was not declared in this scope 89 | aocl_reorder_s8s8s32os32(order, op_b, 'B', static_cast<const int8_t*>(b), static_cast<int8_t*>(b_reordered), k, n, | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp: In function ‘size_t get_reorder_B_size(KERNEL_CBLAS_LAYOUT, KERNEL_CBLAS_TRANSPOSE, size_t, size_t)’: /home/0xhaggis/ktransformers/kt-kernel/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp:95:10: error: ‘aocl_get_reorder_buf_size_s8s8s32os32’ was not declared in this scope 95 | return aocl_get_reorder_buf_size_s8s8s32os32(ToAoclOrder(layout), ToAoclTranspose(transb), 'B', k, n); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ gmake[2]: *** [CMakeFiles/kt_kernel_ext.dir/build.make:541: CMakeFiles/kt_kernel_ext.dir/operators/moe_kernel/mat_kernel/aocl_kernel/kernel.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:265: CMakeFiles/kt_kernel_ext.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 Traceback (most recent call last): File "<string>", line 11, in <module> File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 432, in build_wheel return _build(['bdist_wheel']) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 423, in _build return self._build_with_temp_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir self.run_setup() File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup exec(code, locals()) File "<string>", line 436, in <module> File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/__init__.py", line 115, in setup return distutils.core.setup(**attrs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) ^^^^^^^^^^^^^^^^^^ File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command super().run_command(command) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/command/bdist_wheel.py", line 370, in run self.run_command("build") File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command self.distribution.run_command(command) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command super().run_command(command) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command self.distribution.run_command(command) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command super().run_command(command) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "<string>", line 145, in run File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 96, in run _build_ext.run(self) File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run self.build_extensions() File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 484, in build_extensions self._build_extensions_serial() File "/home/0xhaggis/.cache/uv/builds-v0/.tmpHPtRMl/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 510, in _build_extensions_serial self.build_extension(ext) File "<string>", line 412, in build_extension File "/home/0xhaggis/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--parallel', '256']' returned non-zero exit status 2. hint: This usually indicates a problem with the package or the build environment. ```
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@0xhaggis commented on GitHub (Dec 5, 2025):

The solution is to install https://www.amd.com/en/developer/aocl/eula/aocl-5-1-eula.html?filename=aocl-linux-aocc-5.1.0.tar.gz first.

<!-- gh-comment-id:3614917506 --> @0xhaggis commented on GitHub (Dec 5, 2025): The solution is to install https://www.amd.com/en/developer/aocl/eula/aocl-5-1-eula.html?filename=aocl-linux-aocc-5.1.0.tar.gz first.
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@0xhaggis commented on GitHub (Dec 5, 2025):

Actually, no. That gets the project to compile. After compiling I still get OP's error:

> uv pip install -U .
Using Python 3.11.13 environment at: /home/0xhaggis/ktransformers/.venv
Resolved 54 packages in 252ms
      Built kt-kernel @ file:///home/0xhaggis/ktransformers/kt-kernel
Prepared 8 packages in 5.18s
Uninstalled 2 packages in 61ms
Installed 8 packages in 40ms
 + black==25.11.0
 - fsspec==2025.10.0
 + fsspec==2025.12.0
 + kt-kernel==0.1.0 (from file:///home/0xhaggis/ktransformers/kt-kernel)
 + mypy-extensions==1.1.0
 + pathspec==0.12.1
 + platformdirs==4.5.0
 + pytokens==0.3.0
 - transformers==4.57.1
 + transformers==4.57.3
> python -c "from kt_kernel import KTMoEWrapper; print('✓ kt-kernel installed successfully')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel/__init__.py", line 27, in <module>
    from .experts import KTMoEWrapper
  File "/home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel/experts.py", line 17, in <module>
    from .experts_base import BaseMoEWrapper, KExpertsCPUBuffer
  File "/home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel/experts_base.py", line 18, in <module>
    import kt_kernel_ext
ImportError: /home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel_ext.cpython-311-x86_64-linux-gnu.so: undefined symbol: aocl_gemm_s8s8s32os32
<!-- gh-comment-id:3614919656 --> @0xhaggis commented on GitHub (Dec 5, 2025): Actually, no. That gets the project to compile. After compiling I still get OP's error: ``` > uv pip install -U . Using Python 3.11.13 environment at: /home/0xhaggis/ktransformers/.venv Resolved 54 packages in 252ms Built kt-kernel @ file:///home/0xhaggis/ktransformers/kt-kernel Prepared 8 packages in 5.18s Uninstalled 2 packages in 61ms Installed 8 packages in 40ms + black==25.11.0 - fsspec==2025.10.0 + fsspec==2025.12.0 + kt-kernel==0.1.0 (from file:///home/0xhaggis/ktransformers/kt-kernel) + mypy-extensions==1.1.0 + pathspec==0.12.1 + platformdirs==4.5.0 + pytokens==0.3.0 - transformers==4.57.1 + transformers==4.57.3 > python -c "from kt_kernel import KTMoEWrapper; print('✓ kt-kernel installed successfully')" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel/__init__.py", line 27, in <module> from .experts import KTMoEWrapper File "/home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel/experts.py", line 17, in <module> from .experts_base import BaseMoEWrapper, KExpertsCPUBuffer File "/home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel/experts_base.py", line 18, in <module> import kt_kernel_ext ImportError: /home/0xhaggis/ktransformers/.venv/lib/python3.11/site-packages/kt_kernel_ext.cpython-311-x86_64-linux-gnu.so: undefined symbol: aocl_gemm_s8s8s32os32 ```
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@xzy103 commented on GitHub (Dec 5, 2025):

The solution is to install https://www.amd.com/en/developer/aocl/eula/aocl-5-1-eula.html?filename=aocl-linux-aocc-5.1.0.tar.gz first.

是的，的确需要先安装aocl，我也是安装了的，出现 undefined symbol: aocl_gemm_s8s8s32os32 是因为编译时没有link造成的，通过设置环境变量可以不优雅地解决：

Yes, you really do need to install aocl first. I have it installed as well. The undefined symbol: aocl_gemm_s8s8s32os32 error is caused by not linking it during compilation. You can work around this in a rather inelegant way by setting environment variables:

export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so
<!-- gh-comment-id:3615009781 --> @xzy103 commented on GitHub (Dec 5, 2025): > The solution is to install https://www.amd.com/en/developer/aocl/eula/aocl-5-1-eula.html?filename=aocl-linux-aocc-5.1.0.tar.gz first. 是的，的确需要先安装`aocl`，我也是安装了的，出现 `undefined symbol: aocl_gemm_s8s8s32os32` 是因为编译时没有link造成的，通过设置环境变量可以**不优雅**地解决： Yes, you really do need to install `aocl` first. I have it installed as well. The `undefined symbol: aocl_gemm_s8s8s32os32` error is caused by not linking it during compilation. You can work around this in a **rather inelegant** way by setting environment variables: ```bash export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so ```
bsakaguchi commented 2026-10-29 14:48:15 +00:00
Author
Owner
Copy link

@xzy103 commented on GitHub (Dec 5, 2025):

（发现忘star了，已补）

问题成功解决了！在环境变量中加入这行即可：

export CPUINFER_BLIS_ROOT=$AOCL_ROOT

完整的环境变量配置是：

# AOCL是AMD CPU配套的科学计算工具
export AOCL_ROOT=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc
# 运行时库搜索路径
export LD_LIBRARY_PATH=$AOCL_ROOT/lib:$LD_LIBRARY_PATH
# 头文件路径（给 g++/clang++ 用）
export CPLUS_INCLUDE_PATH=$AOCL_ROOT/include:$CPLUS_INCLUDE_PATH
# 库文件路径（给链接器用）
export LIBRARY_PATH=$AOCL_ROOT/lib:$LIBRARY_PATH
# blis的位置 加入这行
export CPUINFER_BLIS_ROOT=$AOCL_ROOT

解决思路

异常提示undefined symbol: aocl_gemm_s8s8s32os32

找到该symbol与这两个文件关联：
/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis-mt.so
/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so

使用export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so不优雅地解决了，说明所需的链接是存在的，只是编译器找不到。

而这两个文件属于blis这个工具，blis是aocl自带的，已经设置了环境变量AOCL_ROOT，按说应该找得到。

于是我查阅了源码，在setup.py中显示环境变量CPUINFER_BLIS_ROOT指向blis工具所在位置。

可我echo $CPUINFER_BLIS_ROOT却为空，说明这个环境变量没有配置。

我的bils在aocl工具集下，因此是同一个目录，所以只需要设置export CPUINFER_BLIS_ROOT=$AOCL_ROOT。

🎉验证通过！

建议

问题主要是不知道环境变量该怎么设置造成的，希望文档中可以补充相关说明。

<!-- gh-comment-id:3617453279 --> @xzy103 commented on GitHub (Dec 5, 2025): _（发现忘star了，已补）_ 问题成功解决了！在环境变量中加入这行即可： ```bash export CPUINFER_BLIS_ROOT=$AOCL_ROOT ``` 完整的环境变量配置是： ```bash # AOCL是AMD CPU配套的科学计算工具 export AOCL_ROOT=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc # 运行时库搜索路径 export LD_LIBRARY_PATH=$AOCL_ROOT/lib:$LD_LIBRARY_PATH # 头文件路径（给 g++/clang++ 用） export CPLUS_INCLUDE_PATH=$AOCL_ROOT/include:$CPLUS_INCLUDE_PATH # 库文件路径（给链接器用） export LIBRARY_PATH=$AOCL_ROOT/lib:$LIBRARY_PATH # blis的位置 加入这行 export CPUINFER_BLIS_ROOT=$AOCL_ROOT ``` --- **解决思路** 异常提示`undefined symbol: aocl_gemm_s8s8s32os32` 找到该symbol与这两个文件关联： `/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis-mt.so` `/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so` 使用`export LD_PRELOAD=/opt/AMD/aocl/aocl-linux-aocc-5.1.0/aocc/lib/libblis.so`不优雅地解决了，说明所需的链接是存在的，只是**编译器找不到**。 而这两个文件属于blis这个工具，blis是aocl自带的，已经设置了环境变量`AOCL_ROOT`，**按说应该找得到**。 于是我查阅了源码，在`setup.py`中显示环境变量`CPUINFER_BLIS_ROOT`指向blis工具所在位置。 可我`echo $CPUINFER_BLIS_ROOT`却为空，说明**这个环境变量没有配置**。 我的bils在aocl工具集下，因此是同一个目录，所以只需要设置`export CPUINFER_BLIS_ROOT=$AOCL_ROOT`。 🎉验证通过！ **建议** 问题主要是不知道环境变量该怎么设置造成的，希望文档中可以补充相关说明。
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#1073
No description provided.