kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1192] [Bug] 在Ubuntu下编译失败 #818

New issue
Closed
opened 2026-10-29 14:44:56 +00:00 by bsakaguchi · 5 comments
bsakaguchi commented 2026-10-29 14:44:56 +00:00
Owner
Copy link

Originally created by @houkunlin on GitHub (Apr 24, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1192

Originally assigned to: @Atream on GitHub.

检查清单

  • 1. 我已经搜索过相关问题，但未能获得预期的帮助
  • 2. 该问题在最新版本中尚未修复
  • 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
  • 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
  • 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述

编译报错

    File "<string>", line 576, in build_extension
    File "<string>", line 357, in run_command_with_live_tail
    File "/home/ubuntu/miniconda3/envs/ktransformers/lib/python3.11/subprocess.py", line 571, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['cmake', '/home/ubuntu/ktransformers/csrc/balance_serve', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ubuntu/ktransformers/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/ubuntu/miniconda3/envs/ktransformers/bin/python', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=1', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4.post1+cu128torch27fancy']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Building wheel for ktransformers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/ubuntu/miniconda3/envs/ktransformers/bin/python /home/ubuntu/miniconda3/envs/ktransformers/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpxx9dly5h
  cwd: /home/ubuntu/ktransformers
  Building wheel for ktransformers (pyproject.toml) ... error

  ERROR: Failed building wheel for ktransformers
Failed to build ktransformers
ERROR: Failed to build installable wheels for some pyproject.toml based projects (ktransformers)

ktransformers编译报错日志.txt

复现步骤

执行 USE_BALANCE_SERVE=1 USE_NUMA=1 bash ./install.sh 编译命令

环境信息

(ktransformers) ubuntu@ubuntu:~/ktransformers$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0
(ktransformers) ubuntu@ubuntu:~/ktransformers$ python ../check_cuda_torch.py
Python版本: 3.11.11 (main, Dec 11 2024, 16:28:39) [GCC 11.2.0]
PyTorch版本: 2.7.0+cu126
CUDA是否可用: 是
CUDA版本: 12.6
支持的CUDA设备数: 8
当前CUDA设备名称: Tesla T4
在GPU上计算的结果: tensor([5., 7., 9.], device='cuda:0')
(ktransformers) ubuntu@ubuntu:~/ktransformers$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
(ktransformers) ubuntu@ubuntu:~/ktransformers$ uname -a
Linux ds02 5.15.0-138-generic #148-Ubuntu SMP Fri Mar 14 19:05:48 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

(ktransformers) ubuntu@ubuntu:~/ktransformers$ env
SHELL=/bin/bash
CONDA_EXE=/home/ubuntu/miniconda3/bin/conda
PWD=/home/ubuntu/ktransformers
LOGNAME=ubuntu
XDG_SESSION_TYPE=tty
CONDA_PREFIX=/home/ubuntu/miniconda3/envs/ktransformers
MOTD_SHOWN=pam
HOME=/home/ubuntu
LANG=en_US.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
CONDA_PROMPT_MODIFIER=(ktransformers)
SSH_CONNECTION=0.0.0.0 64200 0.0.0.0 22
LESSCLOSE=/usr/bin/lesspipe %s %s
XDG_SESSION_CLASS=user
TERM=xterm
LESSOPEN=| /usr/bin/lesspipe %s
USER=ubuntu
CUDA_PATH=/usr/local/cuda
CONDA_SHLVL=2
DISPLAY=localhost:10.0
SHLVL=1
XDG_SESSION_ID=1
CONDA_PYTHON_EXE=/home/ubuntu/miniconda3/bin/python
LD_LIBRARY_PATH=:/usr/local/cuda/lib64
XDG_RUNTIME_DIR=/run/user/1000
SSH_CLIENT=0.0.0.0 64200 22
CONDA_DEFAULT_ENV=ktransformers
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/home/ubuntu/.local/bin:/home/ubuntu/miniconda3/envs/ktransformers/bin:/home/ubuntu/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda/bin
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
SSH_TTY=/dev/pts/0
CONDA_PREFIX_1=/home/ubuntu/miniconda3
OLDPWD=/home/ubuntu
_=/usr/bin/env

(ktransformers) ubuntu@ubuntu:~/ktransformers$ pip list
Package                  Version
------------------------ -----------
accelerate               1.6.0
annotated-types          0.7.0
anyio                    4.9.0
blessed                  1.20.0
blobfile                 3.0.0
build                    1.2.2.post1
certifi                  2025.1.31
charset-normalizer       3.4.1
colorlog                 6.9.0
cpufeature               0.2.1
distro                   1.9.0
fastapi                  0.115.12
filelock                 3.18.0
fire                     0.7.0
fsspec                   2025.3.2
greenlet                 3.2.1
h11                      0.14.0
httpcore                 1.0.8
httpx                    0.28.1
huggingface-hub          0.30.2
idna                     3.10
Jinja2                   3.1.6
jiter                    0.9.0
jsonpatch                1.33
jsonpointer              3.0.0
langchain                0.3.24
langchain-core           0.3.55
langchain-text-splitters 0.3.8
langsmith                0.3.33
lxml                     5.4.0
MarkupSafe               3.0.2
mpmath                   1.3.0
networkx                 3.4.2
ninja                    1.11.1.4
numpy                    2.2.5
nvidia-cublas-cu12       12.6.4.1
nvidia-cuda-cupti-cu12   12.6.80
nvidia-cuda-nvrtc-cu12   12.6.77
nvidia-cuda-runtime-cu12 12.6.77
nvidia-cudnn-cu12        9.5.1.17
nvidia-cufft-cu12        11.3.0.4
nvidia-cufile-cu12       1.11.1.6
nvidia-curand-cu12       10.3.7.77
nvidia-cusolver-cu12     11.7.1.2
nvidia-cusparse-cu12     12.5.4.2
nvidia-cusparselt-cu12   0.6.3
nvidia-nccl-cu12         2.26.2
nvidia-nvjitlink-cu12    12.6.85
nvidia-nvtx-cu12         12.6.77
openai                   1.76.0
orjson                   3.10.16
packaging                24.2
pillow                   11.2.1
pip                      25.0
protobuf                 6.30.2
psutil                   7.0.0
pycryptodomex            3.22.0
pydantic                 2.11.3
pydantic_core            2.33.1
pyproject_hooks          1.2.0
PyYAML                   6.0.2
pyzmq                    26.4.0
regex                    2024.11.6
requests                 2.32.3
requests-toolbelt        1.0.0
safetensors              0.5.3
sentencepiece            0.2.0
setuptools               75.8.0
six                      1.17.0
sniffio                  1.3.1
SQLAlchemy               2.0.40
starlette                0.46.2
sympy                    1.13.3
tenacity                 9.1.2
termcolor                3.0.1
tiktoken                 0.9.0
tokenizers               0.19.1
torch                    2.7.0
torchaudio               2.7.0
torchvision              0.22.0
tqdm                     4.67.1
transformers             4.43.2
triton                   3.3.0
typing_extensions        4.13.2
typing-inspection        0.4.0
urllib3                  2.4.0
wcwidth                  0.2.13
wheel                    0.45.1
zmq                      0.0.0
zstandard                0.23.0
Originally created by @houkunlin on GitHub (Apr 24, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1192 Originally assigned to: @Atream on GitHub. ### 检查清单 - [x] 1. 我已经搜索过相关问题，但未能获得预期的帮助 - [x] 2. 该问题在最新版本中尚未修复 - [x] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性 - [x] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [x] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭 ### 问题描述 编译报错 ``` File "<string>", line 576, in build_extension File "<string>", line 357, in run_command_with_live_tail File "/home/ubuntu/miniconda3/envs/ktransformers/lib/python3.11/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['cmake', '/home/ubuntu/ktransformers/csrc/balance_serve', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ubuntu/ktransformers/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/ubuntu/miniconda3/envs/ktransformers/bin/python', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-D_GLIBCXX_USE_CXX11_ABI=1', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4.post1+cu128torch27fancy']' returned non-zero exit status 1. error: subprocess-exited-with-error × Building wheel for ktransformers (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. full command: /home/ubuntu/miniconda3/envs/ktransformers/bin/python /home/ubuntu/miniconda3/envs/ktransformers/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpxx9dly5h cwd: /home/ubuntu/ktransformers Building wheel for ktransformers (pyproject.toml) ... error ERROR: Failed building wheel for ktransformers Failed to build ktransformers ERROR: Failed to build installable wheels for some pyproject.toml based projects (ktransformers) ``` [ktransformers编译报错日志.txt](https://github.com/user-attachments/files/19884198/ktransformers.txt) ### 复现步骤 执行 `USE_BALANCE_SERVE=1 USE_NUMA=1 bash ./install.sh` 编译命令 ### 环境信息 ``` (ktransformers) ubuntu@ubuntu:~/ktransformers$ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Fri_Feb_21_20:23:50_PST_2025 Cuda compilation tools, release 12.8, V12.8.93 Build cuda_12.8.r12.8/compiler.35583870_0 (ktransformers) ubuntu@ubuntu:~/ktransformers$ python ../check_cuda_torch.py Python版本: 3.11.11 (main, Dec 11 2024, 16:28:39) [GCC 11.2.0] PyTorch版本: 2.7.0+cu126 CUDA是否可用: 是 CUDA版本: 12.6 支持的CUDA设备数: 8 当前CUDA设备名称: Tesla T4 在GPU上计算的结果: tensor([5., 7., 9.], device='cuda:0') (ktransformers) ubuntu@ubuntu:~/ktransformers$ cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.5 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.5 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy (ktransformers) ubuntu@ubuntu:~/ktransformers$ uname -a Linux ds02 5.15.0-138-generic #148-Ubuntu SMP Fri Mar 14 19:05:48 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux ``` ``` (ktransformers) ubuntu@ubuntu:~/ktransformers$ env SHELL=/bin/bash CONDA_EXE=/home/ubuntu/miniconda3/bin/conda PWD=/home/ubuntu/ktransformers LOGNAME=ubuntu XDG_SESSION_TYPE=tty CONDA_PREFIX=/home/ubuntu/miniconda3/envs/ktransformers MOTD_SHOWN=pam HOME=/home/ubuntu LANG=en_US.UTF-8 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: CONDA_PROMPT_MODIFIER=(ktransformers) SSH_CONNECTION=0.0.0.0 64200 0.0.0.0 22 LESSCLOSE=/usr/bin/lesspipe %s %s XDG_SESSION_CLASS=user TERM=xterm LESSOPEN=| /usr/bin/lesspipe %s USER=ubuntu CUDA_PATH=/usr/local/cuda CONDA_SHLVL=2 DISPLAY=localhost:10.0 SHLVL=1 XDG_SESSION_ID=1 CONDA_PYTHON_EXE=/home/ubuntu/miniconda3/bin/python LD_LIBRARY_PATH=:/usr/local/cuda/lib64 XDG_RUNTIME_DIR=/run/user/1000 SSH_CLIENT=0.0.0.0 64200 22 CONDA_DEFAULT_ENV=ktransformers XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop PATH=/home/ubuntu/.local/bin:/home/ubuntu/miniconda3/envs/ktransformers/bin:/home/ubuntu/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda/bin DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus SSH_TTY=/dev/pts/0 CONDA_PREFIX_1=/home/ubuntu/miniconda3 OLDPWD=/home/ubuntu _=/usr/bin/env ``` ``` (ktransformers) ubuntu@ubuntu:~/ktransformers$ pip list Package Version ------------------------ ----------- accelerate 1.6.0 annotated-types 0.7.0 anyio 4.9.0 blessed 1.20.0 blobfile 3.0.0 build 1.2.2.post1 certifi 2025.1.31 charset-normalizer 3.4.1 colorlog 6.9.0 cpufeature 0.2.1 distro 1.9.0 fastapi 0.115.12 filelock 3.18.0 fire 0.7.0 fsspec 2025.3.2 greenlet 3.2.1 h11 0.14.0 httpcore 1.0.8 httpx 0.28.1 huggingface-hub 0.30.2 idna 3.10 Jinja2 3.1.6 jiter 0.9.0 jsonpatch 1.33 jsonpointer 3.0.0 langchain 0.3.24 langchain-core 0.3.55 langchain-text-splitters 0.3.8 langsmith 0.3.33 lxml 5.4.0 MarkupSafe 3.0.2 mpmath 1.3.0 networkx 3.4.2 ninja 1.11.1.4 numpy 2.2.5 nvidia-cublas-cu12 12.6.4.1 nvidia-cuda-cupti-cu12 12.6.80 nvidia-cuda-nvrtc-cu12 12.6.77 nvidia-cuda-runtime-cu12 12.6.77 nvidia-cudnn-cu12 9.5.1.17 nvidia-cufft-cu12 11.3.0.4 nvidia-cufile-cu12 1.11.1.6 nvidia-curand-cu12 10.3.7.77 nvidia-cusolver-cu12 11.7.1.2 nvidia-cusparse-cu12 12.5.4.2 nvidia-cusparselt-cu12 0.6.3 nvidia-nccl-cu12 2.26.2 nvidia-nvjitlink-cu12 12.6.85 nvidia-nvtx-cu12 12.6.77 openai 1.76.0 orjson 3.10.16 packaging 24.2 pillow 11.2.1 pip 25.0 protobuf 6.30.2 psutil 7.0.0 pycryptodomex 3.22.0 pydantic 2.11.3 pydantic_core 2.33.1 pyproject_hooks 1.2.0 PyYAML 6.0.2 pyzmq 26.4.0 regex 2024.11.6 requests 2.32.3 requests-toolbelt 1.0.0 safetensors 0.5.3 sentencepiece 0.2.0 setuptools 75.8.0 six 1.17.0 sniffio 1.3.1 SQLAlchemy 2.0.40 starlette 0.46.2 sympy 1.13.3 tenacity 9.1.2 termcolor 3.0.1 tiktoken 0.9.0 tokenizers 0.19.1 torch 2.7.0 torchaudio 2.7.0 torchvision 0.22.0 tqdm 4.67.1 transformers 4.43.2 triton 3.3.0 typing_extensions 4.13.2 typing-inspection 0.4.0 urllib3 2.4.0 wcwidth 0.2.13 wheel 0.45.1 zmq 0.0.0 zstandard 0.23.0 ```
bsakaguchi commented 2026-10-29 14:44:56 +00:00
Author
Owner
Copy link

@houkunlin commented on GitHub (Apr 24, 2025):

ktransformers编译报错日志.txt

<!-- gh-comment-id:2826313913 --> @houkunlin commented on GitHub (Apr 24, 2025): [ktransformers编译报错日志.txt](https://github.com/user-attachments/files/19884204/ktransformers.txt)
bsakaguchi commented 2026-10-29 14:44:56 +00:00
Author
Owner
Copy link

@houkunlin commented on GitHub (Apr 24, 2025):

操作步骤是根据 https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md 这个文件的步骤来做的

<!-- gh-comment-id:2826318663 --> @houkunlin commented on GitHub (Apr 24, 2025): 操作步骤是根据 https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md 这个文件的步骤来做的
bsakaguchi commented 2026-10-29 14:44:56 +00:00
Author
Owner
Copy link

@jizhilong commented on GitHub (Apr 24, 2025):

CMake Error in /home/ubuntu/ktransformers/csrc/balance_serve/build/CMakeFiles/CMakeTmp/CMakeLists.txt:
Target "cmTC_23cd7" requires the language dialect "CUDA20" (with compiler
extensions), but CMake does not know the compile flags to use to enable it.

看这个报错，感觉可能和 cmake 版本有关。参考： https://stackoverflow.com/questions/75010206/cuda-12-c20-support-with-cmake-not-working

<!-- gh-comment-id:2827698529 --> @jizhilong commented on GitHub (Apr 24, 2025): > CMake Error in /home/ubuntu/ktransformers/csrc/balance_serve/build/CMakeFiles/CMakeTmp/CMakeLists.txt: > Target "cmTC_23cd7" requires the language dialect "CUDA20" (with compiler > extensions), but CMake does not know the compile flags to use to enable it. 看这个报错，感觉可能和 cmake 版本有关。参考： https://stackoverflow.com/questions/75010206/cuda-12-c20-support-with-cmake-not-working
bsakaguchi commented 2026-10-29 14:44:57 +00:00
Author
Owner
Copy link

@houkunlin commented on GitHub (Apr 25, 2025):

CMake Error in /home/ubuntu/ktransformers/csrc/balance_serve/build/CMakeFiles/CMakeTmp/CMakeLists.txt:
Target "cmTC_23cd7" requires the language dialect "CUDA20" (with compiler
extensions), but CMake does not know the compile flags to use to enable it.

看这个报错，感觉可能和 cmake 版本有关。参考： https://stackoverflow.com/questions/75010206/cuda-12-c20-support-with-cmake-not-working

确实是 CMake 问题，

git clone https://gitlab.kitware.com/cmake/cmake
cd cmake
git checkout release
./configure
make -j32
make install

需要注意：在普通用户下 make install 然后执行 cmake 发现报了CMake Error: Could not find CMAKE_ROOT的错误。
这个错误的原因是在普通用户下执行sudo make install 后，会将二进制安装到/usr/local/bin下，而CMAKE_ROOT依赖的文件安装在/usr/local/share/cmake-3.14
参考链接 https://www.cnblogs.com/zongfanstudy/p/18153439

升级 CMake 版本后编译正常，但是又遇到另外一个问题，这个问题 #425 已经提及

<!-- gh-comment-id:2829147655 --> @houkunlin commented on GitHub (Apr 25, 2025): > > CMake Error in /home/ubuntu/ktransformers/csrc/balance_serve/build/CMakeFiles/CMakeTmp/CMakeLists.txt: > > Target "cmTC_23cd7" requires the language dialect "CUDA20" (with compiler > > extensions), but CMake does not know the compile flags to use to enable it. > > 看这个报错，感觉可能和 cmake 版本有关。参考： https://stackoverflow.com/questions/75010206/cuda-12-c20-support-with-cmake-not-working 确实是 CMake 问题， ``` git clone https://gitlab.kitware.com/cmake/cmake cd cmake git checkout release ./configure make -j32 make install ``` > 需要注意：在普通用户下 `make install` 然后执行 cmake 发现报了CMake Error: Could not find CMAKE_ROOT的错误。 > 这个错误的原因是在普通用户下执行sudo make install 后，会将二进制安装到/usr/local/bin下，而CMAKE_ROOT依赖的文件安装在/usr/local/share/cmake-3.14 > 参考链接 https://www.cnblogs.com/zongfanstudy/p/18153439 升级 CMake 版本后编译正常，但是又遇到另外一个问题，这个问题 #425 已经提及
bsakaguchi commented 2026-10-29 14:44:57 +00:00
Author
Owner
Copy link

@houkunlin commented on GitHub (Apr 27, 2025):

已成功编译并运行，发一下我的环境信息：

gcc --version
gcc (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


g++ --version
g++ (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


cmake --version
cmake version 4.0.1-gf4d41e7
CMake suite maintained and supported by Kitware (kitware.com/cmake).


python check_cuda_torch.py
Python版本: 3.11.11 (main, Dec 11 2024, 16:28:39) [GCC 11.2.0]
PyTorch版本: 2.7.0+cu126
CUDA是否可用: 是
CUDA版本: 12.6
支持的CUDA设备数: 8
当前CUDA设备名称: Tesla T4


nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0


nvidia-smi
Sun Apr 27 01:02:01 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.05              Driver Version: 560.35.05      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+

关于 Tesla T4 GPU 可能运行模型失败问题，能正常加载，但是只要一发起请求就直接报错，可以参考 #425 #374 #712 这几个 issues ，我是参考这几个来处理的，并且成功在 Tesla T4 GPU上运行了 DeepSeek-R1-GGUF-Q6_K / DeepSeek-R1-GGUF-Q4_K_M / DeepSeek-V2-Lite-GGUF-Q4_K_M / DeepSeek-V2-Lite-GGUF-Q8_0 / DeepSeek-V3-GGUF-Q4_K_M 这几个模型但就是速度上仍旧不太理想。

其中 DeepSeek-R1-GGUF-Q6_K 模型启动完毕后，内存占用950G+

其中 DeepSeek-V3-GGUF-Q4_K_M 模型启动完毕后，内存占用 840G+

如果硬件内存不够，可以尝试多分一些 swap 虚拟内存来保证模型加载成功，权当是尝鲜使用，或者做方案验证什么的，我的就是物理内存 32G*12 = 384G，但 Linux 上 free -h 看到的实际内存只有 377G ，完全是没办法加载起 DeepSeek-R1-GGUF-Q6_K / DeepSeek-R1-GGUF-Q4_K_M / DeepSeek-V3-GGUF-Q4_K_M 这些模型的，也是通过分配虚拟内存才加载起来的

<!-- gh-comment-id:2832853635 --> @houkunlin commented on GitHub (Apr 27, 2025): 已成功编译并运行，发一下我的环境信息： ``` gcc --version gcc (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. g++ --version g++ (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cmake --version cmake version 4.0.1-gf4d41e7 CMake suite maintained and supported by Kitware (kitware.com/cmake). python check_cuda_torch.py Python版本: 3.11.11 (main, Dec 11 2024, 16:28:39) [GCC 11.2.0] PyTorch版本: 2.7.0+cu126 CUDA是否可用: 是 CUDA版本: 12.6 支持的CUDA设备数: 8 当前CUDA设备名称: Tesla T4 nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Fri_Feb_21_20:23:50_PST_2025 Cuda compilation tools, release 12.8, V12.8.93 Build cuda_12.8.r12.8/compiler.35583870_0 nvidia-smi Sun Apr 27 01:02:01 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 560.35.05 Driver Version: 560.35.05 CUDA Version: 12.6 | |-----------------------------------------+------------------------+----------------------+ ``` 关于 `Tesla T4` GPU 可能运行模型失败问题，能正常加载，但是只要一发起请求就直接报错，可以参考 #425 #374 #712 这几个 issues ，我是参考这几个来处理的，并且成功在 `Tesla T4` GPU上运行了 `DeepSeek-R1-GGUF-Q6_K` / `DeepSeek-R1-GGUF-Q4_K_M` / `DeepSeek-V2-Lite-GGUF-Q4_K_M` / `DeepSeek-V2-Lite-GGUF-Q8_0` / `DeepSeek-V3-GGUF-Q4_K_M` 这几个模型但就是速度上仍旧不太理想。 其中 `DeepSeek-R1-GGUF-Q6_K` 模型启动完毕后，内存占用950G+ 其中 `DeepSeek-V3-GGUF-Q4_K_M` 模型启动完毕后，内存占用 840G+ 如果硬件内存不够，可以尝试多分一些 swap 虚拟内存来保证模型加载成功，权当是尝鲜使用，或者做方案验证什么的，我的就是物理内存 32G*12 = 384G，但 Linux 上 `free -h` 看到的实际内存只有 377G ，完全是没办法加载起 `DeepSeek-R1-GGUF-Q6_K` / `DeepSeek-R1-GGUF-Q4_K_M` / `DeepSeek-V3-GGUF-Q4_K_M` 这些模型的，也是通过分配虚拟内存才加载起来的
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#818
No description provided.