mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1017] [Bug] No module named 'sched_ext' #718
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#718
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Colacalc on GitHub (Apr 2, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1017
检查清单
问题描述
安装过程遇到了这个问题，其他部分在编译库时有错误。
复现步骤
bash ./start.sh -m ~/Desktop/model/DeepSeek-V3-0324-GGUF-Q4_K_M/config/ -g ~/Desktop/model/DeepSeek-V3-0324-GGUF-Q4_K_M/ -c 16 -t 8192 -l 2048
环境信息
ubuntu22.04 ./run.sh 安装
@orca-zhang commented on GitHub (Apr 2, 2025):
+1
@Warshmao commented on GitHub (Apr 2, 2025):
+1
@Apanoff commented on GitHub (Apr 2, 2025):
+1
@knilink commented on GitHub (Apr 2, 2025):
github.com/kvcache-ai/ktransformers@795524cacc/setup.py (L39)好像要构建时要设置
USE_BALANCE_SERVE=1来构建balancer而且balancer的CMake可能要根据自身的环境改动
github.com/kvcache-ai/ktransformers@795524cacc/csrc/balance_serve/CMakeLists.txt (L3)github.com/kvcache-ai/ktransformers@795524cacc/csrc/balance_serve/CMakeLists.txt (L55)CMake中需要某些命令带后缀
g++-13和python3但我的环境里就没后缀有比如g++，python@pitaozi commented on GitHub (Apr 2, 2025):
@createthis commented on GitHub (Apr 2, 2025):
+1
I also see this at the end of the build log:
@olproff commented on GitHub (Apr 2, 2025):
+1
Hello, i have the same issue:
run ktransformers and there is i have that issue:
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True ktransformers --gguf_path ~/deepseek/unsloth/DeepSeek-R1-GGUF/DeepSeek-R1-UD-Q2_K_XL/ --model_path deepseek-ai/DeepSeek-R1 --model_name unsloth/DeepSeek-R1-UD-Q2_K_XL --cpu_infer 30 --max_new_tokens 8192 --cache_lens 32768 --total_context 32768 --cache_q4 true --temperature 0.6 --top_p 0.95 --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu.yaml --force_think --use_cuda_graph --port 10002 --web True
Traceback (most recent call last):
File "/usr/local/anaconda3/envs/ktransformers/bin/ktransformers", line 5, in
from ktransformers.server.main import main
File "/usr/local/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/main.py", line 11, in
from ktransformers.server.args import ArgumentParser
File "/usr/local/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/args.py", line 3, in
from ktransformers.util.utils import get_free_ports
File "/usr/local/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/util/utils.py", line 17, in
from ktransformers.models.custom_cache import StaticCache
File "/usr/local/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/models/custom_cache.py", line 15, in
from ktransformers.server.balance_serve.settings import sched_ext
File "/usr/local/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/balance_serve/settings.py", line 13, in
import sched_ext
ModuleNotFoundError: No module named 'sched_ext'
@createthis commented on GitHub (Apr 3, 2025):
I think this may be a documentation issue. This documentation: https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md#3-build-ktransformers
... does not match this documentation: https://kvcache-ai.github.io/ktransformers/en/install.html
@jaspers1203 commented on GitHub (Apr 3, 2025):
同样的错误，环境变量设置USE_BALANCE_SERVE=1后编译很多次都是下面的错误。
如果不加则
[3/3] /usr/local/cuda-12.4/bin/nvcc --generate-dependencies-with-compile --dependency-output /data/project/KTransformer/src/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/gptq_marlin/gptq_marlin.o.d -I/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/torch/include -I/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/torch/include/TH -I/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/torch/include/THC -I/usr/local/cuda-12.4/include -I/root/miniconda3/envs/ktransformer/include/python3.11 -c -c /data/project/KTransformer/src/csrc/custom_marlin/gptq_marlin/gptq_marlin.cu -o /data/project/KTransformer/src/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/gptq_marlin/gptq_marlin.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -Xcompiler -fPIC -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=vLLMMarlin -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_89,code=compute_89 -gencode=arch=compute_89,code=sm_89 -std=c++17
g++ -pthread -B /root/miniconda3/envs/ktransformer/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/envs/ktransformer/include -fPIC -O2 -isystem /root/miniconda3/envs/ktransformer/include -pthread -B /root/miniconda3/envs/ktransformer/compiler_compat -shared /data/project/KTransformer/src/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/binding.o /data/project/KTransformer/src/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/gptq_marlin/gptq_marlin.o /data/project/KTransformer/src/build/temp.linux-x86_64-cpython-311/csrc/custom_marlin/gptq_marlin/gptq_marlin_repack.o -L/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/torch/lib -L/usr/local/cuda-12.4/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-311/vLLMMarlin.cpython-311-x86_64-linux-gnu.so
CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/project/KTransformer/src/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/root/miniconda3/envs/ktransformer/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON']
CMake args: ['-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/project/KTransformer/src/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/root/miniconda3/envs/ktransformer/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4+cu124torch26fancy']
build_temp: /data/project/KTransformer/src/csrc/balance_serve/build
Traceback (most recent call last):
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in
main()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
return _build_backend().build_wheel(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/build_meta.py", line 438, in build_wheel
return _build(['bdist_wheel', '--dist-info-dir', str(metadata_directory)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/build_meta.py", line 426, in _build
return self._build_with_temp_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
self.run_setup()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
exec(code, locals())
File "", line 468, in
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/init.py", line 117, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 186, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
dist.run_commands()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 983, in run_commands
self.run_command(cmd)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
super().run_command(command)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
cmd_obj.run()
File "", line 245, in run
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/command/bdist_wheel.py", line 379, in run
self.run_command("build")
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
self.distribution.run_command(command)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
super().run_command(command)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
cmd_obj.run()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 136, in run
self.run_command(cmd_name)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
self.distribution.run_command(command)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
super().run_command(command)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
cmd_obj.run()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 99, in run
_build_ext.run(self)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 365, in run
self.build_extensions()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 900, in build_extensions
build_ext.build_extensions(self)
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 481, in build_extensions
self._build_extensions_serial()
File "/root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 507, in _build_extensions_serial
self.build_extension(ext)
File "", line 396, in build_extension
File "/root/miniconda3/envs/ktransformer/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cmake', '/data/project/KTransformer/src/csrc/balance_serve', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/project/KTransformer/src/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/root/miniconda3/envs/ktransformer/bin/python3.11', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4+cu124torch26fancy']' returned non-zero exit status 1.
error: subprocess-exited-with-error
× Building wheel for ktransformers (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /root/miniconda3/envs/ktransformer/bin/python3.11 /root/miniconda3/envs/ktransformer/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpuuffa5lo
cwd: /data/project/KTransformer/src
Building wheel for ktransformers (pyproject.toml) ... error
ERROR: Failed building wheel for ktransformers
Failed to build ktransformers
ERROR: Failed to build installable wheels for some pyproject.toml based projects (ktransformers)
@WingsLong commented on GitHub (Apr 3, 2025):
一样的错误，请求解决了吗！不知道啥问题
@orca-zhang commented on GitHub (Apr 3, 2025):
git submodule update --init --recursive
apt install g++-13
然后：
USE_BALANCE_SERVE=1 USE_NUMA=1 sh install.sh
@Joluck commented on GitHub (Apr 3, 2025):
same issue, have you found a solution?
@NUSTHR commented on GitHub (Apr 3, 2025):
same issue, anyone can provide some advice?
@jaspers1203 commented on GitHub (Apr 3, 2025):
升级cmake到3.29.0
修改CMakeLists.txt，添加：
set(CMAKE_CUDA_STANDARD 20)
set(CMAKE_CUDA_STANDARD_REQUIRED ON)
可以成功编译并生成libprometheus-cpp-.so等文件：
ll csrc/balance_serve/build/third_party/prometheus-cpp/lib
total 480
drwxr-xr-x 2 root root 4096 Apr 3 10:46 ./
drwxr-xr-x 8 root root 4096 Apr 3 11:22 ../
lrwxrwxrwx 1 root root 29 Apr 3 10:46 libprometheus-cpp-core.so -> libprometheus-cpp-core.so.1.3*
lrwxrwxrwx 1 root root 31 Apr 3 10:46 libprometheus-cpp-core.so.1.3 -> libprometheus-cpp-core.so.1.3.0*
-rwxr-xr-x 1 root root 221928 Apr 3 10:46 libprometheus-cpp-core.so.1.3.0*
lrwxrwxrwx 1 root root 29 Apr 3 10:46 libprometheus-cpp-pull.so -> libprometheus-cpp-pull.so.1.3*
lrwxrwxrwx 1 root root 31 Apr 3 10:46 libprometheus-cpp-pull.so.1.3 -> libprometheus-cpp-pull.so.1.3.0*
-rwxr-xr-x 1 root root 254960 Apr 3 10:46 libprometheus-cpp-pull.so.1.3.0*
不过按照文档启动后如下面信息就不动了：
python ./src/ktransformers/server/main.py
--port 9994
--model_path deepseek-ai/DeepSeek-R1
--gguf_path ./DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M
--optimize_config_path ./src/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu-4.yaml
--max_new_tokens 8192
--cache_lens 32768
--chunk_size 256
--max_batch_size 4
--backend_type balance_serve
2025-04-03 15:37:14,906 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
sched_rpc started with PID: 836934
set start method
Connected to server at tcp://localhost:48245
2025-04-03 15:37:19,423 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
start method already set to spawn
Connected to server at tcp://localhost:48245
Getting inference context from sched_client.
@Joluck commented on GitHub (Apr 3, 2025):
cuda一定要12.4吗？目前12.8 总是cmake报错
@NUSTHR commented on GitHub (Apr 3, 2025):
do you succeed in running it?
@NUSTHR commented on GitHub (Apr 3, 2025):
hey bro, could you access the model now?
@git-shan commented on GitHub (Apr 3, 2025):
File "/home/mtg/miniconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/balance_serve/settings.py", line 13, in
import sched_ext
ModuleNotFoundError: No module named 'sched_ext'
@git-shan commented on GitHub (Apr 3, 2025):
subprocess.CalledProcessError: Command '['cmake', '/home/deepseek/mktr/ktransformers/csrc/ktransformers_ext', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/deepseek/mktr/ktransformers/build/lib.linux-x86_64-cpython-311/', '-DPYTHON_EXECUTABLE=/home/mtg/miniconda3/envs/ktransformers/bin/python', '-DCMAKE_BUILD_TYPE=Release', '-DKTRANSFORMERS_USE_CUDA=ON', '-DLLAMA_NATIVE=ON', '-DEXAMPLE_VERSION_INFO=0.2.4+cu128torch26fancy']' returned non-zero exit status 1.
error: subprocess-exited-with-error
@git-shan commented on GitHub (Apr 3, 2025):
issue on cmake csrc/balance_serve, CUDA 12.8 does not work
@olproff commented on GitHub (Apr 3, 2025):
same issue
@devnen commented on GitHub (Apr 3, 2025):
Same issue as everyone else. Here is my command:
python ktransformers/server/main.py
--model_path deepseek-ai/DeepSeek-V3-0324
--gguf_path /home/user/models_gguf/DeepSeek-V3-0324-UD-Q2_K_XL
--cpu_infer 30
--port 10002
--web True
Here is the error:
Traceback (most recent call last):
File "/home/user/deepseek/ktransformers/ktransformers/server/main.py", line 11, in
from ktransformers.server.args import ArgumentParser
File "/home/user/deepseek/venv/lib/python3.10/site-packages/ktransformers/server/args.py", line 3, in
from ktransformers.util.utils import get_free_ports
File "/home/user/deepseek/venv/lib/python3.10/site-packages/ktransformers/util/utils.py", line 17, in
from ktransformers.models.custom_cache import StaticCache
File "/home/user/deepseek/venv/lib/python3.10/site-packages/ktransformers/models/custom_cache.py", line 15, in
from ktransformers.server.balance_serve.settings import sched_ext
File "/home/user/deepseek/venv/lib/python3.10/site-packages/ktransformers/server/balance_serve/settings.py", line 13, in
import sched_ext
ModuleNotFoundError: No module named 'sched_ext'
I have tried compiling with and without "balance serve" option:
USE_BALANCE_SERVE=1 bash ./install.sh
and
bash ./install.sh
@CYSTEV-chn commented on GitHub (Apr 4, 2025):
change your command as :
python ktransformers/server/main.py --model_path deepseek-ai/DeepSeek-V3 --gguf_path /home/user/models_gguf/DeepSeek-V3 --cpu_infer 30 --port 10002 --web Trueand rename your folds name as DeepSeek-V3. try it.
@bboowx commented on GitHub (Apr 4, 2025):
(ktransformers) wangjian@tufai:~$ python -m ktransformers.local_chat --model_path /home/wangjian/models/R1 --gguf_path /home/wangjian/models/R1-GGUF/DeepSeek-R1-UD-Q2_K_XL 2.51 --optimize_config_path /home/wangjian/models/Chat.yaml –max_new_tokens 4096 --cpu_infer 16 --model_name DeepSeek-R1
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/wangjian/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/local_chat.py", line 25, in
from ktransformers.optimize.optimize import optimize_and_load_gguf
File "/home/wangjian/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/optimize/optimize.py", line 16, in
from ktransformers.util.utils import set_module, load_weights
File "/home/wangjian/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/util/utils.py", line 17, in
from ktransformers.models.custom_cache import StaticCache
File "/home/wangjian/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/models/custom_cache.py", line 15, in
from ktransformers.server.balance_serve.settings import sched_ext
File "/home/wangjian/anaconda3/envs/ktransformers/lib/python3.11/site-packages/ktransformers/server/balance_serve/settings.py", line 13, in
import sched_ext
ModuleNotFoundError: No module named 'sched_ext'
运行模型一样报错~~
@devnen commented on GitHub (Apr 4, 2025):
Thank you for responding. I tried changing the command, but unfortunately, that generated the same error.
I've managed to get it running by modifying the Python code in a few places.
I am now trying to construct the yaml and the command line to take advantage of 192GB vRAM (8 x RTX 3090) and 192GB DDR5 RAM. The goal is maximum tokens per second. I'm open to any suggestions. I am getting 1.6 tokens/sec. with the standard 8 GPUs YAML settings file and a DeepSeek-V3-0324-UD-IQ2_XXS quant from Unsloth.
python /home/user/deepseek/ktransformers/ktransformers/server/main.py
--model_path deepseek-ai/DeepSeek-V3
--gguf_path /home/user/models_gguf/DeepSeek-V3
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu-8.yaml
--cpu_infer 30
--port 10002
--backend_type ktransformers
--force_think
Here's what I have changed:
In
~/deepseek/venv/lib/python3.10/site-packages/ktransformers/server/balance_serve/settings.py:import sched_extstatement:In
~/deepseek/venv/lib/python3.10/site-packages/ktransformers/models/custom_cache.py:Removed the
--web Trueflag from the command since the web interface files were missingThe root issue was a missing C extension module (
sched_ext) that's required for the balance_serve functionality.@CYSTEV-chn commented on GitHub (Apr 4, 2025):
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu-8.yaml
this command is used for KT0.2.3 ,
if you sucessfully installl KT0.2.4 ,
you should use : DeepSeek-V3-Chat-fp8-linear-ggml-experts-serve.yaml
or
DeepSeek-V3-Chat-serve.yaml
the two yaml all [add balance-serve]
you can check origin code Last commit message : https://github.com/kvcache-ai/ktransformers/tree/main/ktransformers/optimize/optimize_rules
i think this maybe the reason of error.
i once the same error as you.
@johndaily23 commented on GitHub (Apr 7, 2025):
I got it running successfully by modifying from CXX11_ABI=0 to CXX11_ABI=1 in multiple cmake and .make files (search the whole codebase), and don't forget to set USE_BALANCE_SERVE=1 env before make
@NUSTHR commented on GitHub (Apr 7, 2025):
Hey bro. Did you revise all the CMake files? or just some of them?
@KathySkyer commented on GitHub (Apr 7, 2025):
I got same issue, did you try it?and succed?
@lsm03624 commented on GitHub (Apr 8, 2025):
解决了一个个坑，又来一个坑，bug太多了。
@AlbertG123 commented on GitHub (Apr 12, 2025):
用[devnen] 的方法，确实可以。把需要用到sched_ext的地方都拿掉.
bash ./install.sh 就可以build pass.
执行python ktransformers.local_chat --model_path deepseek-ai/DeepSeek-V2-Lite-Chat --gguf_path ./DeepSeek-V2-Lite-Chat-GGUF 可以pass. 可以正常chat.
@alfiy commented on GitHub (Apr 22, 2025):
我目前已经成功配置好ktransformers环境，但是运行模型时又出现，No module named 'sched_ext'错误。
@imjamespond commented on GitHub (May 5, 2025):
wsl ubuntu22 py311 simple install.sh同报这个错
NameError: name 'sched_ext' is not defined@chengcheng84 commented on GitHub (May 10, 2025):
@TTThanos commented on GitHub (May 12, 2025):
我遇到了同样的问题
I have the same problem as above
@AlbertG123 commented on GitHub (May 12, 2025):
0.24版本可以用这个办法解决，升级到0.3版本之后，就不行了.
@hahali-li commented on GitHub (May 13, 2025):
Is there a solution for the issues that occurred in version v0.3?
@kuku-jiusgan commented on GitHub (May 14, 2025):
no balance_serve
Traceback (most recent call last):
File "/home/zoutengda/KT-conda/ktransformers/ktransformers/server/main.py", line 10, in
from ktransformers.server.args import ArgumentParser
File "/home/zoutengda/anaconda3/envs/kt0512/lib/python3.11/site-packages/ktransformers/server/args.py", line 3, in
from ktransformers.util.utils import get_free_ports
File "/home/zoutengda/anaconda3/envs/kt0512/lib/python3.11/site-packages/ktransformers/util/utils.py", line 27, in
from ktransformers.models.custom_cache import StaticCache
File "/home/zoutengda/anaconda3/envs/kt0512/lib/python3.11/site-packages/ktransformers/models/custom_cache.py", line 278, in
class KGQACache(nn.Module):
File "/home/zoutengda/anaconda3/envs/kt0512/lib/python3.11/site-packages/ktransformers/models/custom_cache.py", line 296, in KGQACache
def load(self, inference_context: sched_ext.InferenceContext):
^^^^^^^^^
NameError: name 'sched_ext' is not defined
一样的问题（安装时使用了USE_BALANCE_SERVE=1），有无解决办法？
@KathySkyer commented on GitHub (May 14, 2025):
回退到0.24版本就没有这个问题了，0.3不知道怎么解