kvcache-ai/ktransformers
1
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00
Code Issues 469 Projects Packages Wiki Activity

[GH-ISSUE #1081] v0.2.4post1 正确的 运行命令是什么 #760

New issue
Open
opened 2026-10-29 14:44:06 +00:00 by bsakaguchi · 11 comments
bsakaguchi commented 2026-10-29 14:44:06 +00:00
Owner
Copy link

Originally created by @113514089 on GitHub (Apr 8, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1081

v0.2.4post1 对运行命令有什么要求
出现了 KeyError: 'async_server' 这个问题，运行命令是这个
PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9"
ktransformers --model_path deepseek-ai/DeepSeek-V3-0324
--gguf_path /home/uxj/DeepSeek-V3-0324_Q4KM/Q4_K_M/
--port 10002
--optimize_config_path /home/uxj/ktrun/workspace/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml
--use_cuda_graph
--temperature 0.5
--cpu_infer 65
--port 10002
--max_new_tokens 8192

在0.23版本运行正常

还是说需要修改文件。视频看了，没太明白，视频里面有不合理的地方，能详细说明吗？
我看了，有两个地方要改，一个路径名称要求，路径名称规范是什么？能明确吗？
还有要修改一个文件，但是具体修改什么东西，没看明白

Originally created by @113514089 on GitHub (Apr 8, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1081 v0.2.4post1 对运行命令有什么要求 出现了 KeyError: 'async_server' 这个问题，运行命令是这个 PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9" \ ktransformers --model_path deepseek-ai/DeepSeek-V3-0324 \ --gguf_path /home/uxj/DeepSeek-V3-0324_Q4KM/Q4_K_M/ \ --port 10002 \ --optimize_config_path /home/uxj/ktrun/workspace/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml \ --use_cuda_graph \ --temperature 0.5 \ --cpu_infer 65 \ --port 10002 \ --max_new_tokens 8192 在0.23版本运行正常 还是说需要修改文件。视频看了，没太明白，视频里面有不合理的地方，能详细说明吗？ 我看了，有两个地方要改，一个路径名称要求，路径名称规范是什么？能明确吗？ 还有要修改一个文件，但是具体修改什么东西，没看明白
bsakaguchi commented 2026-10-29 14:44:07 +00:00
Author
Owner
Copy link

@sunhaoyu-1990 commented on GitHub (Apr 8, 2025):

你可以参考这个https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md
里边说到运行命令确实是变更了，如下：

python ktransformers/server/main.py \
  --port 10002 \
  --model_path <path_to_safetensor_config> \
  --gguf_path <path_to_gguf_files> \
  --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml \
  --max_new_tokens 1024 \
  --cache_lens 32768 \
  --chunk_size 256 \
  --max_batch_size 4 \
  --backend_type balance_serve \
  --force_think # useful for R1
<!-- gh-comment-id:2785615568 --> @sunhaoyu-1990 commented on GitHub (Apr 8, 2025): 你可以参考这个https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md 里边说到运行命令确实是变更了，如下： ``` python ktransformers/server/main.py \ --port 10002 \ --model_path <path_to_safetensor_config> \ --gguf_path <path_to_gguf_files> \ --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml \ --max_new_tokens 1024 \ --cache_lens 32768 \ --chunk_size 256 \ --max_batch_size 4 \ --backend_type balance_serve \ --force_think # useful for R1 ```
bsakaguchi commented 2026-10-29 14:44:07 +00:00
Author
Owner
Copy link

@113514089 commented on GitHub (Apr 8, 2025):

PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9"
python ktransformers/server/main.py
--model_path deepseek-ai/DeepSeek-V3-0324
--gguf_path /home/uxj/DeepSeek-V3-0324_Q4KM/Q4_K_M/
--optimize_config_path /home/uxj/kt0324/ktrun/workspace/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml
--cache_lens 32768
--chunk_size 256
--max_batch_size 4
--backend_type balance_serve
--port 10002
--max_new_tokens 8192

命令改了，还是不行。

File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/main.py", line 11, in
from ktransformers.server.args import ArgumentParser
File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/args.py", line 2, in
from ktransformers.server.backend.args import ConfigArgs, default_args
File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/backend/args.py", line 79, in
cfg = Config()
^^^^^^^^
File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/singleton.py", line 26, in call
cls._instances[cls] = super(Singleton, cls).call(*args, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/config.py", line 187, in init
self.sched_strategy = cfg["async_server"]["sched_strategy"]
~~~^^^^^^^^^^^^^^^^
KeyError: 'async_server'

<!-- gh-comment-id:2785715327 --> @113514089 commented on GitHub (Apr 8, 2025): PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9" \ python ktransformers/server/main.py \ --model_path deepseek-ai/DeepSeek-V3-0324 \ --gguf_path /home/uxj/DeepSeek-V3-0324_Q4KM/Q4_K_M/ \ --optimize_config_path /home/uxj/kt0324/ktrun/workspace/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml \ --cache_lens 32768 \ --chunk_size 256 \ --max_batch_size 4 \ --backend_type balance_serve \ --port 10002 \ --max_new_tokens 8192 命令改了，还是不行。 File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/main.py", line 11, in <module> from ktransformers.server.args import ArgumentParser File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/args.py", line 2, in <module> from ktransformers.server.backend.args import ConfigArgs, default_args File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/backend/args.py", line 79, in <module> cfg = Config() ^^^^^^^^ File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/singleton.py", line 26, in __call__ cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/config.py", line 187, in __init__ self.sched_strategy = cfg["async_server"]["sched_strategy"] ~~~^^^^^^^^^^^^^^^^ KeyError: 'async_server'
bsakaguchi commented 2026-10-29 14:44:07 +00:00
Author
Owner
Copy link

@qiyuxinlin commented on GitHub (Apr 8, 2025):

PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9" python ktransformers/server/main.py --model_path deepseek-ai/DeepSeek-V3-0324 --gguf_path /home/uxj/DeepSeek-V3-0324_Q4KM/Q4_K_M/ --optimize_config_path /home/uxj/kt0324/ktrun/workspace/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml --cache_lens 32768 --chunk_size 256 --max_batch_size 4 --backend_type balance_serve --port 10002 --max_new_tokens 8192

命令改了，还是不行。

File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/main.py", line 11, in from ktransformers.server.args import ArgumentParser File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/args.py", line 2, in from ktransformers.server.backend.args import ConfigArgs, default_args File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/backend/args.py", line 79, in cfg = Config() ^^^^^^^^ File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/singleton.py", line 26, in call cls._instances[cls] = super(Singleton, cls).call(*args, **kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/config.py", line 187, in init self.sched_strategy = cfg["async_server"]["sched_strategy"] ~~~^^^^^^^^^^^^^^^^ KeyError: 'async_server'

您好，删除您本地 ～/.ktransformers 重新运行下即可

<!-- gh-comment-id:2785949175 --> @qiyuxinlin commented on GitHub (Apr 8, 2025): > PYTHONPATH="$PYTHONPATH" TORCH_CUDA_ARCH_LIST="8.9" python ktransformers/server/main.py --model_path deepseek-ai/DeepSeek-V3-0324 --gguf_path /home/uxj/DeepSeek-V3-0324_Q4KM/Q4_K_M/ --optimize_config_path /home/uxj/kt0324/ktrun/workspace/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml --cache_lens 32768 --chunk_size 256 --max_batch_size 4 --backend_type balance_serve --port 10002 --max_new_tokens 8192 > > 命令改了，还是不行。 > > File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/main.py", line 11, in from ktransformers.server.args import ArgumentParser File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/args.py", line 2, in from ktransformers.server.backend.args import ConfigArgs, default_args File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/backend/args.py", line 79, in cfg = Config() ^^^^^^^^ File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/singleton.py", line 26, in **call** cls._instances[cls] = super(Singleton, cls).**call**(*args, **kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/uxj/kt0324/ktrun/workspace/ktransformers/server/config/config.py", line 187, in **init** self.sched_strategy = cfg["async_server"]["sched_strategy"] ~~~^^^^^^^^^^^^^^^^ KeyError: 'async_server' 您好，删除您本地 ～/.ktransformers 重新运行下即可
bsakaguchi commented 2026-10-29 14:44:07 +00:00
Author
Owner
Copy link

@113514089 commented on GitHub (Apr 8, 2025):

谢谢大神的解答。
删除后。出现了这 import sched_ext
ModuleNotFoundError: No module named 'sched_ext'

<!-- gh-comment-id:2785972807 --> @113514089 commented on GitHub (Apr 8, 2025): 谢谢大神的解答。 删除后。出现了这 import sched_ext ModuleNotFoundError: No module named 'sched_ext'
bsakaguchi commented 2026-10-29 14:44:07 +00:00
Author
Owner
Copy link

@qiyuxinlin commented on GitHub (Apr 8, 2025):

谢谢大神的解答。 删除后。出现了这 import sched_ext ModuleNotFoundError: No module named 'sched_ext'

按照这个文档安装的吗？
https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md

<!-- gh-comment-id:2786091788 --> @qiyuxinlin commented on GitHub (Apr 8, 2025): > 谢谢大神的解答。 删除后。出现了这 import sched_ext ModuleNotFoundError: No module named 'sched_ext' 按照这个文档安装的吗？ https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/balance-serve.md
bsakaguchi commented 2026-10-29 14:44:07 +00:00
Author
Owner
Copy link

@113514089 commented on GitHub (Apr 8, 2025):

用的还是12.4的环境 0.23能跑的环境。需要重新安装所有吗。

<!-- gh-comment-id:2786114605 --> @113514089 commented on GitHub (Apr 8, 2025): 用的还是12.4的环境 0.23能跑的环境。需要重新安装所有吗。
bsakaguchi commented 2026-10-29 14:44:08 +00:00
Author
Owner
Copy link

@qiyuxinlin commented on GitHub (Apr 8, 2025):

用的还是12.4的环境 0.23能跑的环境。需要重新安装所有吗。

是的，最好开一个 conda 环境重新装一遍

<!-- gh-comment-id:2786122870 --> @qiyuxinlin commented on GitHub (Apr 8, 2025): > 用的还是12.4的环境 0.23能跑的环境。需要重新安装所有吗。 是的，最好开一个 conda 环境重新装一遍
bsakaguchi commented 2026-10-29 14:44:08 +00:00
Author
Owner
Copy link

@zhenjieliang commented on GitHub (Apr 9, 2025):

我的系统是manjaro,参照安装教程中，安装到libtbb-dev libssl-dev libcurl4-openssl-dev libaio1 libaio-dev libfmt-dev libgflags-dev zlib1g-dev patchelf,使用manjaro系统下的sudo pacman -S libssh2 curl openssl之后，就让原有的软件降级了，现在安装任何包都报错，

pacman: symbol lookup error: /usr/lib/libcurl.so.4: undefined symbol: libssh2_session_callback_set2

正在考虑重装系统了，搞不定

<!-- gh-comment-id:2790177419 --> @zhenjieliang commented on GitHub (Apr 9, 2025): 我的系统是manjaro,参照安装教程中，安装到libtbb-dev libssl-dev libcurl4-openssl-dev libaio1 libaio-dev libfmt-dev libgflags-dev zlib1g-dev patchelf,使用manjaro系统下的sudo pacman -S libssh2 curl openssl之后，就让原有的软件降级了，现在安装任何包都报错， pacman: symbol lookup error: /usr/lib/libcurl.so.4: undefined symbol: libssh2_session_callback_set2 正在考虑重装系统了，搞不定
bsakaguchi commented 2026-10-29 14:44:08 +00:00
Author
Owner
Copy link

@yeungtuzi commented on GitHub (Apr 11, 2025):

不行就试试docker吧

<!-- gh-comment-id:2796248391 --> @yeungtuzi commented on GitHub (Apr 11, 2025): 不行就试试docker吧
bsakaguchi commented 2026-10-29 14:44:08 +00:00
Author
Owner
Copy link

@lililolo0927 commented on GitHub (Apr 12, 2025):

Try to change model dir ?

<!-- gh-comment-id:2798814187 --> @lililolo0927 commented on GitHub (Apr 12, 2025): Try to change model dir ?
bsakaguchi commented 2026-10-29 14:44:08 +00:00
Author
Owner
Copy link

@lergliu commented on GitHub (Apr 15, 2025):

docker运行ktransformers:v0.2.4post1-AVX512这个镜像，报ImportError: libsched.so: cannot open shared object file: No such file or directory，有大佬知道什么原因吗

<!-- gh-comment-id:2804530939 --> @lergliu commented on GitHub (Apr 15, 2025): docker运行ktransformers:v0.2.4post1-AVX512这个镜像，报ImportError: libsched.so: cannot open shared object file: No such file or directory，有大佬知道什么原因吗
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/sync-sglang
gh-pages
docs/kt-ft-figure2-single-column-20261010
perf/bf16-grad-norm-20261010
docs/kt-ft-paper-20261008
release/20260911-carrier-repair
release/20260911-ci-python
release/20260911-pypi
ci/four-main-pypi-release
ci/model-e2e-acceptance
release/0.7.0.post3-frozen-pypi
fix/glm53-index-share-sync
release/0.7.0.post2-frozen-pypi
hotfix/0.7.0.post2-pypi-runtime
fix/avx2-wheel-feature-isolation
fix/release-self-hosted-runner
release/automate-kt-sft-pypi
support-glm5-next-djw
docs/deepseek-v4-flash-docker
ovowei-patch-3
support-glm52-djw
doc-reorg-entrypoints-support-matrix
bump-sglang-pr31
docs-v061-refresh
fp4-moe-amx
flatten-ktransformers-shim
archive-kt-sft
kt-kernel-torch-range-v061
sft-whl
sft
docs/gosim-2026-and-roadmap-link
ovowei-patch-2
ovowei-patch-1
revert-1918-fix/rocm-cpuinfer-stream-bridge
chore/kt-layerwise-prefill-main
chore/kt-layerwise-prefill-label
sft-rel
support-glm51-djw
release/v0.5.3
chore/kt-run-numa-nodes-unified
mtp
ksft-sglang
avx2
fix/sglang-kt-detection
layerwise-prefill-opt
feat/mistral-loader-support
fix-docker-build
minimax-m2.5
qwen3.5
add-maintainers
oql-doc
cli-dev-ljq
ljq-cli-dev-new
JimmyPeilinLi-patch-2
kimi_k2.5
oql-dev
experts_sched
JimmyPeilinLi-patch-1
ErvinXie-patch-1
ksft-llama
glm47_fp8
debug_llama
refactor_sft
ljq-cli-dev
refactor_sft_int4_debug
lut-nvfp4
fix-import
fix-deploy-djw
cpu-fix
ljq-dev-1
route-moe
sosp25-ae
v0.7.2
paper-ktft-v1.1
paper-ktft-v1
v0.7.1
v0.7.0.post4
v0.7.0.post1
v0.7.0
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post3
v0.6.2.post2
v0.6.2.post1
v0.6.2
v0.6.1
v0.6.1.post1
v0.5.3
v0.5.2.post1
v0.5.1
v0.5.0.post1
v0.5.0
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.3.2
v0.3.1
v0.3
v0.2.4post1
v0.2.4
v0.2.3post2
v0.2.3post1
v0.2.3
v0.2.2
v0.2.2rc2
v0.2.2rc1
v0.2.1.post1
v0.2.1
v0.2.0
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

  
stale
No labels
bug
duplicate
enhancement
good first issue
help wanted
pull-request
stale
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kvcache-ai/ktransformers#760
No description provided.