mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-11-05 01:07:28 +00:00
[GH-ISSUE #1413] [Bug] 0.3.2版本KT运行始终卡在Getting inference context from sched_client. sched_rpc started with PID: xxx #947
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/ktransformers#947
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @xufengnian on GitHub (Jul 2, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1413
检查清单
问题描述
一直追KT的最新版，发现昨天0.3.2版本更新以后，果断升级，
但是无论是DS-R1-0528还是Qwen3，都卡在了
下面位置
Getting inference context from sched_client.
sched_rpc started with PID: 3445
注：0.3.1及更早版本都可以正常run起来
查看了#1254，解决方法不适用我：我的model_path都是绝对路径，且都是hf下载的，之前版本都可以跑
相关运行记录
loading model.layers.89.self_attn.q_norm.weight to cuda
loading model.layers.89.self_attn.k_norm.weight to cuda
loading model.layers.89.input_layernorm.weight to cuda
loading model.layers.89.post_attention_layernorm.weight to cuda
loading model.layers.90.self_attn.q_norm.weight to cuda
loading model.layers.90.self_attn.k_norm.weight to cuda
loading model.layers.90.input_layernorm.weight to cuda
loading model.layers.90.post_attention_layernorm.weight to cuda
loading model.layers.91.self_attn.q_norm.weight to cuda
loading model.layers.91.self_attn.k_norm.weight to cuda
loading model.layers.91.input_layernorm.weight to cuda
loading model.layers.91.post_attention_layernorm.weight to cuda
loading model.layers.92.self_attn.q_norm.weight to cuda
loading model.layers.92.self_attn.k_norm.weight to cuda
loading model.layers.92.input_layernorm.weight to cuda
loading model.layers.92.post_attention_layernorm.weight to cuda
loading model.layers.93.self_attn.q_norm.weight to cuda
loading model.layers.93.self_attn.k_norm.weight to cuda
loading model.layers.93.input_layernorm.weight to cuda
loading model.layers.93.post_attention_layernorm.weight to cuda
loading model.norm.weight to cuda
Getting inference context from sched_client.
sched_rpc started with PID: 3445
/.ktransformers/rpc.log/.ktransformers/logs$ cat rpc.log(ktransformers-030-env) xxx@xxx-Super-Server:
[2025-07-02 15:10:05.304] [info] [scheduler.cpp:31] Number of available GPUs: 1, want 1
[2025-07-02 15:10:05.305] [info] [scheduler.cpp:66] Each GPU Total: 2196MiB, Model Params: 0MiB, KVCache: 2196MiB, Left: 0MiB
[2025-07-02 15:10:05.305] [info] [scheduler.cpp:87] total_kvcache_pages is auto derived as 128
[2025-07-02 15:10:05.305] [info] [scheduler.cpp:933] Using Strategy FCFS
[2025-07-02 15:10:05.305] [info] [scheduler.cpp:459]
Scheduler Settings:
model_name: DeepSeek-R1-0528
quant_type: BF16
model_path: /home/xxx/model/DeepSeek-R1-0528
params_count: 0
layer_count: 61
num_k_heads: 1
k_head_dim: 576
bytes_per_params: 2
bytes_per_kv_cache_element: 2
page_size: 256
gpu_device_id: 0
gpu_memory_size: 2.30G
memory_utilization_percentage: 1
max_batch_size: 4
recommended_chunk_prefill_token_count: 127
sched_metrics_port: 45591
kvc2_config_path: /home/xxx/.ktransformers/kvc2
kvc2_root_path: /mnt/data/kvc
memory_pool_size_GB: 500
evict_count: 40
kvc2_metrics_port: 35035
load_from_disk: false
save_to_disk: true
strategy_name: FCFS
gpu_device_count: 1
load_model_configs no file at "/home/xxx/.ktransformers/kvc2/model_configs.json"
Load from "/home/xxx/model/DeepSeek-R1-0528/config.json"
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1372] Creating KVC2 using these config
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1373] GPU Only: false
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1374] Load: false, Save: true
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1375] Path: /mnt/data/kvc
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1376] Config Path: /home/xxx/.ktransformers/kvc2
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1377] Num Token/Page: 256, Memory Pool Size: 500.00G
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1379] Evict Count: 40, Metrics Port: 35035
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1380] Recompute Ratio: 0.20
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1384] GPU Devices: 0
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1385] Layer Count: 61, Total KVCache Pages: 128
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1387] Num Token/Page: 256, Num K Heads: 1
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1388] K Head Dim: 576, Tensor Type: 15
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1390] MemcpyCudaStreams/Device: 4
load_model_configs from "/home/xxx/.ktransformers/kvc2/model_configs.json"
Loaded Model Configs
load_quant_configs no file at "/home/xxx/.ktransformers/kvc2/quant_configs.json"
[2025-07-02 15:10:05.305] [info] [prefix.cpp:1401] Creating kvc2 metrics exporter on 0.0.0.0:35035
[2025-07-02 15:10:05.306] [info] [prefix.cpp:278] DiskCacheManager root path: /mnt/data/kvc
[2025-07-02 15:42:18.867] [info] [scheduler.cpp:31] Number of available GPUs: 1, want 1
[2025-07-02 15:42:18.868] [info] [scheduler.cpp:66] Each GPU Total: 2196MiB, Model Params: 0MiB, KVCache: 2196MiB, Left: 0MiB
[2025-07-02 15:42:18.868] [info] [scheduler.cpp:87] total_kvcache_pages is auto derived as 128
[2025-07-02 15:42:18.868] [info] [scheduler.cpp:933] Using Strategy FCFS
[2025-07-02 15:42:18.868] [info] [scheduler.cpp:459]
Scheduler Settings:
model_name: DeepSeek-R1-0528
quant_type: BF16
model_path: /home/xxx/model/DeepSeek-R1-0528
params_count: 0
layer_count: 61
num_k_heads: 1
k_head_dim: 576
bytes_per_params: 2
bytes_per_kv_cache_element: 2
page_size: 256
gpu_device_id: 0
gpu_memory_size: 2.30G
memory_utilization_percentage: 1
max_batch_size: 4
recommended_chunk_prefill_token_count: 127
sched_metrics_port: 36237
kvc2_config_path: /home/xxx/.ktransformers/kvc2
kvc2_root_path: /mnt/data/kvc
memory_pool_size_GB: 500
evict_count: 40
kvc2_metrics_port: 58327
load_from_disk: false
save_to_disk: true
strategy_name: FCFS
gpu_device_count: 1
load_model_configs from "/home/xxx/.ktransformers/kvc2/model_configs.json"
Loaded Model Configs
Load from "/home/xxx/model/DeepSeek-R1-0528/config.json"
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1372] Creating KVC2 using these config
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1373] GPU Only: false
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1374] Load: false, Save: true
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1375] Path: /mnt/data/kvc
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1376] Config Path: /home/xxx/.ktransformers/kvc2
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1377] Num Token/Page: 256, Memory Pool Size: 500.00G
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1379] Evict Count: 40, Metrics Port: 58327
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1380] Recompute Ratio: 0.20
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1384] GPU Devices: 0
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1385] Layer Count: 61, Total KVCache Pages: 128
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1387] Num Token/Page: 256, Num K Heads: 1
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1388] K Head Dim: 576, Tensor Type: 15
[2025-07-02 15:42:18.869] [info] [prefix.cpp:1390] MemcpyCudaStreams/Device: 4
load_model_configs from "/home/xxx/.ktransformers/kvc2/model_configs.json"
Loaded Model Configs
复现步骤
python ktransformers/server/main.py --architectures Qwen3MoeForCausalLM --host 0.0.0.0 --port 20000 --api_key sk-xxx --model_path /home/xxx/model/Qwen3-235B-A22B --gguf_path /home/xxx/model/Qwen3-235B-A22B-Q4_0/Q4_0 --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml --max_new_tokens 8192 --cache_lens 32768 --chunk_size 256 --max_batch_size 4 --cpu_infer 40 --backend_type balance_serve
python ktransformers/server/main.py
--host 0.0.0.0
--port 20000
--api_key sk-xxx
--model_path /home/xxxmodel/DeepSeek-R1-0528
--gguf_path /home/xxx/model/DeepSeek-R1-0528-GGUF-Q4/Q4_K_M
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml
--max_new_tokens 8192
--cache_lens 32768
--chunk_size 256
--max_batch_size 4
--cpu_infer 40
--backend_type balance_serve
环境信息
环境：Ubuntu 22.04.4 LTS+Python 3.11
内存：768G DDR4
显卡 ：4090D 24G
CPU:Intel(R) Xeon(R) Silver 4310 CPU
@Atream commented on GitHub (Jul 3, 2025):
～/.ktransformers/log下会有更详细的日志
把disk_path改成当前账户有权限访问并修改的目录，或者使用sudo运行
@xufengnian commented on GitHub (Jul 3, 2025):
@Atream 非常感谢，问题已经解决
@BG8CFB commented on GitHub (Jul 27, 2025):
大佬，请问你是怎么解决的，我已经尝试了root权限执行，并且kvc目录给权限。没解决这个问题。
@SaiHoCao commented on GitHub (Aug 17, 2025):
@xufengnian 请问我修改后也是没效果，可以方便告诉怎么弄的吗
@xufengnian commented on GitHub (Aug 18, 2025):
~/.ktransformers/config.yaml配置里有个disk_path的配置，默认 /mnt/data/kvc，但是这个路径不一定在主机上有，因此需要修改一下，改成当前用户有权限的配置，比如~/kvc(实际用绝对路径)