[GH-ISSUE #1318] [Bug] 程序停置一段时间后对话卡住 #887

New issue

Closed

opened 2026-10-29 14:45:46 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:45:46 +00:00

Owner

Originally created by @jiyif11 on GitHub (May 18, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1318

检查清单

1. 我已经搜索过相关问题，但未能获得预期的帮助
2. 该问题在最新版本中尚未修复
3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述

停置数小时后再次调用api就处于卡住状态，日志：
INFO: 192.168.1.4:35544 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2025-05-18 01:16:00,823 DEBUG /root/test/.env/lib/python3.12/site-packages/ktransformers/server/backend/interfaces/balance_serve.py[418]: get input ids of shape torch.Size([1, 135])

复现步骤

deepseek R1 Q 4
`max_new_tokens: 8192
cache_lens: 65535

web:
mount: False
open_cross_domain: True

ext:
cpu_infer: 10

long_context:
max_seq_len: 32768
block_size: 128
local_windows_len: 4096
second_select_num: 32
anchor_type: DYNAMIC
kv_type: FP16
dense_layer_num: 2
anchor_num: 1
preselect_block: True
head_select_mode: SHARED
preselect_block_count: 32
layer_step: 1
token_step:

local_chat:
prompt_file: ""

async_server:
sched_strategy: "FCFS"
sched_port: 56441
sched_metrics_port: 54321
kvc2_metrics_port: 54391
max_batch_size: 4 # decode count + prefill count, in one mini batch

attn:
page_size: 256
chunk_size: 256
kvc2:
gpu_only: true
utilization_percentage: 1.0
cpu_memory_size_GB: 500`

环境信息

ubuntu-24.04
i9
ddr5 5200

Originally created by @jiyif11 on GitHub (May 18, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1318 ### 检查清单 - [x] 1. 我已经搜索过相关问题，但未能获得预期的帮助 - [x] 2. 该问题在最新版本中尚未修复 - [x] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性 - [x] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [x] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭 ### 问题描述停置数小时后再次调用api就处于卡住状态，日志： `INFO: 192.168.1.4:35544 - "POST /v1/chat/completions HTTP/1.1" 200 OK 2025-05-18 01:16:00,823 DEBUG /root/test/.env/lib/python3.12/site-packages/ktransformers/server/backend/interfaces/balance_serve.py[418]: get input ids of shape torch.Size([1, 135])` ### 复现步骤 deepseek R1 Q 4 `max_new_tokens: 8192 cache_lens: 65535 web: mount: False open_cross_domain: True ext: cpu_infer: 10 long_context: max_seq_len: 32768 block_size: 128 local_windows_len: 4096 second_select_num: 32 anchor_type: DYNAMIC kv_type: FP16 dense_layer_num: 2 anchor_num: 1 preselect_block: True head_select_mode: SHARED preselect_block_count: 32 layer_step: 1 token_step: local_chat: prompt_file: "" async_server: sched_strategy: "FCFS" sched_port: 56441 sched_metrics_port: 54321 kvc2_metrics_port: 54391 max_batch_size: 4 # decode count + prefill count, in one mini batch attn: page_size: 256 chunk_size: 256 kvc2: gpu_only: true utilization_percentage: 1.0 cpu_memory_size_GB: 500` ### 环境信息 ubuntu-24.04 i9 ddr5 5200

bsakaguchi closed this issue

2026-10-29 14:45:46 +00:00