[GH-ISSUE #1028] [Bug] 0.2.4版调查

bsakaguchi commented

2026-10-29 14:43:36 +00:00

Owner

Originally created by @zhangjiekui on GitHub (Apr 3, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1028

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.
3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

安装了编译依赖后，编译不成功。
重启服务器，结果机器弄挂掉了

Reproduction

想了解一下，0.2.4版：（1）有没有build成功的？（2）正常运行的？（3）生成速度有没有提升？

Environment

Ubuntu 22.04

Originally created by @zhangjiekui on GitHub (Apr 3, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1028 ### Checklist - [ ] 1. I have searched related issues but cannot get the expected help. - [ ] 2. The bug has not been fixed in the latest version. - [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [ ] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [ ] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug 安装了编译依赖后，编译不成功。重启服务器，结果机器弄挂掉了 ### Reproduction 想了解一下，0.2.4版：（1）有没有build成功的？（2）正常运行的？（3）生成速度有没有提升？ ### Environment Ubuntu 22.04

bsakaguchi closed this issue

2026-10-29 14:43:36 +00:00

bsakaguchi commented

2026-10-29 14:43:37 +00:00

Author

Owner

@CYSTEV-chn commented on GitHub (Apr 3, 2025):

编译成功的有不少，正常加载deepseek r1的少，但是也有，能开启对话的目前就看到一个。

@CYSTEV-chn commented on GitHub (Apr 3, 2025): 编译成功的有不少，正常加载deepseek r1的少，但是也有，能开启对话的目前就看到一个。

bsakaguchi commented

2026-10-29 14:43:37 +00:00

Author

Owner

@SaturMars commented on GitHub (Apr 3, 2025):

成功了啊，运行了啊，变慢了啊。

@SaturMars commented on GitHub (Apr 3, 2025): 成功了啊，运行了啊，变慢了啊。

bsakaguchi commented

2026-10-29 14:43:37 +00:00

Author

Owner

@344303947 commented on GitHub (Apr 3, 2025):

编译成功了，但是运行加载报错。

@344303947 commented on GitHub (Apr 3, 2025): 编译成功了，但是运行加载报错。

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@344303947 commented on GitHub (Apr 3, 2025):

删除了~/.ktrxxx文件和修改了部分代码成功了
# asyncserver
self.sched_strategy = cfg["async_server"]["sched_strategy"]
self.sched_port = cfg["async_server"]["sched_port"]
self.sched_metrics_port = cfg["async_server"]["sched_metrics_port"]
self.kvc2_metrics_port = cfg["async_server"]["kvc2_metrics_port"]
self.max_batch_size = cfg["async_server"]["max_batch_size"]
self.page_size = cfg["attn"]["page_size"]
self.chunk_size = cfg["attn"]["chunk_size"]
self.memory_gpu_only = cfg["kvc2"]["gpu_only"]
self.cache_lens = ((self.cache_lens + self.page_size - 1) // self.page_size) * self.page_size
self.gpu_memory_size = 257661*self.cache_lens
self.utilization_percentage = 1.0 #cfg["kvc2"]["utilization_percentage"]
self.cpu_memory_size_GB = cfg["kvc2"]["cpu_memory_size_GB"]
# only support 2 prefill task
self.max_prefill_batch_size = 2
self.max_decode_batch_size = self.max_batch_size - self.max_prefill_batch_size

@344303947 commented on GitHub (Apr 3, 2025): 删除了~/.ktrxxx文件和修改了部分代码成功了 # asyncserver self.sched_strategy = cfg["async_server"]["sched_strategy"] self.sched_port = cfg["async_server"]["sched_port"] self.sched_metrics_port = cfg["async_server"]["sched_metrics_port"] self.kvc2_metrics_port = cfg["async_server"]["kvc2_metrics_port"] self.max_batch_size = cfg["async_server"]["max_batch_size"] self.page_size = cfg["attn"]["page_size"] self.chunk_size = cfg["attn"]["chunk_size"] self.memory_gpu_only = cfg["kvc2"]["gpu_only"] self.cache_lens = ((self.cache_lens + self.page_size - 1) // self.page_size) * self.page_size self.gpu_memory_size = 2*576*61*self.cache_lens self.utilization_percentage = 1.0 #cfg["kvc2"]["utilization_percentage"] self.cpu_memory_size_GB = cfg["kvc2"]["cpu_memory_size_GB"] # only support 2 prefill task self.max_prefill_batch_size = 2 self.max_decode_batch_size = self.max_batch_size - self.max_prefill_batch_size

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@CYSTEV-chn commented on GitHub (Apr 3, 2025):

删除了~/.ktrxxx文件和修改了部分代码成功了 # asyncserver self.sched_strategy = cfg["async_server"]["sched_strategy"] self.sched_port = cfg["async_server"]["sched_port"] self.sched_metrics_port = cfg["async_server"]["sched_metrics_port"] self.kvc2_metrics_port = cfg["async_server"]["kvc2_metrics_port"] self.max_batch_size = cfg["async_server"]["max_batch_size"] self.page_size = cfg["attn"]["page_size"] self.chunk_size = cfg["attn"]["chunk_size"] self.memory_gpu_only = cfg["kvc2"]["gpu_only"] self.cache_lens = ((self.cache_lens + self.page_size - 1) // self.page_size) * self.page_size self.gpu_memory_size = 2_576_61*self.cache_lens self.utilization_percentage = 1.0 #cfg["kvc2"]["utilization_percentage"] self.cpu_memory_size_GB = cfg["kvc2"]["cpu_memory_size_GB"] # only support 2 prefill task self.max_prefill_batch_size = 2 self.max_decode_batch_size = self.max_batch_size - self.max_prefill_batch_size

大佬改的是哪个文件的代码啊。。。。

@CYSTEV-chn commented on GitHub (Apr 3, 2025): > 删除了~/.ktrxxx文件和修改了部分代码成功了 # asyncserver self.sched_strategy = cfg["async_server"]["sched_strategy"] self.sched_port = cfg["async_server"]["sched_port"] self.sched_metrics_port = cfg["async_server"]["sched_metrics_port"] self.kvc2_metrics_port = cfg["async_server"]["kvc2_metrics_port"] self.max_batch_size = cfg["async_server"]["max_batch_size"] self.page_size = cfg["attn"]["page_size"] self.chunk_size = cfg["attn"]["chunk_size"] self.memory_gpu_only = cfg["kvc2"]["gpu_only"] self.cache_lens = ((self.cache_lens + self.page_size - 1) // self.page_size) * self.page_size self.gpu_memory_size = 2_576_61*self.cache_lens self.utilization_percentage = 1.0 #cfg["kvc2"]["utilization_percentage"] self.cpu_memory_size_GB = cfg["kvc2"]["cpu_memory_size_GB"] # only support 2 prefill task self.max_prefill_batch_size = 2 self.max_decode_batch_size = self.max_batch_size - self.max_prefill_batch_size 大佬改的是哪个文件的代码啊。。。。

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@344303947 commented on GitHub (Apr 3, 2025):

ktransformers/server/config/config.py

看报错位置，改这个地方还有环境里面的这个

@344303947 commented on GitHub (Apr 3, 2025): ktransformers/server/config/config.py 看报错位置，改这个地方还有环境里面的这个

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@344303947 commented on GitHub (Apr 3, 2025):

也是运行成功了，实际调用接口不会响应

@344303947 commented on GitHub (Apr 3, 2025): 也是运行成功了，实际调用接口不会响应 ![Image](https://github.com/user-attachments/assets/8d864423-df65-4a6a-88d9-f651cb678f56)

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@shinchou commented on GitHub (Apr 3, 2025):

ubuntu22.04, 删了.ktransformers目录, conda环境起了个新的, 原来的项目目录删了, 编译失败。
无奈, 切回0.2.3post2, 继续观望。

@shinchou commented on GitHub (Apr 3, 2025): ubuntu22.04, 删了.ktransformers目录, conda环境起了个新的, 原来的项目目录删了, 编译失败。无奈, 切回0.2.3post2, 继续观望。

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@Pb-207 commented on GitHub (Apr 3, 2025):

跑起来了，但是并发功能没有生效，推理中无法终止。

@Pb-207 commented on GitHub (Apr 3, 2025): 跑起来了，但是并发功能没有生效，推理中无法终止。

bsakaguchi commented

2026-10-29 14:43:38 +00:00

Author

Owner

@344303947 commented on GitHub (Apr 4, 2025):

跑成功了，并发4路成功，单路和原来没区别，并发大概速度会根据并发数量对半下降，挺好用的。
新版会有多余的显卡打印日志输出，建议官网优化下。

【Ktransformers 0.2.4支持DeepSeek-V3-0324并发部署测试】https://www.bilibili.com/video/BV1JLZoYGENu?vd_source=954828d4da44f3115ef5b39efde8a320

@344303947 commented on GitHub (Apr 4, 2025): 跑成功了，并发4路成功，单路和原来没区别，并发大概速度会根据并发数量对半下降，挺好用的。新版会有多余的显卡打印日志输出，建议官网优化下。【Ktransformers 0.2.4支持DeepSeek-V3-0324并发部署测试】https://www.bilibili.com/video/BV1JLZoYGENu?vd_source=954828d4da44f3115ef5b39efde8a320

bsakaguchi commented

2026-10-29 14:43:39 +00:00

Author

Owner

@qiyuxinlin commented on GitHub (Apr 5, 2025):

跑成功了，并发4路成功，单路和原来没区别，并发大概速度会根据并发数量对半下降，挺好用的。新版会有多余的显卡打印日志输出，建议官网优化下。

【Ktransformers 0.2.4支持DeepSeek-V3-0324并发部署测试】https://www.bilibili.com/video/BV1JLZoYGENu?vd_source=954828d4da44f3115ef5b39efde8a320

非常感谢您提供了视频教程！很高兴听到并发运行效果良好，关于新版多余的显卡日志输出，我们会尽快优化并在后续版本中改进，感谢支持！

@qiyuxinlin commented on GitHub (Apr 5, 2025): > 跑成功了，并发4路成功，单路和原来没区别，并发大概速度会根据并发数量对半下降，挺好用的。新版会有多余的显卡打印日志输出，建议官网优化下。 > > 【Ktransformers 0.2.4支持DeepSeek-V3-0324并发部署测试】https://www.bilibili.com/video/BV1JLZoYGENu?vd_source=954828d4da44f3115ef5b39efde8a320 非常感谢您提供了视频教程！很高兴听到并发运行效果良好，关于新版多余的显卡日志输出，我们会尽快优化并在后续版本中改进，感谢支持！

bsakaguchi commented

2026-10-29 14:43:39 +00:00

Author

Owner

@NUSTHR commented on GitHub (Apr 8, 2025):

跑起来了，但是并发功能没有生效，推理中无法终止。

一样的问题，很奇怪

@NUSTHR commented on GitHub (Apr 8, 2025): > 跑起来了，但是并发功能没有生效，推理中无法终止。一样的问题，很奇怪

Rows
Columns

[GH-ISSUE #1028] [Bug] 0.2.4版调查 #728

Checklist

Describe the bug

Reproduction

Environment