kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #287] chat回复乱码 #181

New issue

Closed

opened 2026-10-29 14:35:27 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-29 14:35:27 +00:00

Owner

Originally created by @SanchezGao on GitHub (Feb 14, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/287

现象：

部署后可以运行，但是chat回复是乱码。

步骤

python环境已安装（非虚拟环境，直接在本机的3.11.2上安装的）
运行命令 ktransformers --model_path deepseek-ai/DeepSeek-R1 --gguf_path D:/AI/models/DeepSeek-R1-UD-IQ1_S --optimize_config_path D:/AI/ktransformers/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml --port 8002 --max_new_tokens 2048 --cpu_infer 26 --web True
打开 http://localhost:8002/web/index.html#/chat
输入问题回车
十几秒后开始回复乱码

希望结果
回复正常

硬件环境
OS: win11
GPU: NVIDIA RTX A2000 12G
CPU: Intel(R) Xeon(R) w7-3465X 2.50 GHz
RAM: 192G
Cores: 28

软件环境
ktransformers: 0.2.0 ktransformers-0.2.0+cu124torch24avx512-cp311-cp311-win_amd64.whl
python: 3.11.2
cuda: 12.4
torch: 2.4

model: deepseek-ai/DeepSeek-R1
gguf: https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main/DeepSeek-R1-UD-IQ1_S
optimize_config: DeepSeek-V3-Chat.yaml
cpu_infer: 26

附件

Originally created by @SanchezGao on GitHub (Feb 14, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/287 **现象：** 部署后可以运行，但是chat回复是乱码。 **步骤** - python环境已安装（非虚拟环境，直接在本机的3.11.2上安装的） - 运行命令 ktransformers --model_path deepseek-ai/DeepSeek-R1 --gguf_path D:/AI/models/DeepSeek-R1-UD-IQ1_S --optimize_config_path D:/AI/ktransformers/ktransformers/ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat.yaml --port 8002 --max_new_tokens 2048 --cpu_infer 26 --web True - 打开 http://localhost:8002/web/index.html#/chat - 输入问题回车 - 十几秒后开始回复乱码 **希望结果** 回复正常 **硬件环境** OS: win11 GPU: NVIDIA RTX A2000 12G CPU: Intel(R) Xeon(R) w7-3465X 2.50 GHz RAM: 192G Cores: 28 **软件环境** ktransformers: 0.2.0 [ktransformers-0.2.0+cu124torch24avx512-cp311-cp311-win_amd64.whl](https://github.com/kvcache-ai/ktransformers/releases/download/v0.2.0/ktransformers-0.2.0+cu124torch24avx512-cp311-cp311-win_amd64.whl) python: 3.11.2 cuda: 12.4 torch: 2.4 model: deepseek-ai/DeepSeek-R1 gguf: https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main/DeepSeek-R1-UD-IQ1_S optimize_config: DeepSeek-V3-Chat.yaml cpu_infer: 26 **附件** <img width="925" alt="Image" src="https://github.com/user-attachments/assets/5d27d7c6-1176-4346-b8ae-1914e6719ada" />

bsakaguchi closed this issue

2026-10-29 14:35:27 +00:00

bsakaguchi commented

2026-10-29 14:35:28 +00:00

Author

Owner

@vc5409ftu commented on GitHub (Feb 15, 2025):

显卡架构太老

@vc5409ftu commented on GitHub (Feb 15, 2025): 显卡架构太老

bsakaguchi commented

2026-10-29 14:35:28 +00:00

Author

Owner

@gavid0124 commented on GitHub (Feb 19, 2025):

我也遇到了乱码的问题。

@gavid0124 commented on GitHub (Feb 19, 2025): 我也遇到了乱码的问题。

bsakaguchi commented

2026-10-29 14:35:28 +00:00

Author

Owner

@gavid0124 commented on GitHub (Feb 19, 2025):

显卡架构太老

我用的4090的显卡，也是有乱码，请问您解决了吗？

@gavid0124 commented on GitHub (Feb 19, 2025): > 显卡架构太老我用的4090的显卡，也是有乱码，请问您解决了吗？

bsakaguchi commented

2026-10-29 14:35:28 +00:00

Author

Owner

@chengligen commented on GitHub (Feb 21, 2025):

我也遇到了乱码的问题。
显卡是a100-80G版本

@chengligen commented on GitHub (Feb 21, 2025): 我也遇到了乱码的问题。显卡是a100-80G版本

bsakaguchi commented

2026-10-29 14:35:28 +00:00

Author

Owner

@johnray22 commented on GitHub (Feb 21, 2025):

IQ1还不支持

@johnray22 commented on GitHub (Feb 21, 2025): IQ1还不支持

bsakaguchi commented

2026-10-29 14:35:28 +00:00

Author

Owner

@SanchezGao commented on GitHub (Feb 21, 2025):

出现乱码之后就没有继续尝试了，感觉KT还在前期阶段，准备观望一段时间再说。 @gavid0124 @chengligen @johnray22 @vc5409ftu

@SanchezGao commented on GitHub (Feb 21, 2025): 出现乱码之后就没有继续尝试了，感觉KT还在前期阶段，准备观望一段时间再说。 @gavid0124 @chengligen @johnray22 @vc5409ftu