kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #871] [bug]TypeError: expected string or bytes-like object, got 'NoneType' #625

New issue

Open

opened 2026-10-29 14:42:05 +00:00 by bsakaguchi · 6 comments

bsakaguchi commented

2026-10-29 14:42:05 +00:00

Owner

Originally created by @lhtpluto on GitHub (Mar 12, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/871

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.
3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.

Describe the bug

执行（run)： python ktransformers/local_chat.py --model_path /home/ktransformers/models0/DeepSeek-R1-IQ1S-FP8 --optimize_config_path ktr ansformers/optimize/optimize_rules/DeepSeek-V3-Chat-fp8-linear-ggml-experts.yaml --cpu_infer 61

或执行(or run ):python ktransformers/local_chat.py --model_path /home/ktransformers/models0/DeepSeek-R1-IQ1S-FP8 --gguf_path /home/ktransformers/models01 --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-fp8-linear-ggml-experts.yaml --cpu_infer 61

提示错误(OSError)：OSError: /home/ktransformers/models0/DeepSeek-R1-IQ1S-FP8 does not appear to have a file named config.json.

PS: /home/ktransformers/models01/DeepSeek-R1-UD-IQ1_S.gguf

如果使用（if use）https://huggingface.co/deepseek-ai/DeepSeek-R1/tree/main中的 config.json和configuration_deepseek.py

报错（ValueError）：

Reproduction

我按照https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/fp8_kernel.md的说明，直接下载了 KVCache-ai/DeepSeek-R1-GGML-FP8-混合

Environment

OS：
Ubuntu 24.04.2 LTS (GNU/Linux 6.8.0-55-generic x86_64)：Linux version 6.8.0-55-generic (buildd@lcy02-amd64-095) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #57-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 23:42:21 UTC 2025

GPU： RTX4090

CPU： intel 8581C

RAM: 64GB*8

Originally created by @lhtpluto on GitHub (Mar 12, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/871 ### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. - [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback. - [x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed. - [x] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed. ### Describe the bug 执行（run)： python ktransformers/local_chat.py --model_path /home/ktransformers/models0/DeepSeek-R1-IQ1S-FP8 --optimize_config_path ktr ansformers/optimize/optimize_rules/DeepSeek-V3-Chat-fp8-linear-ggml-experts.yaml --cpu_infer 61 或执行(or run ):python ktransformers/local_chat.py --model_path /home/ktransformers/models0/DeepSeek-R1-IQ1S-FP8 --gguf_path /home/ktransformers/models01 --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-fp8-linear-ggml-experts.yaml --cpu_infer 61 提示错误(OSError)：OSError: /home/ktransformers/models0/DeepSeek-R1-IQ1S-FP8 does not appear to have a file named config.json. PS: /home/ktransformers/models01/DeepSeek-R1-UD-IQ1_S.gguf ![Image](https://github.com/user-attachments/assets/e0b07e45-f042-467c-8a55-4a0e2fd1c242) 如果使用（if use）https://huggingface.co/deepseek-ai/DeepSeek-R1/tree/main中的[config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1/blob/main/config.json)和[configuration_deepseek.py](https://huggingface.co/deepseek-ai/DeepSeek-R1/blob/main/configuration_deepseek.py) 报错（ValueError）： ![Image](https://github.com/user-attachments/assets/34d1e03c-51ad-4b3c-8208-6ab255c9f8b5) ### Reproduction 我按照https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/fp8_kernel.md的说明，直接下载了[KVCache-ai/DeepSeek-R1-GGML-FP8-混合](https://huggingface.co/KVCache-ai/DeepSeek-R1) ### Environment OS： Ubuntu 24.04.2 LTS (GNU/Linux 6.8.0-55-generic x86_64)：Linux version 6.8.0-55-generic (buildd@lcy02-amd64-095) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #57-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 23:42:21 UTC 2025 GPU： RTX4090 CPU： intel 8581C RAM: 64GB*8

bsakaguchi commented

2026-10-29 14:42:06 +00:00

Author

Owner

@lhtpluto commented on GitHub (Mar 12, 2025):

刚刚看到 https://github.com/kvcache-ai/ktransformers/issues/360

it worked

================

新的错误（new error）：ModuleNotFoundError: No module named 'flash_attn'

@lhtpluto commented on GitHub (Mar 12, 2025): 刚刚看到 https://github.com/kvcache-ai/ktransformers/issues/360 it worked ================ 新的错误（new error）：ModuleNotFoundError: No module named 'flash_attn' ![Image](https://github.com/user-attachments/assets/ad424bb7-5a89-4680-8646-fd9bd2936b03) ![Image](https://github.com/user-attachments/assets/3c69f8ae-78d3-4d48-8447-6f53c8767f27)

bsakaguchi commented

2026-10-29 14:42:06 +00:00

Author

Owner

@lhtpluto commented on GitHub (Mar 12, 2025):

https://kvcache-ai.github.io/ktransformers/en/install.html
"同时，您应该从https://github.com/Dao-AILab/flash-attention/releases下载并安装相应版本的flash-attention。"

pip install flash_attn-2.7.4.post1+cu12torch2.6cxx11abiTRUE-cp311-cp311-linux_x86_64.whl

报错（ImportError）：

@lhtpluto commented on GitHub (Mar 12, 2025): https://kvcache-ai.github.io/ktransformers/en/install.html "同时，您应该从https://github.com/Dao-AILab/flash-attention/releases下载并安装相应版本的flash-attention。" pip install flash_attn-2.7.4.post1+cu12torch2.6cxx11abiTRUE-cp311-cp311-linux_x86_64.whl 报错（ImportError）： ![Image](https://github.com/user-attachments/assets/11010682-5aae-48f5-86bf-5b7a9d00a286)

bsakaguchi commented

2026-10-29 14:42:06 +00:00

Author

Owner

@lhtpluto commented on GitHub (Mar 12, 2025):

pip install flash-attn --no-build-isolation
python setup.py install

报错（TypeError）：TypeError: expected string or bytes-like object, got 'NoneType'

@lhtpluto commented on GitHub (Mar 12, 2025): pip install flash-attn --no-build-isolation python setup.py install 报错（TypeError）：TypeError: expected string or bytes-like object, got 'NoneType' ![Image](https://github.com/user-attachments/assets/b4921318-569f-43d4-9c11-26cf2dd8a23d)

bsakaguchi commented

2026-10-29 14:42:07 +00:00

Author

Owner

@Azure-Tang commented on GitHub (Mar 12, 2025):

pip install flash-attn --no-build-isolation
python setup.py install

报错（TypeError）：TypeError: expected string or bytes-like object, got 'NoneType'

这个看着像torch没装好？在命令行输入python然后import troch看看？如果不行的话重装一下torch

@Azure-Tang commented on GitHub (Mar 12, 2025): > pip install flash-attn --no-build-isolation > python setup.py install > > 报错（TypeError）：TypeError: expected string or bytes-like object, got 'NoneType' > > ![Image](https://github.com/user-attachments/assets/b4921318-569f-43d4-9c11-26cf2dd8a23d) > > 这个看着像torch没装好？在命令行输入python然后import troch看看？如果不行的话重装一下torch

bsakaguchi commented

2026-10-29 14:42:07 +00:00

Author

Owner

@lhtpluto commented on GitHub (Mar 13, 2025):

pip install flash-attn --no-build-isolation
python setup.py install
报错（TypeError）：TypeError: expected string or bytes-like object, got 'NoneType'

这个看着像torch没装好？在命令行输入python然后import troch看看？如果不行的话重装一下torch

感觉您分析的对，稍后我测试下

@lhtpluto commented on GitHub (Mar 13, 2025): > > pip install flash-attn --no-build-isolation > > python setup.py install > > 报错（TypeError）：TypeError: expected string or bytes-like object, got 'NoneType' > > ![Image](https://github.com/user-attachments/assets/b4921318-569f-43d4-9c11-26cf2dd8a23d) > > 这个看着像torch没装好？在命令行输入python然后import troch看看？如果不行的话重装一下torch 感觉您分析的对，稍后我测试下

bsakaguchi commented

2026-10-29 14:42:07 +00:00

Author

Owner

@Atream commented on GitHub (Mar 15, 2025):

flash attn如果报错，请选择正确的whl包安装，一般来说如果symbol not found则应该安装abi False版本的whl包

@Atream commented on GitHub (Mar 15, 2025): flash attn如果报错，请选择正确的whl包安装，一般来说如果symbol not found则应该安装abi False版本的whl包