[GH-ISSUE #757] [Bug] #547

New issue

Closed

opened 2026-10-29 14:41:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:41:05 +00:00

Owner

Originally created by @liugu2023 on GitHub (Mar 1, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/757

检查清单

1. 我已经搜索过相关问题，但未能获得预期的帮助
2. 该问题在最新版本中尚未修复
3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述

在运行ktransformers时，运行时报错Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/archive/liugu/ds/ktransformers/ktransformers/local_chat.py", line 183, in
fire.Fire(local_chat)
File "/archive/liugu/ds/ds/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/archive/liugu/ds/ds/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/archive/liugu/ds/ds/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/archive/liugu/ds/ktransformers/ktransformers/local_chat.py", line 110, in local_chat
optimize_and_load_gguf(model, optimize_config_path, gguf_path, config)
File "/archive/liugu/ds/ktransformers/ktransformers/optimize/optimize.py", line 133, in optimize_and_load_gguf
gguf_loader = GGUFLoader(gguf_path)
^^^^^^^^^^^^^^^^^^^^^
File "/archive/liugu/ds/ktransformers/ktransformers/util/custom_gguf.py", line 205, in init
self.load_gguf(f)
File "/archive/liugu/ds/ktransformers/ktransformers/util/custom_gguf.py", line 213, in load_gguf
assert f.read(4) == b'GGUF'
^^^^^^^^^^^^^^^^^^^^
AssertionError 但GGUF文件格式正确

复现步骤

我使用了以下命令python -m ktransformers.local_chat --model_path /archive/liugu/ds/DeepSeek-R1-Distill-Llama-70B --gguf_path /archive/liugu/ds/DeepSeek-R1-Distill-Llama-70B-GGUF ，运行的是Deepseek R1 70B

环境信息

操作系统：Debian GNU/Linux 12 (bookworm)
GPU： Tesla P100
CPU： Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
pytorch版本：12.5
cuda版本：11.8

Originally created by @liugu2023 on GitHub (Mar 1, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/757 ### 检查清单 - [x] 1. 我已经搜索过相关问题，但未能获得预期的帮助 - [x] 2. 该问题在最新版本中尚未修复 - [x] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性 - [x] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [x] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭 ### 问题描述在运行ktransformers时，运行时报错Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/archive/liugu/ds/ktransformers/ktransformers/local_chat.py", line 183, in <module> fire.Fire(local_chat) File "/archive/liugu/ds/ds/lib/python3.11/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/archive/liugu/ds/ds/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/archive/liugu/ds/ds/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/archive/liugu/ds/ktransformers/ktransformers/local_chat.py", line 110, in local_chat optimize_and_load_gguf(model, optimize_config_path, gguf_path, config) File "/archive/liugu/ds/ktransformers/ktransformers/optimize/optimize.py", line 133, in optimize_and_load_gguf gguf_loader = GGUFLoader(gguf_path) ^^^^^^^^^^^^^^^^^^^^^ File "/archive/liugu/ds/ktransformers/ktransformers/util/custom_gguf.py", line 205, in __init__ self.load_gguf(f) File "/archive/liugu/ds/ktransformers/ktransformers/util/custom_gguf.py", line 213, in load_gguf assert f.read(4) == b'GGUF' ^^^^^^^^^^^^^^^^^^^^ AssertionError 但GGUF文件格式正确 ### 复现步骤我使用了以下命令python -m ktransformers.local_chat --model_path /archive/liugu/ds/DeepSeek-R1-Distill-Llama-70B --gguf_path /archive/liugu/ds/DeepSeek-R1-Distill-Llama-70B-GGUF ，运行的是Deepseek R1 70B ### 环境信息操作系统：Debian GNU/Linux 12 (bookworm) GPU： Tesla P100 CPU： Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz pytorch版本：12.5 cuda版本：11.8

bsakaguchi closed this issue

2026-10-29 14:41:05 +00:00