[GH-ISSUE #493] struct.error: unpack requires a buffer of 8 bytes #351

New issue

Closed

opened 2026-10-29 14:38:20 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:38:20 +00:00

Owner

Originally created by @KnightHand on GitHub (Feb 19, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/493

请问有人碰到过这个问题吗？

Traceback (most recent call last):
File "/root/home/infra/ktransformers/./ktransformers/local_chat.py", line 187, in
fire.Fire(local_chat)
File "/root/.miniconda/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.miniconda/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/root/.miniconda/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/home/infra/ktransformers/./ktransformers/local_chat.py", line 111, in local_chat
optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config)
File "/root/home/infra/ktransformers/./ktransformers/optimize/optimize.py", line 126, in optimize_and_load_gguf
gguf_loader=GGUFLoader(gguf_path)
^^^^^^^^^^^^^^^^^^^^^
File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 190, in init
self.load_gguf(f)
File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 210, in load_gguf
info[name] = read_value(f, data_type)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 394, in read_value
return [read_value(f, elem_type) for _ in range(count)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 394, in
return [read_value(f, elem_type) for _ in range(count)]
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 356, in read_value
length = struct.unpack("<Q", f.read(8))[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: unpack requires a buffer of 8 bytes

命令很基础：
numactl -N 1 -m 1 python ./ktransformers/local_chat.py --model_path ~/llm/model/DeepSeek-R1 --gguf_path ~/models/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M --optimize_rule_path ./ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu.yaml --cpu_infer 64 --max_new_tokens 10000

Originally created by @KnightHand on GitHub (Feb 19, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/493 请问有人碰到过这个问题吗？ Traceback (most recent call last): File "/root/home/infra/ktransformers/./ktransformers/local_chat.py", line 187, in <module> fire.Fire(local_chat) File "/root/.miniconda/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 135, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.miniconda/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/root/.miniconda/envs/ktransformers/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/root/home/infra/ktransformers/./ktransformers/local_chat.py", line 111, in local_chat optimize_and_load_gguf(model, optimize_rule_path, gguf_path, config) File "/root/home/infra/ktransformers/./ktransformers/optimize/optimize.py", line 126, in optimize_and_load_gguf gguf_loader=GGUFLoader(gguf_path) ^^^^^^^^^^^^^^^^^^^^^ File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 190, in __init__ self.load_gguf(f) File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 210, in load_gguf info[name] = read_value(f, data_type) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 394, in read_value return [read_value(f, elem_type) for _ in range(count)] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 394, in <listcomp> return [read_value(f, elem_type) for _ in range(count)] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/home/infra/ktransformers/./ktransformers/util/custom_gguf.py", line 356, in read_value length = struct.unpack("<Q", f.read(8))[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ struct.error: unpack requires a buffer of 8 bytes 命令很基础： `numactl -N 1 -m 1 python ./ktransformers/local_chat.py --model_path ~/llm/model/DeepSeek-R1 --gguf_path ~/models/DeepSeek-R1-GGUF/DeepSeek-R1-Q4_K_M --optimize_rule_path ./ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu.yaml --cpu_infer 64 --max_new_tokens 10000`

bsakaguchi

2026-10-29 14:38:20 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:38:21 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！