[GH-ISSUE #114] 复现InternLM2.5-7B-Chat-1M报错 #66

New issue

Open

opened 2026-10-29 14:33:17 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:33:17 +00:00

Owner

Originally created by @Cherishyt on GitHub (Nov 13, 2024).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/114

Exception: can't find blk.0.attn_kv_a_mqa.weight in GGUF file!
这边复现InternLM2.5-7B-Chat-1M这个模型，但是报上面这个错，blk.0.attn_kv_a_mqa应该是DeepSpeek模型的gguf文件才有的，可以帮忙看下吗？
执行语句python -m ktransformers.local_chat --model_name LlamaForCausalLM --model_path ./InternLM2.5-7B-Chat-1M/internlm2_5-7b-chat-1m --gguf_path ./InternLM2.5-7B-Chat-1M-GGUF --optimize_config_path ./optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml

Originally created by @Cherishyt on GitHub (Nov 13, 2024). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/114 Exception: can't find blk.0.attn_kv_a_mqa.weight in GGUF file!\ 这边复现InternLM2.5-7B-Chat-1M这个模型，但是报上面这个错，blk.0.attn_kv_a_mqa应该是DeepSpeek模型的gguf文件才有的，可以帮忙看下吗？执行语句python -m ktransformers.local_chat --model_name LlamaForCausalLM --model_path ./InternLM2.5-7B-Chat-1M/internlm2_5-7b-chat-1m --gguf_path ./InternLM2.5-7B-Chat-1M-GGUF --optimize_config_path ./optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml

bsakaguchi commented

2026-10-29 14:33:18 +00:00

Author

Owner

@KMSorSMS commented on GitHub (Feb 11, 2025):

Exception: can't find blk.0.attn_kv_a_mqa.weight in GGUF file! 这边复现InternLM2.5-7B-Chat-1M这个模型，但是报上面这个错，blk.0.attn_kv_a_mqa应该是DeepSpeek模型的gguf文件才有的，可以帮忙看下吗？执行语句python -m ktransformers.local_chat --model_name LlamaForCausalLM --model_path ./InternLM2.5-7B-Chat-1M/internlm2_5-7b-chat-1m --gguf_path ./InternLM2.5-7B-Chat-1M-GGUF --optimize_config_path ./optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml

能提供更详细的报错信息吗

@KMSorSMS commented on GitHub (Feb 11, 2025): > Exception: can't find blk.0.attn_kv_a_mqa.weight in GGUF file! 这边复现InternLM2.5-7B-Chat-1M这个模型，但是报上面这个错，blk.0.attn_kv_a_mqa应该是DeepSpeek模型的gguf文件才有的，可以帮忙看下吗？执行语句python -m ktransformers.local_chat --model_name LlamaForCausalLM --model_path ./InternLM2.5-7B-Chat-1M/internlm2_5-7b-chat-1m --gguf_path ./InternLM2.5-7B-Chat-1M-GGUF --optimize_config_path ./optimize/optimize_rules/Internlm2_5-7b-Chat-1m.yaml 能提供更详细的报错信息吗

bsakaguchi commented

2026-10-29 14:33:18 +00:00

Author

Owner

@ayttop commented on GitHub (Jun 15, 2025):

https://github.com/kvcache-ai/ktransformers/issues/1390

how run InternLM2.5-7B-Chat-1M in ktransfrmerrrs?

@ayttop commented on GitHub (Jun 15, 2025): https://github.com/kvcache-ai/ktransformers/issues/1390 how run InternLM2.5-7B-Chat-1M in ktransfrmerrrs?

bsakaguchi referenced this issue

2026-10-29 14:51:34 +00:00

[PR #67] [MERGED] [fix] fix bugs about Qwen2-57B, install requirement, DockerFile #1341