[GH-ISSUE #1273] [Feature] 求支持 DeepSeek-Prover-V2-671B #862

New issue

Open

opened 2026-10-29 14:45:31 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 14:45:31 +00:00

Owner

Originally created by @wuhongyewhy on GitHub (May 8, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1273

检查清单

1. 如果您提出的不是新功能而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
2. 为方便社区交流，我将使用中文/英文或附上英文/中文翻译（如使用其他语言）。未附带翻译的非英文/中文内容可能会被关闭

需求背景

求支持模型 unsloth/DeepSeek-Prover-V2-671B-GGUF 根据官方说明，DeepSeek-Prover-V2-671B is trained on top of DeepSeek-V3-Base. DeepSeek-Prover-V2-671B shares the same architecture as DeepSeek-V3. 应该支持prover-V2不用太多的代码吧。我现在用V3的设置运行prover-V2,错误如下：

......
Injecting model.norm as default
Injecting lm_head as ktransformers.operators.linear . KTransformersLinear
loading token_embd.weight to cpu
loading token_embd.weight with CPU
loading blk.0.attn_q_a_norm.weight to cuda:0
loading blk.0.attn_kv_a_norm.weight to cuda:0
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/wuhy/deepseek/ktransformers/ktransformers/local_chat.py", line 185, in
fire.Fire(local_chat)
......
这里省略一些内容
......
File "/home/wuhy/deepseek/ktransformers/ktransformers/util/utils.py", line 123, in load_weights
load_cur_state_dict(module, gguf_loader, prefix)
File "/home/wuhy/deepseek/ktransformers/ktransformers/util/utils.py", line 118, in load_cur_state_dict
raise Exception(f"can't find {translated_key} in GGUF file!")
Exception: can't find blk.0.attn_kv_b.weight in GGUF file!

看上面好像是 prover-V2的模型内部命名有些改变？我对这个不懂，帮不上忙改代码支持prover-V2. 希望各位牛人，如果不用太多代码的情况下支持prover-V2.

相关资源

No response

Originally created by @wuhongyewhy on GitHub (May 8, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1273 ### 检查清单 - [x] 1. 如果您提出的不是新功能而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [x] 2. 为方便社区交流，我将使用中文/英文或附上英文/中文翻译（如使用其他语言）。未附带翻译的非英文/中文内容可能会被关闭 ### 需求背景求支持模型 unsloth/DeepSeek-Prover-V2-671B-GGUF 根据官方说明，DeepSeek-Prover-V2-671B is trained on top of DeepSeek-V3-Base. DeepSeek-Prover-V2-671B shares the same architecture as DeepSeek-V3. 应该支持prover-V2不用太多的代码吧。我现在用V3的设置运行prover-V2,错误如下： ...... Injecting model.norm as default Injecting lm_head as ktransformers.operators.linear . KTransformersLinear loading token_embd.weight to cpu loading token_embd.weight with CPU loading blk.0.attn_q_a_norm.weight to cuda:0 loading blk.0.attn_kv_a_norm.weight to cuda:0 Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/home/wuhy/deepseek/ktransformers/ktransformers/local_chat.py", line 185, in <module> fire.Fire(local_chat) ...... 这里省略一些内容 ...... File "/home/wuhy/deepseek/ktransformers/ktransformers/util/utils.py", line 123, in load_weights load_cur_state_dict(module, gguf_loader, prefix) File "/home/wuhy/deepseek/ktransformers/ktransformers/util/utils.py", line 118, in load_cur_state_dict raise Exception(f"can't find {translated_key} in GGUF file!") Exception: can't find blk.0.attn_kv_b.weight in GGUF file! 看上面好像是 prover-V2的模型内部命名有些改变？我对这个不懂，帮不上忙改代码支持prover-V2. 希望各位牛人，如果不用太多代码的情况下支持prover-V2. ### 相关资源 _No response_

bsakaguchi commented

2026-10-29 14:45:32 +00:00

Author

Owner

@Atream commented on GitHub (May 9, 2025):

prover-V2的结构和V3/R1完全一致，应该是GGUF中的某些变量名被修改了，可以查看下GGUF中有没有类似的Tensor，和V3/R1的GGUF（或者老版本的GGUF）对比下Tensor的名字

@Atream commented on GitHub (May 9, 2025): prover-V2的结构和V3/R1完全一致，应该是GGUF中的某些变量名被修改了，可以查看下GGUF中有没有类似的Tensor，和V3/R1的GGUF（或者老版本的GGUF）对比下Tensor的名字

bsakaguchi commented

2026-10-29 14:45:32 +00:00

Author

Owner

@wuhongyewhy commented on GitHub (May 12, 2025):

输出看了一下，unsloth量化后prover有14层，V3和R1有16层。完全不会运行prover啊。大家有相同需要的吗？顶起来。

@wuhongyewhy commented on GitHub (May 12, 2025): 输出看了一下，unsloth量化后prover有14层，V3和R1有16层。完全不会运行prover啊。大家有相同需要的吗？顶起来。

bsakaguchi commented

2026-10-29 14:45:32 +00:00

Author

Owner

@igerry commented on GitHub (May 31, 2025):