[GH-ISSUE #723] [BUG]DeepSeek-R1-Distill-Qwen-32B is not supported #527

New issue

Closed

opened 2026-10-29 14:40:51 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 14:40:51 +00:00

Owner

Originally created by @yeungtuzi on GitHub (Feb 27, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/723

It seems Qwen requires transformers==4.47+, but ktransformer is using 4.43.2.

[Command]
ktransformers
--model_dir /root/DeepSeek-R1-Distill-Qwen-32B/
--model_path deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
--device 1
--model_name DeepSeek-R1
--fast_safetensors TRUE
...
[Error log]
File "/root/ktransformers/ktransformers/server/backend/interfaces/ktransformers.py", line 37, in init
self.model = custom_modelsconfig.architectures[0]
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'Qwen2ForCausalLM'

Originally created by @yeungtuzi on GitHub (Feb 27, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/723 It seems Qwen requires transformers==4.47+, but ktransformer is using 4.43.2. [Command] ktransformers \ --model_dir /root/DeepSeek-R1-Distill-Qwen-32B/ \ --model_path deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \ --device 1 \ --model_name DeepSeek-R1 \ --fast_safetensors TRUE \ ... [Error log] File "/root/ktransformers/ktransformers/server/backend/interfaces/ktransformers.py", line 37, in __init__ self.model = custom_models[config.architectures[0]](config) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ KeyError: 'Qwen2ForCausalLM'

bsakaguchi

2026-10-29 14:40:51 +00:00

closed this issue
added the
bug

stale
labels

bsakaguchi commented

2026-10-29 14:40:52 +00:00

Author

Owner

@ubergarm commented on GitHub (Feb 27, 2025):

#611

@ubergarm commented on GitHub (Feb 27, 2025): #611

bsakaguchi commented

2026-10-29 14:40:52 +00:00

Author

Owner

@ubergarm commented on GitHub (Feb 28, 2025):

https://github.com/kvcache-ai/ktransformers/issues/642#issuecomment-2679436553

你正在尝试运行的是稠密型Llama/Qwen模型架构，而非真正的R1 MoE模型架构。我建议你尝试使用llama.cpp来运行稠密模型。祝你好运！

@ubergarm commented on GitHub (Feb 28, 2025): https://github.com/kvcache-ai/ktransformers/issues/642#issuecomment-2679436553 你正在尝试运行的是稠密型Llama/Qwen模型架构，而非真正的R1 MoE模型架构。我建议你尝试使用llama.cpp来运行稠密模型。祝你好运！

bsakaguchi commented

2026-10-29 14:40:52 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！