[GH-ISSUE #38] How to convert it to GGUF/GGML for general use? #26

New issue

Open

opened 2026-10-26 21:08:55 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:08:55 +00:00

Owner

Originally created by @YuanfengZhang on GitHub (Mar 27, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/38

Sorry for this dummy question but I did search for some answers and try before.
Using llama.cpp

python ./convert-hf-to-gguf.py \
../../deepseek-vl-7b-chat \
--outtype f16 \
--outfile ../../deepseek-vl-7b-chat/deepseek-v1-7b-chat.gguf

returned

Loading model: deepseek-vl-7b-chat
Traceback (most recent call last):
File "/home/zhangyuanfeng/software/ollama/llm/llama.cpp/./convert-hf-to-gguf.py", line 2099, in
main()
File "/home/zhangyuanfeng/software/ollama/llm/llama.cpp/./convert-hf-to-gguf.py", line 2079, in main
model_class = Model.from_model_architecture(hparams["architectures"][0])
File "/home/zhangyuanfeng/software/ollama/llm/llama.cpp/./convert-hf-to-gguf.py", line 215, in from_model_architecture
raise NotImplementedError(f'Architecture {arch!r} not supported!') from None
NotImplementedError: Architecture 'MultiModalityCausalLM' not supported!

So is there any feasible method? Thx.

Originally created by @YuanfengZhang on GitHub (Mar 27, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/38 Sorry for this dummy question but I did search for some answers and try before. Using llama.cpp ```shell python ./convert-hf-to-gguf.py \ ../../deepseek-vl-7b-chat \ --outtype f16 \ --outfile ../../deepseek-vl-7b-chat/deepseek-v1-7b-chat.gguf ``` returned > Loading model: deepseek-vl-7b-chat Traceback (most recent call last): File "/home/zhangyuanfeng/software/ollama/llm/llama.cpp/./convert-hf-to-gguf.py", line 2099, in <module> main() File "/home/zhangyuanfeng/software/ollama/llm/llama.cpp/./convert-hf-to-gguf.py", line 2079, in main model_class = Model.from_model_architecture(hparams["architectures"][0]) File "/home/zhangyuanfeng/software/ollama/llm/llama.cpp/./convert-hf-to-gguf.py", line 215, in from_model_architecture raise NotImplementedError(f'Architecture {arch!r} not supported!') from None NotImplementedError: Architecture 'MultiModalityCausalLM' not supported! So is there any feasible method? Thx.

bsakaguchi referenced this issue

2026-10-26 21:09:20 +00:00

[PR #26] [MERGED] docs: update readme #68