deepseek-ai/DeepSeek-VL
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-VL.git synced 2026-11-05 00:47:24 +00:00
Code Issues 49 Projects Packages Wiki Activity

[GH-ISSUE #11] Support for M1 Mac, or non-cuda devices #9

New issue
Open
opened 2026-10-26 21:08:45 +00:00 by bsakaguchi · 3 comments
bsakaguchi commented 2026-10-26 21:08:45 +00:00
Owner
Copy link

Originally created by @mattkanwisher on GitHub (Mar 12, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/11

Seems like parts of the code explicitly call cuda functions, so you can't just switch it to MPS for Macs. Any roadmap for supporting Macs?

Io.py

def load_pretrained_model(model_path: str):
    vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path)
    tokenizer = vl_chat_processor.tokenizer

    vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(
        model_path, trust_remote_code=True
    )
    vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()

error

  File "/Users/user/projects/ai/DeepSeek-VL/deepseek_vl/utils/io.py", line 37, in load_pretrained_model
    vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2528, in cuda
    return super().cuda(*args, **kwargs)
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 911, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply
    module._apply(fn)
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply
    module._apply(fn)
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply
    module._apply(fn)
  [Previous line repeated 2 more times]
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 825, in _apply
    param_applied = fn(param)
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 911, in <lambda>
    return self._apply(lambda t: t.cuda(device))
  File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
Originally created by @mattkanwisher on GitHub (Mar 12, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/11 Seems like parts of the code explicitly call cuda functions, so you can't just switch it to MPS for Macs. Any roadmap for supporting Macs? Io.py ```python def load_pretrained_model(model_path: str): vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path) tokenizer = vl_chat_processor.tokenizer vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True ) vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval() ``` error ``` File "/Users/user/projects/ai/DeepSeek-VL/deepseek_vl/utils/io.py", line 37, in load_pretrained_model vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval() File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2528, in cuda return super().cuda(*args, **kwargs) File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 911, in cuda return self._apply(lambda t: t.cuda(device)) File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply module._apply(fn) File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply module._apply(fn) File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 802, in _apply module._apply(fn) [Previous line repeated 2 more times] File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 825, in _apply param_applied = fn(param) File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 911, in <lambda> return self._apply(lambda t: t.cuda(device)) File "/Users/user/miniconda3/envs/py39/lib/python3.9/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled ```
bsakaguchi commented 2026-10-26 21:08:46 +00:00
Author
Owner
Copy link

@soloice commented on GitHub (Mar 13, 2024):

We don't run it on Mac internally, so nobody is doing this. But I guess this should be easy and just need a little bit coding. We would appreciate it if you can do this and make a pull request.

<!-- gh-comment-id:1993637122 --> @soloice commented on GitHub (Mar 13, 2024): We don't run it on Mac internally, so nobody is doing this. But I guess this should be easy and just need a little bit coding. We would appreciate it if you can do this and make a pull request.
bsakaguchi commented 2026-10-26 21:08:46 +00:00
Author
Owner
Copy link

@AbeEstrada commented on GitHub (Mar 14, 2024):

This PR https://github.com/deepseek-ai/DeepSeek-VL/pull/22

<!-- gh-comment-id:1997807814 --> @AbeEstrada commented on GitHub (Mar 14, 2024): This PR https://github.com/deepseek-ai/DeepSeek-VL/pull/22
bsakaguchi commented 2026-10-26 21:08:46 +00:00
Author
Owner
Copy link

@mattkanwisher commented on GitHub (Mar 17, 2024):

confirmed #22 makes it run now, albeit slowly, some of the ops are still unsupported by mps by PyTorch but fall back to cpu

<!-- gh-comment-id:2002405691 --> @mattkanwisher commented on GitHub (Mar 17, 2024): confirmed #22 makes it run now, albeit slowly, some of the ops are still unsupported by mps by PyTorch but fall back to cpu
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
enhancement

   
environment

   
pull-request
Mirrored from GitHub Pull Request

No labels
enhancement
environment
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-VL#9
No description provided.