deepseek-ai/DeepSeek-VL2
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-VL2.git synced 2026-11-05 01:07:23 +00:00
Code Issues 123 Projects Packages Wiki Activity

[GH-ISSUE #119] bf16 is only supported on A100+ GPUs，how can i use torch.float16 #97

New issue
Open
opened 2026-10-26 21:11:54 +00:00 by bsakaguchi · 6 comments
bsakaguchi commented 2026-10-26 21:11:54 +00:00
Owner
Copy link

Originally created by @Nick280375880 on GitHub (Mar 20, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/119

raise NotImplementedError(msg)
NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(7, 729, 16, 72) (torch.bfloat16)
key : shape=(7, 729, 16, 72) (torch.bfloat16)
value : shape=(7, 729, 16, 72) (torch.bfloat16)
attn_bias : <class 'NoneType'>
p : 0.0
fa3F@0.0.0 is not supported because:
requires device with capability > (9, 0) but your GPU has capability (7, 5) (too old)
bf16 is only supported on A100+ GPUs
operator wasn't built - see python -m xformers.info for more info
only head-dim 64,128,256 is supported
fa2F@v2.7.2.post1 is not supported because:
requires device with capability > (8, 0) but your GPU has capability (7, 5) (too old)
bf16 is only supported on A100+ GPUs
cutlassF-pt is not supported because:
bf16 is only supported on A100+ GPUs

Originally created by @Nick280375880 on GitHub (Mar 20, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/119 raise NotImplementedError(msg) NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs: query : shape=(7, 729, 16, 72) (torch.bfloat16) key : shape=(7, 729, 16, 72) (torch.bfloat16) value : shape=(7, 729, 16, 72) (torch.bfloat16) attn_bias : <class 'NoneType'> p : 0.0 `fa3F@0.0.0` is not supported because: requires device with capability > (9, 0) but your GPU has capability (7, 5) (too old) bf16 is only supported on A100+ GPUs operator wasn't built - see `python -m xformers.info` for more info only head-dim 64,128,256 is supported `fa2F@v2.7.2.post1` is not supported because: requires device with capability > (8, 0) but your GPU has capability (7, 5) (too old) bf16 is only supported on A100+ GPUs `cutlassF-pt` is not supported because: bf16 is only supported on A100+ GPUs
bsakaguchi commented 2026-10-26 21:11:56 +00:00
Author
Owner
Copy link

@caligoDan commented on GitHub (Mar 22, 2025):

I fixed this, adding a pr

<!-- gh-comment-id:2745016125 --> @caligoDan commented on GitHub (Mar 22, 2025): I fixed this, adding a pr
bsakaguchi commented 2026-10-26 21:11:56 +00:00
Author
Owner
Copy link

@zsc1220 commented on GitHub (Mar 25, 2025):

the same problem, how to solve this ?

<!-- gh-comment-id:2750170757 --> @zsc1220 commented on GitHub (Mar 25, 2025): the same problem, how to solve this ?
bsakaguchi commented 2026-10-26 21:11:56 +00:00
Author
Owner
Copy link

@caop-kie commented on GitHub (Mar 29, 2025):

I got the same issue with V100. Using RTX 40-series GPU fixed the issue for me.

<!-- gh-comment-id:2764225181 --> @caop-kie commented on GitHub (Mar 29, 2025): I got the same issue with V100. Using RTX 40-series GPU fixed the issue for me.
bsakaguchi commented 2026-10-26 21:11:57 +00:00
Author
Owner
Copy link

@caligoDan commented on GitHub (Mar 29, 2025):

Jus use Qwen

<!-- gh-comment-id:2764233734 --> @caligoDan commented on GitHub (Mar 29, 2025): Jus use Qwen
bsakaguchi commented 2026-10-26 21:11:57 +00:00
Author
Owner
Copy link

@xgx10010 commented on GitHub (Apr 8, 2025):

modify the dtype = torch.bfloat16 into dtype = torch.float32 in inference.py, try it

<!-- gh-comment-id:2785055441 --> @xgx10010 commented on GitHub (Apr 8, 2025): modify the dtype = torch.bfloat16 into dtype = torch.float32 in inference.py, try it
bsakaguchi commented 2026-10-26 21:11:57 +00:00
Author
Owner
Copy link

@Entr0opsy commented on GitHub (Jul 11, 2025):

Yes this has worked for me!

<!-- gh-comment-id:3061622057 --> @Entr0opsy commented on GitHub (Jul 11, 2025): Yes this has worked for me!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-VL2#97
No description provided.