mirror of
https://github.com/deepseek-ai/DeepSeek-VL2.git
synced 2026-11-05 01:07:23 +00:00
[GH-ISSUE #5] DeepSeek-VL2 can't support old nvidia device like P40 ,1080 , what to do to let this project support these old device? #4
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-VL2#4
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @qmzpg on GitHub (Dec 15, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/5
Add pad token = ['<｜▁pad▁｜>'] to the tokenizer
<｜▁pad▁｜>:2
Add image token = [''] to the tokenizer
:128815
Add grounding-related tokens = ['<|ref|>', '<|/ref|>', '<|det|>', '<|/det|>', '<|grounding|>'] to the tokenizer with input_ids
<|ref|>:128816
<|/ref|>:128817
<|det|>:128818
<|/det|>:128819
<|grounding|>:128820
Add chat tokens = ['<|User|>', '<|Assistant|>'] to the tokenizer with input_ids
<|User|>:128821
<|Assistant|>:128822
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:58<00:00, 7.29s/it]
You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the
__call__method is faster than using a method to encode the text followed by a call to thepadmethod to get a padded encoding.Traceback (most recent call last):
File "/home/baochaoqian/dnn/test/llm/DeepSeek-VL2-main/test001.py", line 54, in
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)
File "/home/baochaoqian/dnn/test/llm/DeepSeek-VL2-main/deepseek_vl/models/modeling_deepseek_vl_v2.py", line 325, in prepare_inputs_embeds
images_feature = self.vision(total_tiles)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/accelerate/hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/baochaoqian/dnn/test/llm/DeepSeek-VL2-main/deepseek_vl/models/siglip_vit.py", line 548, in forward
x = self.forward_features(x)
File "/home/baochaoqian/dnn/test/llm/DeepSeek-VL2-main/deepseek_vl/models/siglip_vit.py", line 529, in forward_features
x = self.blocks(x)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/accelerate/hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/container.py", line 219, in forward
input = module(input)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/accelerate/hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/baochaoqian/dnn/test/llm/DeepSeek-VL2-main/deepseek_vl/models/siglip_vit.py", line 231, in forward
x = x + self.drop_path1(self.ls1(self.attn(self.norm1(x))))
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/accelerate/hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/baochaoqian/dnn/test/llm/DeepSeek-VL2-main/deepseek_vl/models/siglip_vit.py", line 143, in forward
x = memory_efficient_attention(q, k, v, p=self.attn_drop.p if self.training else 0.)
File "/home/baochaoqian/.local/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 276, in memory_efficient_attention
return _memory_efficient_attention(
File "/home/baochaoqian/.local/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 403, in _memory_efficient_attention
return _fMHA.apply(
File "/home/baochaoqian/fixfolder001/Anaconda/python310/lib/python3.10/site-packages/torch/autograd/function.py", line 574, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/home/baochaoqian/.local/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 74, in forward
out, op_ctx = _memory_efficient_attention_forward_requires_grad(
File "/home/baochaoqian/.local/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 428, in _memory_efficient_attention_forward_requires_grad
op = _dispatch_fw(inp, True)
File "/home/baochaoqian/.local/lib/python3.10/site-packages/xformers/ops/fmha/dispatch.py", line 119, in _dispatch_fw
return _run_priority_list(
File "/home/baochaoqian/.local/lib/python3.10/site-packages/xformers/ops/fmha/dispatch.py", line 55, in _run_priority_list
raise NotImplementedError(msg)
NotImplementedError: No operator found for
memory_efficient_attention_forwardwith inputs:query : shape=(7, 729, 16, 72) (torch.bfloat16)
key : shape=(7, 729, 16, 72) (torch.bfloat16)
value : shape=(7, 729, 16, 72) (torch.bfloat16)
attn_bias : <class 'NoneType'>
p : 0.0
flshattF@2.5.6-ptis not supported because:requires device with capability > (8, 0) but your GPU has capability (6, 1) (too old)
bf16 is only supported on A100+ GPUs
cutlassF-ptis not supported because:bf16 is only supported on A100+ GPUs
smallkFis not supported because:max(query.shape[-1] != value.shape[-1]) > 32
dtype=torch.bfloat16 (supported: {torch.float32})
bf16 is only supported on A100+ GPUs
unsupported embed per head: 72
@ApolloRay commented on GitHub (Dec 16, 2024):
bf16，try in A100 A800
@cnroge commented on GitHub (Feb 11, 2025):
康康pip list