deepseek-ai/DeepSeek-VL
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-VL.git synced 2026-11-05 00:47:24 +00:00
Code Issues 49 Projects Packages Wiki Activity

[GH-ISSUE #37] error when i run inference.py #27

New issue
Closed
opened 2026-10-26 21:08:55 +00:00 by bsakaguchi · 6 comments
bsakaguchi commented 2026-10-26 21:08:55 +00:00
Owner
Copy link

Originally created by @Yangfan-96 on GitHub (Mar 26, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/37

Traceback (most recent call last):
File "inference.py", line 56, in
outputs = vl_gpt.language_model.generate(
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/generation/utils.py", line 1527, in generate
result = self._greedy_search(
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/generation/utils.py", line 2411, in _greedy_search
outputs = self(
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1196, in forward
outputs = self.model(
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 990, in forward
causal_mask = self._update_causal_mask(attention_mask, inputs_embeds, cache_position)
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1076, in _update_causal_mask
causal_mask = torch.triu(causal_mask, diagonal=1)
RuntimeError: "triu_tril_cuda_template" not implemented for 'BFloat16'

python=3.8
accelerate 0.28.0
aiofiles 23.2.1
altair 5.2.0
annotated-types 0.6.0
anyio 4.3.0
attrdict 2.0.1
attrs 23.2.0
Brotli 1.0.9
certifi 2024.2.2
charset-normalizer 2.0.4
click 8.1.7
colorama 0.4.5
contourpy 1.1.1
cycler 0.12.1
deepseek_vl 1.0.0 /app
einops 0.7.0
exceptiongroup 1.2.0
fastapi 0.110.0
ffmpy 0.3.2
filelock 3.13.1
fonttools 4.50.0
fsspec 2024.3.1
gmpy2 2.1.2
gradio 3.48.0
gradio_client 0.6.1
h11 0.14.0
httpcore 1.0.4
httpx 0.27.0
huggingface-hub 0.22.0
idna 3.4
importlib_metadata 7.1.0
importlib_resources 6.4.0
Jinja2 3.1.3
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
latex2mathml 3.77.0
Markdown 3.4.1
MarkupSafe 2.1.3
matplotlib 3.7.5
mdtex2html 1.3.0
mkl-fft 1.3.8
mkl-random 1.2.4
mkl-service 2.4.0
mpmath 1.3.0
networkx 3.1
numpy 1.24.3
orjson 3.9.15
packaging 24.0
pandas 2.0.3
pillow 10.2.0
pip 23.3.1
pkgutil_resolve_name 1.3.10
psutil 5.9.8
pydantic 2.6.4
pydantic_core 2.16.3
pydub 0.25.1
Pygments 2.12.0
pyparsing 3.1.2
pypinyin 0.50.0
PySocks 1.7.1
python-dateutil 2.9.0.post0
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.1
referencing 0.34.0
regex 2023.12.25
requests 2.31.0
rpds-py 0.18.0
safetensors 0.4.2
semantic-version 2.10.0
sentencepiece 0.1.96
setuptools 68.2.2
six 1.16.0
sniffio 1.3.1
starlette 0.36.3
sympy 1.12
tiktoken 0.5.2
timm 0.9.16
tokenizers 0.15.2
toolz 0.12.1
torch 2.0.1
torchaudio 2.0.2
torchvision 0.15.2
tqdm 4.64.0
transformers 4.39.1
triton 2.0.0
typing_extensions 4.9.0
tzdata 2024.1
urllib3 2.1.0
uvicorn 0.29.0
websockets 11.0.3
wheel 0.41.2
zipp 3.18.1

Originally created by @Yangfan-96 on GitHub (Mar 26, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/37 Traceback (most recent call last): File "inference.py", line 56, in <module> outputs = vl_gpt.language_model.generate( File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/generation/utils.py", line 1527, in generate result = self._greedy_search( File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/generation/utils.py", line 2411, in _greedy_search outputs = self( File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1196, in forward outputs = self.model( File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 990, in forward causal_mask = self._update_causal_mask(attention_mask, inputs_embeds, cache_position) File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1076, in _update_causal_mask causal_mask = torch.triu(causal_mask, diagonal=1) RuntimeError: "triu_tril_cuda_template" not implemented for 'BFloat16' **python=3.8** accelerate 0.28.0 aiofiles 23.2.1 altair 5.2.0 annotated-types 0.6.0 anyio 4.3.0 attrdict 2.0.1 attrs 23.2.0 Brotli 1.0.9 certifi 2024.2.2 charset-normalizer 2.0.4 click 8.1.7 colorama 0.4.5 contourpy 1.1.1 cycler 0.12.1 deepseek_vl 1.0.0 /app einops 0.7.0 exceptiongroup 1.2.0 fastapi 0.110.0 ffmpy 0.3.2 filelock 3.13.1 fonttools 4.50.0 fsspec 2024.3.1 gmpy2 2.1.2 gradio 3.48.0 gradio_client 0.6.1 h11 0.14.0 httpcore 1.0.4 httpx 0.27.0 huggingface-hub 0.22.0 idna 3.4 importlib_metadata 7.1.0 importlib_resources 6.4.0 Jinja2 3.1.3 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 kiwisolver 1.4.5 latex2mathml 3.77.0 Markdown 3.4.1 MarkupSafe 2.1.3 matplotlib 3.7.5 mdtex2html 1.3.0 mkl-fft 1.3.8 mkl-random 1.2.4 mkl-service 2.4.0 mpmath 1.3.0 networkx 3.1 numpy 1.24.3 orjson 3.9.15 packaging 24.0 pandas 2.0.3 pillow 10.2.0 pip 23.3.1 pkgutil_resolve_name 1.3.10 psutil 5.9.8 pydantic 2.6.4 pydantic_core 2.16.3 pydub 0.25.1 Pygments 2.12.0 pyparsing 3.1.2 pypinyin 0.50.0 PySocks 1.7.1 python-dateutil 2.9.0.post0 python-multipart 0.0.9 pytz 2024.1 PyYAML 6.0.1 referencing 0.34.0 regex 2023.12.25 requests 2.31.0 rpds-py 0.18.0 safetensors 0.4.2 semantic-version 2.10.0 sentencepiece 0.1.96 setuptools 68.2.2 six 1.16.0 sniffio 1.3.1 starlette 0.36.3 sympy 1.12 tiktoken 0.5.2 timm 0.9.16 tokenizers 0.15.2 toolz 0.12.1 torch 2.0.1 torchaudio 2.0.2 torchvision 0.15.2 tqdm 4.64.0 transformers 4.39.1 triton 2.0.0 typing_extensions 4.9.0 tzdata 2024.1 urllib3 2.1.0 uvicorn 0.29.0 websockets 11.0.3 wheel 0.41.2 zipp 3.18.1
bsakaguchi commented 2026-10-26 21:08:57 +00:00
Author
Owner
Copy link

@Yangfan-96 commented on GitHub (Mar 28, 2024):

pip install transformers==4.38.2

<!-- gh-comment-id:2024325492 --> @Yangfan-96 commented on GitHub (Mar 28, 2024): pip install transformers==4.38.2
bsakaguchi commented 2026-10-26 21:08:57 +00:00
Author
Owner
Copy link

@Elfsong commented on GitHub (Apr 15, 2024):

pip install transformers==4.38.2

it works! Thank you!

<!-- gh-comment-id:2056594707 --> @Elfsong commented on GitHub (Apr 15, 2024): > pip install transformers==4.38.2 it works! Thank you!
bsakaguchi commented 2026-10-26 21:08:57 +00:00
Author
Owner
Copy link

@kharanshaik commented on GitHub (May 21, 2024):

Worked, Thank you.!

<!-- gh-comment-id:2121758672 --> @kharanshaik commented on GitHub (May 21, 2024): Worked, Thank you.!
bsakaguchi commented 2026-10-26 21:08:57 +00:00
Author
Owner
Copy link

@ArthurZucker commented on GitHub (May 23, 2024):

https://github.com/meta-llama/llama3/issues/80 torch had breaking changes, that's the problem, not transformers

<!-- gh-comment-id:2126455332 --> @ArthurZucker commented on GitHub (May 23, 2024): https://github.com/meta-llama/llama3/issues/80 torch had breaking changes, that's the problem, not transformers
bsakaguchi commented 2026-10-26 21:08:57 +00:00
Author
Owner
Copy link

@mliuby commented on GitHub (Jul 5, 2024):

pip install transformers==4.38.2

Thanks! It works!!

<!-- gh-comment-id:2211034856 --> @mliuby commented on GitHub (Jul 5, 2024): > pip install transformers==4.38.2 Thanks! It works!!
bsakaguchi commented 2026-10-26 21:08:57 +00:00
Author
Owner
Copy link

@irisguoyf commented on GitHub (Jul 11, 2024):

upgrade to torch 2.2.2 and transformers 4.41.2 can solve this problem.

<!-- gh-comment-id:2221982440 --> @irisguoyf commented on GitHub (Jul 11, 2024): upgrade to torch 2.2.2 and transformers 4.41.2 can solve this problem.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
enhancement

   
environment

   
pull-request
Mirrored from GitHub Pull Request

No labels
enhancement
environment
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-VL#27
No description provided.