mirror of
https://github.com/deepseek-ai/DeepSeek-VL2.git
synced 2026-11-05 01:07:23 +00:00
[GH-ISSUE #74] Does not support Flash Attention 2.0 yet #59
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-VL2#59
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @moonriver0922 on GitHub (Feb 11, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/74
Thanks for the great work.
When I f-t tiny model it in my dataset, it shows
ValueError: DeepseekVLV2ForCausalLM does not support Flash Attention 2.0 yet. Please request to add support where the model is hosted, on its model hub page: https://huggingface.co/deepseek-ai/deepseek-vl2-tiny/discussions/new or in the Transformers GitHub repo: https://github.com/huggingface/transformers/issues/newAre there any fine-tuning tutorials available？
@gromtang commented on GitHub (Feb 14, 2025):
I met the same problem