deepseek-ai/ESFT
1
0
Fork 0
mirror of https://github.com/deepseek-ai/ESFT.git synced 2026-11-05 00:57:35 +00:00
Code Issues 7 Projects Packages Wiki Activity

[GH-ISSUE #13] train.py issues #9

New issue
Closed
opened 2026-10-26 21:06:38 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-26 21:06:38 +00:00
Owner
Copy link

Originally created by @y-he2 on GitHub (Jan 23, 2025).
Original GitHub issue: https://github.com/deepseek-ai/ESFT/issues/13

Attempting to run the example command directly raised:

.../site-packages/transformers/modeling_utils.py", line 1747, in _check_and_enable_flash_attn_2
    raise ImportError(f"{preface} the package flash_attn seems to be not installed. {install_message}")
ImportError: FlashAttention2 has been toggled on, but it cannot be used due to the following error: the package flash_attn seems to be not installed. Please refer to the documentation of https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 to install Flash Attention 2.

Perhaps its a good idea to include this into the docs for clarity?

When referred to the HF link:
https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2
and attempted to install it, raised:

  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

This could be a local error on my side, however I indeed could'nt see DeepSeek ESFT being in the list on the Flash Attention page under:
"FlashAttention-2 is currently supported for the following architectures: ...."

Originally created by @y-he2 on GitHub (Jan 23, 2025). Original GitHub issue: https://github.com/deepseek-ai/ESFT/issues/13 Attempting to run the example command directly raised: ``` .../site-packages/transformers/modeling_utils.py", line 1747, in _check_and_enable_flash_attn_2 raise ImportError(f"{preface} the package flash_attn seems to be not installed. {install_message}") ImportError: FlashAttention2 has been toggled on, but it cannot be used due to the following error: the package flash_attn seems to be not installed. Please refer to the documentation of https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 to install Flash Attention 2. ``` Perhaps its a good idea to include this into the docs for clarity? When referred to the HF link: https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 and attempted to install it, raised: ``` Preparing metadata (setup.py) ... error error: subprocess-exited-with-error ``` This could be a local error on my side, however I indeed could'nt see DeepSeek ESFT being in the list on the Flash Attention page under: "FlashAttention-2 is currently supported for the following architectures: ...."
bsakaguchi commented 2026-10-26 21:06:41 +00:00
Author
Owner
Copy link

@yeshsurya commented on GitHub (Feb 4, 2025):

It works with latest flash-attn

<!-- gh-comment-id:2634195938 --> @yeshsurya commented on GitHub (Feb 4, 2025): It works with latest flash-attn
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/ESFT#9
No description provided.