deepseek-ai/Janus
1
0
Fork 0
mirror of https://github.com/deepseek-ai/Janus.git synced 2026-11-05 00:57:18 +00:00
Code Issues 183 Projects Packages Wiki Activity

[GH-ISSUE #29] Errors with Attention Mechanisms in Janus Inference Script (FlashAttention 2.0 & _flash_supports_window_size) #24

New issue
Open
opened 2026-10-26 21:10:12 +00:00 by bsakaguchi · 3 comments
bsakaguchi commented 2026-10-26 21:10:12 +00:00
Owner
Copy link

Originally created by @AlanPonnachan on GitHub (Dec 21, 2024).
Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/29

I encountered multiple issues when running the Janus-1.3B model inference script, both with and without enabling FlashAttention. These errors prevent successful execution of the model in a standard environment, such as Google Colab. Below are the details of the issues:

Problem 1: Error Without FlashAttention

When running the inference script without FlashAttention, the following error is raised during the generate function call:
NameError: name '_flash_supports_window_size' is not defined

Problem 2: FlashAttention 2.0 Unsupported

To address the above error, I attempted to install FlashAttention-2 and enable it via the attn_implementation="flash_attention_2" argument. However, this raises the following error:
ValueError: MultiModalityCausalLM does not support Flash Attention 2.0 yet. Please request to add support where the model is hosted, on its model hub page: https://huggingface.co/deepseek-ai/Janus-1.3B/discussions/new

Steps to Reproduce:

  1. Install FlashAttention 2.0 via:
    pip install flash-attn --no-build-isolation
    2.Modify the script to include:
vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(
    model_path, trust_remote_code=True, attn_implementation="flash_attention_2"
)

Thank you for your assistance! Let me know if you require additional details or logs to reproduce the issues.

Originally created by @AlanPonnachan on GitHub (Dec 21, 2024). Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/29 I encountered multiple issues when running the Janus-1.3B model [inference script](https://github.com/deepseek-ai/Janus/blob/main/inference.py), both with and without enabling FlashAttention. These errors prevent successful execution of the model in a standard environment, such as Google Colab. Below are the details of the issues: ### Problem 1: Error Without FlashAttention When running the inference script without FlashAttention, the following error is raised during the generate function call: `NameError: name '_flash_supports_window_size' is not defined` ### Problem 2: FlashAttention 2.0 Unsupported To address the above error, I attempted to install FlashAttention-2 and enable it via the attn_implementation="flash_attention_2" argument. However, this raises the following error: `ValueError: MultiModalityCausalLM does not support Flash Attention 2.0 yet. Please request to add support where the model is hosted, on its model hub page: https://huggingface.co/deepseek-ai/Janus-1.3B/discussions/new` Steps to Reproduce: 1. Install FlashAttention 2.0 via: `pip install flash-attn --no-build-isolation` 2.Modify the script to include: ``` vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True, attn_implementation="flash_attention_2" ) ``` Thank you for your assistance! Let me know if you require additional details or logs to reproduce the issues.
bsakaguchi commented 2026-10-26 21:10:14 +00:00
Author
Owner
Copy link

@HussamCheema commented on GitHub (Jan 28, 2025):

I am getting the following error:

MultiModalityCausalLM does not support Flash Attention 2.0 yet.

<!-- gh-comment-id:2618594480 --> @HussamCheema commented on GitHub (Jan 28, 2025): I am getting the following **error**: MultiModalityCausalLM does not support Flash Attention 2.0 yet.
bsakaguchi commented 2026-10-26 21:10:14 +00:00
Author
Owner
Copy link

@Purshow commented on GitHub (Feb 1, 2025):

我收到以下错误：

MultiModalityCausalLM 尚不支持 Flash Attention 2.0。

hey! I have the same question. Have you done it?

<!-- gh-comment-id:2628867684 --> @Purshow commented on GitHub (Feb 1, 2025): > 我收到以下**错误**： > > MultiModalityCausalLM 尚不支持 Flash Attention 2.0。 hey! I have the same question. Have you done it?
bsakaguchi commented 2026-10-26 21:10:14 +00:00
Author
Owner
Copy link

@OHaiYo-lzy commented on GitHub (Feb 10, 2025):

same error

<!-- gh-comment-id:2647093916 --> @OHaiYo-lzy commented on GitHub (Feb 10, 2025): same error
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/Janus#24
No description provided.