deepseek-ai/Janus
1
0
Fork 0
mirror of https://github.com/deepseek-ai/Janus.git synced 2026-11-05 00:57:18 +00:00
Code Issues 183 Projects Packages Wiki Activity

[GH-ISSUE #185] ValueError: The checkpoint you are trying to load has model type multi_modality but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is o… #148

New issue
Open
opened 2026-10-26 21:13:03 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-26 21:13:03 +00:00
Owner
Copy link

Originally created by @xem123 on GitHub (Feb 25, 2025).
Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/185

please help me，I want to fine-tune the Janus-Pro-7B model. The model itself works correctly, but after installing ms-swift, when I run the command:

Bash
swift sft --model Janus-pro-7Bmodel --dataset data.jsonl --train_type lora --torch_dtype bfloat16 --num_train_epochs 5 --per_device_train_batch_size 1 --per_device_eval_batch_size 1 --learning_rate 1e-4 --lora_rank 8 --lora_alpha 32 --target_modules all-linear --freeze_vit true
it throws an error:

ValueError: The checkpoint you are trying to load has model type 'multi_modality' but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

My current configuration/software versions are as follows:
Image

Image

Image

Originally created by @xem123 on GitHub (Feb 25, 2025). Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/185 please help me，I want to fine-tune the Janus-Pro-7B model. The model itself works correctly, but after installing ms-swift, when I run the command: Bash swift sft --model Janus-pro-7Bmodel --dataset data.jsonl --train_type lora --torch_dtype bfloat16 --num_train_epochs 5 --per_device_train_batch_size 1 --per_device_eval_batch_size 1 --learning_rate 1e-4 --lora_rank 8 --lora_alpha 32 --target_modules all-linear --freeze_vit true it throws an error: ValueError: The checkpoint you are trying to load has model type 'multi_modality' but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date. My current configuration/software versions are as follows: ![Image](https://github.com/user-attachments/assets/dbc9a916-11cd-4e14-9b61-486b90bd4ad5) ![Image](https://github.com/user-attachments/assets/7d2e04e7-055a-405e-8383-5669389c517a) ![Image](https://github.com/user-attachments/assets/080d6f7c-bce0-4693-acfa-64c408a4afd2)
bsakaguchi commented 2026-10-26 21:13:04 +00:00
Author
Owner
Copy link

@tarunn90 commented on GitHub (Apr 4, 2025):

I have also encountered the same error when running the following:

pip install transformers

model_name = "deepseek-ai/Janus-Pro-7B"

# Load model and processor
processor = AutoProcessor.from_pretrained(model_name)
model = AutoModelForVision2Seq.from_pretrained(model_name)
<!-- gh-comment-id:2779101013 --> @tarunn90 commented on GitHub (Apr 4, 2025): I have also encountered the same error when running the following: ``` pip install transformers model_name = "deepseek-ai/Janus-Pro-7B" # Load model and processor processor = AutoProcessor.from_pretrained(model_name) model = AutoModelForVision2Seq.from_pretrained(model_name) ```
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/Janus#148
No description provided.