deepseek-ai/Janus
1
0
Fork 0
mirror of https://github.com/deepseek-ai/Janus.git synced 2026-11-05 00:57:18 +00:00
Code Issues 183 Projects Packages Wiki Activity

[GH-ISSUE #219] New Benchmark for Unified Evaluation #178

New issue
Open
opened 2026-10-26 21:13:40 +00:00 by bsakaguchi · 0 comments
bsakaguchi commented 2026-10-26 21:13:40 +00:00
Owner
Copy link

Originally created by @Eli-YiLi on GitHub (May 18, 2025).
Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/219

We would like to express our gratitude for the outstanding contributions of the DeepSeek Janus series. In our latest research, we have developed a benchmark specifically designed for unified models, where Janus-Pro-7B ranks first.

The proposed evaluation framework, UniEval, is capable of simultaneously evaluating both understanding and generation tasks with high diversity and difficulty compared to existing text-image generation benchmarks.

We promote this benchmark here, hoping to provide better evaluations for stronger models in the future and inspire improvements in related models.

The project page is: https://xmed-lab.github.io/UniEval/
The code link is: https://github.com/xmed-lab/UniEval

Originally created by @Eli-YiLi on GitHub (May 18, 2025). Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/219 We would like to express our gratitude for the outstanding contributions of the DeepSeek Janus series. In our latest research, we have developed a benchmark specifically designed for unified models, where Janus-Pro-7B ranks first. The proposed evaluation framework, UniEval, is capable of simultaneously evaluating both understanding and generation tasks with high diversity and difficulty compared to existing text-image generation benchmarks. We promote this benchmark here, hoping to provide better evaluations for stronger models in the future and inspire improvements in related models. The project page is: https://xmed-lab.github.io/UniEval/ The code link is: https://github.com/xmed-lab/UniEval
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request
Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/Janus#178
No description provided.