[GH-ISSUE #23] About image to image. #16

New issue

Closed

opened 2026-10-26 21:10:05 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:10:05 +00:00

Owner

Originally created by @brayevalerien on GitHub (Oct 28, 2024).
Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/23

First of all thank you for releasing the weights and the inference code of Janus alongside the paper.

The model design seems to easily allow for img2img generation, where the user gives Janus a text prompt and one (or more) images and Janus generate an image in return.

Could you please add a small example of how to do that in the readme if Janus supports it? I've tried merging the visual understanding and image generation codes with no success but I'm probably missing something.

Documenting the janus library could help future users too.

Originally created by @brayevalerien on GitHub (Oct 28, 2024). Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/23 First of all thank you for releasing the weights and the inference code of Janus alongside the paper. The model design seems to easily allow for img2img generation, where the user gives Janus a text prompt and one (or more) images and Janus generate an image in return. Could you please add a small example of how to do that in the readme if Janus supports it? I've tried merging the visual understanding and image generation codes with no success but I'm probably missing something. Documenting the `janus` library could help future users too.

bsakaguchi closed this issue

2026-10-26 21:10:06 +00:00

bsakaguchi commented

2026-10-26 21:10:07 +00:00

Author

Owner

@charlesCXK commented on GitHub (Oct 30, 2024):

Thank you for your attention!
In this version, Janus does not utilize image-to-image generation data and is focused solely on the basic text-to-image ability. If you want to implement this feature, you need to include the relevant data and train a new model. When training the model for image-to-image generation, we think it might be better to use the Generation Encoder to encode the reference image as well.

@charlesCXK commented on GitHub (Oct 30, 2024): Thank you for your attention! In this version, Janus does not utilize image-to-image generation data and is focused solely on the basic text-to-image ability. If you want to implement this feature, you need to include the relevant data and train a new model. When training the model for image-to-image generation, we think it might be better to use the Generation Encoder to encode the reference image as well.

bsakaguchi commented

2026-10-26 21:10:07 +00:00

Author

Owner

@brayevalerien commented on GitHub (Oct 30, 2024):

Thanks a lot @charlesCXK for your answer.

After reading the paper I thought end-to-end training for txt+img to txt+img had been done, enabling any combination of these two modalities for both understanding and generation, my bad.

Fine-tuning could be possible to do for us but full training is not really an option unfortunatly.

Can't wait to see where we will be two papers down the line!

@brayevalerien commented on GitHub (Oct 30, 2024): Thanks a lot @charlesCXK for your answer. After reading the paper I thought end-to-end training for txt+img to txt+img had been done, enabling any combination of these two modalities for both understanding and generation, my bad. Fine-tuning could be possible to do for us but full training is not really an option unfortunatly. Can't wait to see where we will be two papers down the line!

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/Janus#16

No description provided.

Rows
Columns