[GH-ISSUE #182] A problem with assistant content #147

New issue

Open

opened 2026-10-26 21:13:02 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:13:02 +00:00

Owner

Originally created by @yanghui1-arch on GitHub (Feb 23, 2025).
Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/182

I can run Janus-Pro-7B demo. However it doesn't work when I try to change the assistant content from "" to "你好。" or "你好.". As long as '.' or '。' exists in the assistant content Janus doesnt work. just output an empty string.
Here my codes.


import torch
from transformers import AutoModelForCausalLM
from janus.models import MultiModalityCausalLM, VLChatProcessor
from janus.utils.io import load_pil_images

# specify the path to the model
model_path = "deepseek-ai/Janus-Pro-7B"
vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer

vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(
    model_path, trust_remote_code=True
)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()


conversation = [
    {
        "role": "<|User|>",
        "content": f"<image_placeholder>\n{tell me what you see.}",
        "images": [image],
    },
    {"role": "<|Assistant|>", "content": "这是你好。"},  # if content is "", janus can work. Otherwise doesnt
]

# load images and prepare for inputs
pil_images = load_pil_images(conversation)
prepare_inputs = vl_chat_processor(
    conversations=conversation, images=pil_images, force_batchify=True
).to(vl_gpt.device)

# # run image encoder to get the image embeddings
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)

# # run the model to get the response
outputs = vl_gpt.language_model.generate(
    inputs_embeds=inputs_embeds,
    attention_mask=prepare_inputs.attention_mask,
    pad_token_id=tokenizer.eos_token_id,
    bos_token_id=tokenizer.bos_token_id,
    eos_token_id=tokenizer.eos_token_id,
    max_new_tokens=512,
    do_sample=False,
    use_cache=True,
)

answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=True)
print(f"{prepare_inputs['sft_format'][0]}", answer)

Originally created by @yanghui1-arch on GitHub (Feb 23, 2025). Original GitHub issue: https://github.com/deepseek-ai/Janus/issues/182 I can run Janus-Pro-7B demo. However it doesn't work when I try to change the assistant content from "" to "你好。" or "你好.". As long as '.' or '。' exists in the assistant content Janus doesnt work. just output an empty string. Here my codes. ```python import torch from transformers import AutoModelForCausalLM from janus.models import MultiModalityCausalLM, VLChatProcessor from janus.utils.io import load_pil_images # specify the path to the model model_path = "deepseek-ai/Janus-Pro-7B" vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path) tokenizer = vl_chat_processor.tokenizer vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True ) vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval() conversation = [ { "role": "<|User|>", "content": f"<image_placeholder>\n{tell me what you see.}", "images": [image], }, {"role": "<|Assistant|>", "content": "这是你好。"}, # if content is "", janus can work. Otherwise doesnt ] # load images and prepare for inputs pil_images = load_pil_images(conversation) prepare_inputs = vl_chat_processor( conversations=conversation, images=pil_images, force_batchify=True ).to(vl_gpt.device) # # run image encoder to get the image embeddings inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs) # # run the model to get the response outputs = vl_gpt.language_model.generate( inputs_embeds=inputs_embeds, attention_mask=prepare_inputs.attention_mask, pad_token_id=tokenizer.eos_token_id, bos_token_id=tokenizer.bos_token_id, eos_token_id=tokenizer.eos_token_id, max_new_tokens=512, do_sample=False, use_cache=True, ) answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=True) print(f"{prepare_inputs['sft_format'][0]}", answer) ```

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/Janus#147

No description provided.

Rows
Columns