[GH-ISSUE #116] The deepseek-vl2-tiny gets the wrong output #95

New issue

Open

opened 2026-10-26 21:11:52 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:11:52 +00:00

Owner

Originally created by @MRYUjg on GitHub (Mar 15, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/116

`import torch
from transformers import AutoModelForCausalLM

from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM
from deepseek_vl2.utils.io import load_pil_images

specify the path to the model

model_path = "/whut_data/Yujg/DeepSeek-VL2/deepseek-vl2-tiny"
vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer

vl_gpt = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, torch_dtype=torch.bfloat16)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()

single image conversation example

conversation = [
{
"role": "<|User|>",
"content": "\nDescribe the image.",
"images": ["/whut_data/Yujg/DeepSeek-VL2/images/visual_grounding_1.jpeg"],
},
{"role": "<|Assistant|>", "content": ""},
]

load images and prepare for inputs

pil_images = load_pil_images(conversation)
prepare_inputs = vl_chat_processor(
conversations=conversation,
images=pil_images,
force_batchify=True,
system_prompt=""
).to(vl_gpt.device)

run image encoder to get the image embeddings

inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)

run the model to get the response

outputs = vl_gpt.language_model.generate(
inputs_embeds=inputs_embeds,
attention_mask=prepare_inputs.attention_mask,
pad_token_id=tokenizer.eos_token_id,
bos_token_id=tokenizer.bos_token_id,
eos_token_id=tokenizer.eos_token_id,
max_new_tokens=512,
do_sample=False,
use_cache=True
)

answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=True)
print(f"{prepare_inputs['sft_format'][0]}", answer)`

and I get this:
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Add pad token = ['<｜▁pad▁｜>'] to the tokenizer <｜▁pad▁｜>:2 Add image token = ['<image>'] to the tokenizer <image>:128815 Add grounding-related tokens = ['<|ref|>', '<|/ref|>', '<|det|>', '<|/det|>', '<|grounding|>'] to the tokenizer with input_ids <|ref|>:128816 <|/ref|>:128817 <|det|>:128818 <|/det|>:128819 <|grounding|>:128820 Add chat tokens = ['<|User|>', '<|Assistant|>'] to the tokenizer with input_ids <|User|>:128821 <|Assistant|>:128822

I can't find a way to fix it!!

Originally created by @MRYUjg on GitHub (Mar 15, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/116 `import torch from transformers import AutoModelForCausalLM from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM from deepseek_vl2.utils.io import load_pil_images # specify the path to the model model_path = "/whut_data/Yujg/DeepSeek-VL2/deepseek-vl2-tiny" vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path) tokenizer = vl_chat_processor.tokenizer vl_gpt = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, torch_dtype=torch.bfloat16) vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval() ## single image conversation example conversation = [ { "role": "<|User|>", "content": "<image>\nDescribe the image.", "images": ["/whut_data/Yujg/DeepSeek-VL2/images/visual_grounding_1.jpeg"], }, {"role": "<|Assistant|>", "content": ""}, ] # load images and prepare for inputs pil_images = load_pil_images(conversation) prepare_inputs = vl_chat_processor( conversations=conversation, images=pil_images, force_batchify=True, system_prompt="" ).to(vl_gpt.device) # run image encoder to get the image embeddings inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs) # run the model to get the response outputs = vl_gpt.language_model.generate( inputs_embeds=inputs_embeds, attention_mask=prepare_inputs.attention_mask, pad_token_id=tokenizer.eos_token_id, bos_token_id=tokenizer.bos_token_id, eos_token_id=tokenizer.eos_token_id, max_new_tokens=512, do_sample=False, use_cache=True ) answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=True) print(f"{prepare_inputs['sft_format'][0]}", answer)` and I get this: `Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Add pad token = ['<｜▁pad▁｜>'] to the tokenizer <｜▁pad▁｜>:2 Add image token = ['<image>'] to the tokenizer <image>:128815 Add grounding-related tokens = ['<|ref|>', '<|/ref|>', '<|det|>', '<|/det|>', '<|grounding|>'] to the tokenizer with input_ids <|ref|>:128816 <|/ref|>:128817 <|det|>:128818 <|/det|>:128819 <|grounding|>:128820 Add chat tokens = ['<|User|>', '<|Assistant|>'] to the tokenizer with input_ids <|User|>:128821 <|Assistant|>:128822` I can't find a way to fix it!!

bsakaguchi referenced this issue

2026-10-26 21:12:37 +00:00

[PR #95] Fix the retry function error by remove the incorrect chunk_size parameter #141