mirror of
https://github.com/deepseek-ai/DeepSeek-VL2.git
synced 2026-11-05 01:07:23 +00:00
[GH-ISSUE #64] Unable to get the expected answer #50
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-VL2#50
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @SherlockHolmes221 on GitHub (Feb 7, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/64
get the answer, the boxes result is wrong:
`
Python version is above 3.10, patching the collections module.
weights/deepseek-vl2-small
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Add pad token = ['<｜▁pad▁｜>'] to the tokenizer
<｜▁pad▁｜>:100002
Add image token = [''] to the tokenizer
:100003
Add grounding-related tokens = ['<|ref|>', '<|/ref|>', '<|det|>', '<|/det|>', '<|grounding|>'] to the tokenizer with input_ids
<|ref|>:100004
<|/ref|>:100005
<|det|>:100006
<|/det|>:100007
<|grounding|>:100008
Add chat tokens = ['<|User|>', '<|Assistant|>'] to the tokenizer with input_ids
<|User|>:100009
<|Assistant|>:100010
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:07<00:00, 1.76s/it]
You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the
__call__method is faster than using a method to encode the text followed by a call to thepadmethod to get a padded encoding.<|User|>:
<|ref|>The giraffe at the back.<|/ref|>.
<|Assistant|>: <|ref|>The giraffe at back.<|/ref|><|det|>[[555549, 269, 9555549, 20, 222, 22222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 222222, 222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222, 22222, 2222222,
`
code:
`
import torch
from transformers import AutoModelForCausalLM
from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM
from deepseek_vl2.utils.io import load_pil_images
model_path = "weights/deepseek-vl2-small"
print(model_path)
vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer
vl_gpt: DeepseekVLV2ForCausalLM = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()
conversation = [
{
"role": "<|User|>",
"content": "\n<|ref|>The giraffe at the back.<|/ref|>.",
"images": ["./images/visual_grounding_1.jpeg"],
},
{"role": "<|Assistant|>", "content": ""},
]
pil_images = load_pil_images(conversation)
prepare_inputs = vl_chat_processor(
conversations=conversation,
images=pil_images,
force_batchify=True,
system_prompt=""
).to(vl_gpt.device)
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)
outputs = vl_gpt.language.generate(
inputs_embeds=inputs_embeds,
attention_mask=prepare_inputs.attention_mask,
pad_token_id=tokenizer.eos_token_id,
bos_token_id=tokenizer.bos_token_id,
eos_token_id=tokenizer.eos_token_id,
max_new_tokens=512,
do_sample=False,
use_cache=True
)
answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=False)
print(f"{prepare_inputs['sft_format'][0]}", answer)
`
@HubHop commented on GitHub (Feb 7, 2025):
Hi @SherlockHolmes221，could you please check your python environment. In particular, please make sure "transformers==4.38.2".
@SherlockHolmes221 commented on GitHub (Feb 8, 2025):
my python environment:
transformers 4.38.2
transformers-stream-generator 0.0.4
@cacheee commented on GitHub (Feb 10, 2025):
readme中的inference代码和gradio代码run出来结果不一致，我的异常和该问题描述的相似
@ran337287 commented on GitHub (Feb 14, 2025):
俺也一样