[GH-ISSUE #76] bad results with OCR inferece #62

New issue

Open

opened 2026-10-26 21:11:08 +00:00 by bsakaguchi · 12 comments

bsakaguchi commented 2026-10-26 21:11:08 +00:00

Owner

Copy link

Originally created by @katie312 on GitHub (Feb 12, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/76

I input a pic like this:

it seems like a very easy task, but there is so much problem in the output (structure is fine, but some words are wrong). Is there something wrong with the inference code? or does it only support English.

the code:

import torch
from transformers import AutoModelForCausalLM

from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM
from deepseek_vl2.utils.io import load_pil_images


# specify the path to the model
model_path = "/deepseek-vl2-model"
vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer

vl_gpt: DeepseekVLV2ForCausalLM = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()

## single image conversation example
## Please note that <|ref|> and <|/ref|> are designed specifically for the object localization feature. These special tokens are not required for normal conversations.
## If you would like to experience the grounded captioning functionality (responses that include both object localization and reasoning), you need to add the special token <|grounding|> at the beginning of the prompt. Examples could be found in Figure 9 of our paper.
conversation = [
    {
        "role": "<|User|>",
        "content": "<image>\n把图片里面的所有内容进行ocr识别，markdown格式输出",
        "images": ["./396_2352fdc7zf.png"],
    },
    {"role": "<|Assistant|>", "content": ""},
]

# load images and prepare for inputs
pil_images = load_pil_images(conversation)
prepare_inputs = vl_chat_processor(
    conversations=conversation,
    images=pil_images,
    force_batchify=True,
    system_prompt=""
).to(vl_gpt.device)

# run image encoder to get the image embeddings
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)

# run the model to get the response
outputs = vl_gpt.language.generate(
    inputs_embeds=inputs_embeds,
    attention_mask=prepare_inputs.attention_mask,
    pad_token_id=tokenizer.eos_token_id,
    bos_token_id=tokenizer.bos_token_id,
    eos_token_id=tokenizer.eos_token_id,
    max_new_tokens=512,
    do_sample=False,
    use_cache=True
)

answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=False)
print(f"{prepare_inputs['sft_format'][0]}", answer)

Originally created by @katie312 on GitHub (Feb 12, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/76 I input a pic like this:  it seems like a very easy task, but there is so much problem in the output (structure is fine, but some words are wrong). Is there something wrong with the inference code? or does it only support English. the code: ``` import torch from transformers import AutoModelForCausalLM from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM from deepseek_vl2.utils.io import load_pil_images # specify the path to the model model_path = "/deepseek-vl2-model" vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path) tokenizer = vl_chat_processor.tokenizer vl_gpt: DeepseekVLV2ForCausalLM = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True) vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval() ## single image conversation example ## Please note that <|ref|> and <|/ref|> are designed specifically for the object localization feature. These special tokens are not required for normal conversations. ## If you would like to experience the grounded captioning functionality (responses that include both object localization and reasoning), you need to add the special token <|grounding|> at the beginning of the prompt. Examples could be found in Figure 9 of our paper. conversation = [ { "role": "<|User|>", "content": "<image>\n把图片里面的所有内容进行ocr识别，markdown格式输出", "images": ["./396_2352fdc7zf.png"], }, {"role": "<|Assistant|>", "content": ""}, ] # load images and prepare for inputs pil_images = load_pil_images(conversation) prepare_inputs = vl_chat_processor( conversations=conversation, images=pil_images, force_batchify=True, system_prompt="" ).to(vl_gpt.device) # run image encoder to get the image embeddings inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs) # run the model to get the response outputs = vl_gpt.language.generate( inputs_embeds=inputs_embeds, attention_mask=prepare_inputs.attention_mask, pad_token_id=tokenizer.eos_token_id, bos_token_id=tokenizer.bos_token_id, eos_token_id=tokenizer.eos_token_id, max_new_tokens=512, do_sample=False, use_cache=True ) answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=False) print(f"{prepare_inputs['sft_format'][0]}", answer) ```

bsakaguchi commented 2026-10-26 21:11:10 +00:00

Author

Owner

Copy link

@ibbol commented on GitHub (Feb 12, 2025):

我用没问题啊

<!-- gh-comment-id:2652824697 --> @ibbol commented on GitHub (Feb 12, 2025): 我用没问题啊 

bsakaguchi commented 2026-10-26 21:11:10 +00:00

Author

Owner

Copy link

@ibbol commented on GitHub (Feb 12, 2025):

<!-- gh-comment-id:2652829366 --> @ibbol commented on GitHub (Feb 12, 2025): 

bsakaguchi commented 2026-10-26 21:11:11 +00:00

Author

Owner

Copy link

@katie312 commented on GitHub (Feb 12, 2025):

请问这个在线的是什么模型呀，我这边本地下载用的small版本。
其实原图不是这张，因为是公司的数据不方便放出来，但是大致布局差不多，就几个图标+段文字，输出效果特别差🥲

<!-- gh-comment-id:2652875778 --> @katie312 commented on GitHub (Feb 12, 2025): >  请问这个在线的是什么模型呀，我这边本地下载用的small版本。 其实原图不是这张，因为是公司的数据不方便放出来，但是大致布局差不多，就几个图标+段文字，输出效果特别差🥲

bsakaguchi commented 2026-10-26 21:11:11 +00:00

Author

Owner

Copy link

@ibbol commented on GitHub (Feb 12, 2025):

请问这个在线的是什么模型呀，我这边本地下载用的small版本。 其实原图不是这张，因为是公司的数据不方便放出来，但是大致布局差不多，就几个图标+段文字，输出效果特别差🥲

tiny版本，我也想部署small版，但是我是4张24G的4090，不知道怎么分配显存，按照别人的方法拆分layer好像还是显存不够。

<!-- gh-comment-id:2652883123 --> @ibbol commented on GitHub (Feb 12, 2025): > >  > > 请问这个在线的是什么模型呀，我这边本地下载用的small版本。 其实原图不是这张，因为是公司的数据不方便放出来，但是大致布局差不多，就几个图标+段文字，输出效果特别差🥲 tiny版本，我也想部署small版，但是我是4张24G的4090，不知道怎么分配显存，按照别人的方法拆分layer好像还是显存不够。

bsakaguchi commented 2026-10-26 21:11:11 +00:00

Author

Owner

Copy link

@katie312 commented on GitHub (Feb 12, 2025):

请问这个在线的是什么模型呀，我这边本地下载用的small版本。 其实原图不是这张，因为是公司的数据不方便放出来，但是大致布局差不多，就几个图标+段文字，输出效果特别差🥲

tiny版本，我也想部署small版，但是我是4张24G的4090，不知道怎么分配显存，按照别人的方法拆分layer好像还是显存不够。

我用的L40作推理，能跑起来，但是为什么tiny比small效果还好，严重怀疑我代码有问题了🥲，或者麻烦测试一下复杂一点的文档看看效果呢？

<!-- gh-comment-id:2652904524 --> @katie312 commented on GitHub (Feb 12, 2025): > > >  > > > > > > 请问这个在线的是什么模型呀，我这边本地下载用的small版本。 其实原图不是这张，因为是公司的数据不方便放出来，但是大致布局差不多，就几个图标+段文字，输出效果特别差🥲 > > tiny版本，我也想部署small版，但是我是4张24G的4090，不知道怎么分配显存，按照别人的方法拆分layer好像还是显存不够。 我用的L40作推理，能跑起来，但是为什么tiny比small效果还好，严重怀疑我代码有问题了🥲，或者麻烦测试一下复杂一点的文档看看效果呢？

bsakaguchi commented 2026-10-26 21:11:12 +00:00

Author

Owner

Copy link

@ibbol commented on GitHub (Feb 12, 2025):

<!-- gh-comment-id:2652917663 --> @ibbol commented on GitHub (Feb 12, 2025):  

bsakaguchi commented 2026-10-26 21:11:12 +00:00

Author

Owner

Copy link

@ibbol commented on GitHub (Feb 12, 2025):

<!-- gh-comment-id:2652932143 --> @ibbol commented on GitHub (Feb 12, 2025): 

bsakaguchi commented 2026-10-26 21:11:13 +00:00

Author

Owner

Copy link

@Giserlei123 commented on GitHub (Feb 12, 2025):

我直接使用可视化界面，识别效果很好，但是我自己用你这段代码效果也是很差

<!-- gh-comment-id:2653201548 --> @Giserlei123 commented on GitHub (Feb 12, 2025): 我直接使用可视化界面，识别效果很好，但是我自己用你这段代码效果也是很差

bsakaguchi commented 2026-10-26 21:11:13 +00:00

Author

Owner

Copy link

@ibbol commented on GitHub (Feb 12, 2025):

我用楼主的代码好像没问题啊？

<!-- gh-comment-id:2653234440 --> @ibbol commented on GitHub (Feb 12, 2025):  我用楼主的代码好像没问题啊？

bsakaguchi commented 2026-10-26 21:11:13 +00:00

Author

Owner

Copy link

@katie312 commented on GitHub (Feb 12, 2025):

这个看起来也还可以，不过pdf貌似也不太复杂，但是这张图估计我这边测试效果都不行，我明天换张图试试

<!-- gh-comment-id:2653798411 --> @katie312 commented on GitHub (Feb 12, 2025): >  这个看起来也还可以，不过pdf貌似也不太复杂，但是这张图估计我这边测试效果都不行，我明天换张图试试

bsakaguchi commented 2026-10-26 21:11:14 +00:00

Author

Owner

Copy link

@katie312 commented on GitHub (Feb 12, 2025):

我直接使用可视化界面，识别效果很好，但是我自己用你这段代码效果也是很差

不会真是代码问题吧，请问您用的可视化界面是例程里给的吗？

<!-- gh-comment-id:2653800305 --> @katie312 commented on GitHub (Feb 12, 2025): > 我直接使用可视化界面，识别效果很好，但是我自己用你这段代码效果也是很差 不会真是代码问题吧，请问您用的可视化界面是例程里给的吗？

bsakaguchi commented 2026-10-26 21:11:15 +00:00

Author

Owner

Copy link

@katie312 commented on GitHub (Feb 12, 2025):

我用楼主的代码好像没问题啊？
🥲裂开，这也太奇怪了，会不会是我这边环境问题，我没完全按照要求给的版本来配

<!-- gh-comment-id:2653802311 --> @katie312 commented on GitHub (Feb 12, 2025): >  我用楼主的代码好像没问题啊？ 🥲裂开，这也太奇怪了，会不会是我这边环境问题，我没完全按照要求给的版本来配

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

No results found.

Labels

Clear labels   

pull-request

Mirrored from GitHub Pull Request

No labels

pull-request

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepSeek-VL2#62

No description provided.