deepseek-ai/DeepSeek-VL
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-VL.git synced 2026-11-05 00:47:24 +00:00
Code Issues 49 Projects Packages Wiki Activity

[GH-ISSUE #19] Can you address the work you have referenced? #14

New issue
Closed
opened 2026-10-26 21:08:49 +00:00 by bsakaguchi · 5 comments
bsakaguchi commented 2026-10-26 21:08:49 +00:00
Owner
Copy link

Originally created by @lucasjinreal on GitHub (Mar 13, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/19

For instance:

  1. SAM pretrained model didn't listed or quated in readme;
  2. the whole architecture looks very same to Vary, please add quate to it and possible made modifications and compare to original work?

thanks.

Originally created by @lucasjinreal on GitHub (Mar 13, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL/issues/19 For instance: 1. SAM pretrained model didn't listed or quated in readme; 2. the whole architecture looks very same to Vary, please add quate to it and possible made modifications and compare to original work? thanks.
bsakaguchi commented 2026-10-26 21:08:50 +00:00
Author
Owner
Copy link

@RERV commented on GitHub (Mar 13, 2024):

Hi, thanks for your interest.

  1. Our DeepSeek-VL use pure SAM pretrained by Meta AI, as we mentioned and cited in our paper.
  2. We have cited Vary, and also closely discussed with Haoran Wei, the first Author of Vary.
<!-- gh-comment-id:1994114184 --> @RERV commented on GitHub (Mar 13, 2024): Hi, thanks for your interest. 1. Our DeepSeek-VL use pure SAM pretrained by Meta AI, as we mentioned and cited in our paper. 2. We have cited Vary, and also closely discussed with Haoran Wei, the first Author of Vary.
bsakaguchi commented 2026-10-26 21:08:50 +00:00
Author
Owner
Copy link

@lucasjinreal commented on GitHub (Mar 13, 2024):

It would be better write these citations in README.

The modifications insights would be benificial to community as I noticed there are some differences between Vary and deepseek-VL.

Since the whole code base not really opensourced, and DeepSeek acutally always being the frontier in Chinese opensource, you can give more dicussions publicly for others to learn. These information might be curcial for users who use deepseek, not just open a model.

<!-- gh-comment-id:1994151782 --> @lucasjinreal commented on GitHub (Mar 13, 2024): It would be better write these citations in README. The modifications insights would be benificial to community as I noticed there are some differences between Vary and deepseek-VL. Since the whole code base not really opensourced, and DeepSeek acutally always being the frontier in Chinese opensource, you can give more dicussions publicly for others to learn. These information might be curcial for users who use deepseek, not just open a model.
bsakaguchi commented 2026-10-26 21:08:50 +00:00
Author
Owner
Copy link

@LingyvKong commented on GitHub (Mar 15, 2024):

Agree with @lucasjinreal , looking forward to seeing the comparison results with Vary.

<!-- gh-comment-id:1999088792 --> @LingyvKong commented on GitHub (Mar 15, 2024): Agree with @lucasjinreal , looking forward to seeing the comparison results with [Vary](https://github.com/Ucas-HaoranWei/Vary).
bsakaguchi commented 2026-10-26 21:08:50 +00:00
Author
Owner
Copy link

@RERV commented on GitHub (Mar 16, 2024):

Thanks for your interest. It is important to note that our choice of DeepSeek-VL for the vision encoder was not based on Vary, and the motivations behind Vary and DeepSeek-VL on the vision encoder are quite different.

  • Vary aims to equip the model with OCR capabilities by specifically training on documents using SAM, then merging with the original CLIP to enhance the model's OCR abilities.
  • On the other hand, DeepSeek-VL aims to enable the model to process high-resolution images of 1024 * 1024 dimensions as input, thereby avoiding the loss of low-level vision information in general real-world tasks. During training, we treat them as a unified Hybrid Encoder and conduct collaborative training on all our data. We have also tried other vision encoders, discussion regarding the model's encoder is also addressed in the main text.

In summary, the motivations and specific initialization parameters differ between Vary and DeepSeek-VL on the vision encoder, as do their training strategies. They share similarities in utilizing a Hybrid Encoder and SAM as the second encoder, as we cited in our report. However, the Hybrid Encoder concept itself is common (not Vary's core contribution), and the SAM structure is not unique to Vary's work. Therefore, we did not extensively elaborate on the differences between the two in the paper and the GitHub repository.

<!-- gh-comment-id:2002031419 --> @RERV commented on GitHub (Mar 16, 2024): Thanks for your interest. It is important to note that our choice of DeepSeek-VL for the vision encoder was not based on Vary, and the motivations behind Vary and DeepSeek-VL on the vision encoder are quite different. - Vary aims to equip the model with OCR capabilities by specifically training on documents using SAM, then merging with the original CLIP to enhance the model's OCR abilities. - On the other hand, DeepSeek-VL aims to enable the model to process high-resolution images of 1024 * 1024 dimensions as input, thereby avoiding the loss of low-level vision information in general real-world tasks. During training, we treat them as a unified Hybrid Encoder and conduct collaborative training on all our data. We have also tried other vision encoders, discussion regarding the model's encoder is also addressed in the main text. In summary, the motivations and specific initialization parameters differ between Vary and DeepSeek-VL on the vision encoder, as do their training strategies. They share similarities in utilizing a Hybrid Encoder and SAM as the second encoder, as we cited in our report. However, the Hybrid Encoder concept itself is common (not Vary's core contribution), and the SAM structure is not unique to Vary's work. Therefore, we did not extensively elaborate on the differences between the two in the paper and the GitHub repository.
bsakaguchi commented 2026-10-26 21:08:50 +00:00
Author
Owner
Copy link

@lucasjinreal commented on GitHub (Mar 17, 2024):

From the code I can notice that the SAM used way are actually a little bit different then Vary, from the aspect of Hybrid it could treat as same.

But here is the question: I tested Deepseek VL, the OCR ability are weak, why not choose using Vary's training strategies instead using combined directly? Is it better? (from result clearly it's not)

Also, there is a high level feature jump to output in your version of SAM, is it better than Vary? (I didn't saw any ablation study in paper in this part)

<!-- gh-comment-id:2002290420 --> @lucasjinreal commented on GitHub (Mar 17, 2024): From the code I can notice that the SAM used way are actually a little bit different then Vary, from the aspect of Hybrid it could treat as same. But here is the question: I tested Deepseek VL, the OCR ability are weak, why not choose using Vary's training strategies instead using combined directly? Is it better? (from result clearly it's not) Also, there is a high level feature jump to output in your version of SAM, is it better than Vary? (I didn't saw any ablation study in paper in this part)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
enhancement

   
environment

   
pull-request
Mirrored from GitHub Pull Request

No labels
enhancement
environment
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-VL#14
No description provided.