mirror of
https://github.com/deepseek-ai/DeepSeek-VL2.git
synced 2026-11-05 01:07:23 +00:00
[GH-ISSUE #129] Significant Discrepancies Between Batch Inference and Single Inference in DeepSeekVL2 #108
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-VL2#108
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @j1ajunzhu on GitHub (Apr 8, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/129
Description:
I've noticed that when using DeepSeekVL2-tiny, predictions for the same input differ significantly between single inference (processing one input at a time) and batch inference (processing multiple inputs together). The outputs for a single item in a batch are notably different from its output when run alone. I suspect this might relate to how batch normalization or dropout is handled, or a difference in how the data is preprocessed. Any guidance on why this discrepancy occurs, or how to align the results, would be greatly appreciated.