[GH-ISSUE #20] How does training and inference efficiency compare to other MLLMs? #18

New issue

Open

opened 2026-10-26 21:10:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:10:17 +00:00

Owner

Originally created by @rushzy on GitHub (Dec 25, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/20

The model size of DeepSeek-VL2 is 27.5B, and the trainable parameters of lora fine-tuning are 212M, which is about 9 times that of lora fine-tuning InternVL2-8B (23M)。

Using the same data, the training time is about 20 times that of InternVL2-8B. Is this in line with expectations?

I use the swift framework for fine-tuning, and the training script is as follows

nproc_per_node=2

CUDA_VISIBLE_DEVICES=0,1 \
NPROC_PER_NODE=$nproc_per_node \
swift sft \
    --model /mnt/dolphinfs/hdd_pool/docker/user/rushzy/deepseek-ai/deepseek-vl2 \
    --train_type lora \
    --dataset /mnt/dolphinfs/hdd_pool/docker/user/rushzy/data/train.jsonl \
    --num_train_epochs 3 \
    --learning_rate 8e-5 \
    --lora_rank 8 \
    --lora_alpha 12 \
    --max_length 4096 \
    --save_only_model True \
    --eval_steps 2000 \
    --save_steps 2000 \
    --save_total_limit -1 \
    --output_dir /mnt/dolphinfs/hdd_pool/docker/user/rushzy/output/test \
    --deepspeed ./ds_configs/ds_zero3_cosine.json \
    --lazy_tokenize True \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 4

Originally created by @rushzy on GitHub (Dec 25, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-VL2/issues/20 The model size of DeepSeek-VL2 is 27.5B, and the trainable parameters of lora fine-tuning are 212M, which is about 9 times that of lora fine-tuning InternVL2-8B (23M)。 Using the same data, the training time is about 20 times that of InternVL2-8B. Is this in line with expectations? I use the swift framework for fine-tuning, and the training script is as follows ``` nproc_per_node=2 CUDA_VISIBLE_DEVICES=0,1 \ NPROC_PER_NODE=$nproc_per_node \ swift sft \ --model /mnt/dolphinfs/hdd_pool/docker/user/rushzy/deepseek-ai/deepseek-vl2 \ --train_type lora \ --dataset /mnt/dolphinfs/hdd_pool/docker/user/rushzy/data/train.jsonl \ --num_train_epochs 3 \ --learning_rate 8e-5 \ --lora_rank 8 \ --lora_alpha 12 \ --max_length 4096 \ --save_only_model True \ --eval_steps 2000 \ --save_steps 2000 \ --save_total_limit -1 \ --output_dir /mnt/dolphinfs/hdd_pool/docker/user/rushzy/output/test \ --deepspeed ./ds_configs/ds_zero3_cosine.json \ --lazy_tokenize True \ --per_device_train_batch_size 2 \ --gradient_accumulation_steps 4 ```

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepSeek-VL2#18

No description provided.

Rows
Columns