[GH-ISSUE #173] What is the correct padding side for train/eval of base model for FIM? #152

New issue

Open

opened 2026-10-26 21:04:08 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:04:08 +00:00

Owner

Originally created by @zhzhangcc on GitHub (Jul 6, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/173

I see in https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/finetune/finetune_deepseekcoder.py#L132 that the padding side for the instruct model is set to right.
However, the tokenizer's default is `left. And there are no other official examples I can find for a fine-tune on the base model.

So what should the padding side by for the base model during train and eval?

Originally created by @zhzhangcc on GitHub (Jul 6, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/173 I see in https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/finetune/finetune_deepseekcoder.py#L132 that the padding side for the instruct model is set to `right`. However, the tokenizer's default is `left. And there are no other official examples I can find for a fine-tune on the base model. So what should the padding side by for the base model during train and eval?