[GH-ISSUE #41] Hi, I have a question about fine tuning. #41

New issue

Open

opened 2026-10-26 21:07:31 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 21:07:31 +00:00

Owner

Originally created by @Yindy07 on GitHub (Aug 5, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder-V2/issues/41

when inferring from a model to FIM, I want to fine-tune it with a dataset, but the format of the dataset is
{"instruction" : "", "output": ""} should I use this format, or should I add tokens such as ?
"<｜fim▁hole｜>" "<｜fim▁begin｜>", etc.
when inferring the model, I am using a template like <｜fim▁begin｜>{prefix}<｜fim▁hole｜>{suffix}<｜fim▁end｜>,
is this correct?
For example :
"<｜fim▁begin｜> handleMoveFocus = (<｜fim▁hole｜>) => {\n let index =<｜fim▁end｜>"
Is this the right way to ask for reasoning in this form?

Originally created by @Yindy07 on GitHub (Aug 5, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder-V2/issues/41 1. when inferring from a model to FIM, I want to fine-tune it with a dataset, but the format of the dataset is {"instruction" : "", "output": ""} should I use this format, or should I add tokens such as ? "<｜fim▁hole｜>" "<｜fim▁begin｜>", etc. 2. when inferring the model, I am using a template like <｜fim▁begin｜>{prefix}<｜fim▁hole｜>{suffix}<｜fim▁end｜>, is this correct? For example : "<｜fim▁begin｜> handleMoveFocus = (<｜fim▁hole｜>) => {\n let index =<｜fim▁end｜>" Is this the right way to ask for reasoning in this form?

bsakaguchi referenced this issue

2026-10-26 21:08:05 +00:00

[PR #44] add fine-tuning code with lora support #93