mirror of
https://github.com/deepseek-ai/DeepSeek-Coder.git
synced 2026-11-05 00:57:19 +00:00
[GH-ISSUE #132] Pretraining code #113
Labels
No labels
documentation
ENV
pull-request
research question
WIP
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
deepseek-ai/DeepSeek-Coder#113
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Calvinnncy97 on GitHub (Mar 5, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/132
May I know if the pretraining code will be released? Besides, what is the data format for the pretraining? Would love to know these as it will be very helpful in continued pretraining of the model.
Thank you.
Best regards
@guoday commented on GitHub (Mar 12, 2024):
Hello, there are currently no plans to open-source the pre-training code. The format for pre-training is consistent with most of the methods currently in use, where different samples are concatenated together and separated by a special token.
@Calvinnncy97 commented on GitHub (Mar 15, 2024):
Got it. Thank you.