[GH-ISSUE #10] About the Max Sequence Length #9

New issue

Closed

opened 2026-10-26 21:02:01 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:02:01 +00:00

Owner

Originally created by @xu3kev on GitHub (Nov 4, 2023).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/10

Hi,

Some of the released models have config 'max_position_embeddings' set to 8192 instead of 16384. However, in the repo readme, the models are said to have a 16k context size. Could you clarify what the max sequence length of these models is? Thanks!

Originally created by @xu3kev on GitHub (Nov 4, 2023). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/10 Hi, Some of the released models have [config](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base/blob/9f75359417c2ccb41d8e18806de2f7b1d85e4390/config.json#L11) 'max_position_embeddings' set to 8192 instead of 16384. However, in the repo readme, the models are said to have a 16k context size. Could you clarify what the max sequence length of these models is? Thanks!

bsakaguchi closed this issue

2026-10-26 21:02:02 +00:00

bsakaguchi commented

2026-10-26 21:02:03 +00:00

Author

Owner

@pkuzqh commented on GitHub (Nov 5, 2023):

Sorry for the mistake. We have fixed it. All models' context lengths are 16384.

@pkuzqh commented on GitHub (Nov 5, 2023): Sorry for the mistake. We have fixed it. All models' context lengths are 16384.

bsakaguchi commented

2026-10-26 21:02:03 +00:00

Author

Owner

@xu3kev commented on GitHub (Nov 5, 2023):

Thank you!

@xu3kev commented on GitHub (Nov 5, 2023): Thank you!