[GH-ISSUE #19] A question about the num_hidden_layers=61？ #18

New issue

Closed

opened 2026-10-26 21:10:58 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:10:58 +00:00

Owner

Originally created by @DoubleEspresso-7 on GitHub (Dec 28, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/19

I find the num_hidden_layers=61 and num_nextn_predict_layers = 1, but num_hidden_layers = 60 for the deepseekV2, so I want to know the new layer is the MTP module or the main transformer blocks is just 61?

Hope for your answer!

Originally created by @DoubleEspresso-7 on GitHub (Dec 28, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/19 I find the num_hidden_layers=61 and num_nextn_predict_layers = 1, but num_hidden_layers = 60 for the deepseekV2, so I want to know the new layer is the MTP module or the main transformer blocks is just 61? Hope for your answer!

bsakaguchi closed this issue

2026-10-26 21:10:58 +00:00

bsakaguchi commented

2026-10-26 21:10:59 +00:00

Author

Owner

@DoubleEspresso-7 commented on GitHub (Dec 28, 2024):

I find the answer in https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/blob/main/README_WEIGHTS.md， and the main transformer blocks is just 61

@DoubleEspresso-7 commented on GitHub (Dec 28, 2024): I find the answer in https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/blob/main/README_WEIGHTS.md， and the main transformer blocks is just 61

bsakaguchi referenced this issue

2026-10-26 21:27:04 +00:00

[GH-ISSUE #1121] Feature Request: On-Device Persistent Memory with Cryptographic Identity Binding (Local Storage Architecture) #703