[GH-ISSUE #11] Will it compare performance with llama-moe? #9

New issue

Closed

opened 2026-10-26 21:01:34 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:01:34 +00:00

Owner

Originally created by @ccccj on GitHub (Jan 16, 2024).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-MoE/issues/11

llama-moe：
https://github.com/pjlab-sys4nlp/llama-moe/tree/main

Or will a training framework be released with llama as the base model?

Originally created by @ccccj on GitHub (Jan 16, 2024). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-MoE/issues/11 llama-moe： https://github.com/pjlab-sys4nlp/llama-moe/tree/main Or will a training framework be released with llama as the base model?

bsakaguchi closed this issue

2026-10-26 21:01:34 +00:00

bsakaguchi commented

2026-10-26 21:01:36 +00:00

Author

Owner

@DeepSeekDDM commented on GitHub (Feb 4, 2024):

Our DeepSeekMoE model is trained from scratch, but not resumed or initialized from the LLaMA/LLaMA2 checkpoints.
As for absolute performance, you can refer to our and their papers for details.

@DeepSeekDDM commented on GitHub (Feb 4, 2024): Our DeepSeekMoE model is trained from scratch, but not resumed or initialized from the LLaMA/LLaMA2 checkpoints. As for absolute performance, you can refer to our and their papers for details.

bsakaguchi referenced this issue

2026-10-26 21:02:00 +00:00

[GH-ISSUE #47] CUDA error: unspecified launch failure #42

bsakaguchi referenced this issue

2026-10-26 21:02:06 +00:00

[PR #9] [CLOSED] Update README.md #46