[GH-ISSUE #463] Does anyone know how to reproduce the accuracy of DeepSeek R1 as officially reported on various datasets? #372

Closed
opened 2026-10-26 21:20:13 +00:00 by bsakaguchi · 3 comments
Owner

Originally created by @jifa513 on GitHub (Feb 20, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/463

Does anyone know how to reproduce the accuracy of DeepSeek R1 as officially reported on various datasets?
有人知道怎么复现DeepSeek R1官方公布的在各个数据集上的精度吗?

Originally created by @jifa513 on GitHub (Feb 20, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/463 Does anyone know how to reproduce the accuracy of DeepSeek R1 as officially reported on various datasets? 有人知道怎么复现DeepSeek R1官方公布的在各个数据集上的精度吗?
bsakaguchi 2026-10-26 21:20:13 +00:00
Author
Owner

@liangruogu commented on GitHub (Feb 21, 2025):

同问,不清楚如何测试,也没有找到具体标准或者脚本

<!-- gh-comment-id:2673307133 --> @liangruogu commented on GitHub (Feb 21, 2025): 同问,不清楚如何测试,也没有找到具体标准或者脚本
Author
Owner

@github-actions[bot] commented on GitHub (Apr 23, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

<!-- gh-comment-id:2822752198 --> @github-actions[bot] commented on GitHub (Apr 23, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
Author
Owner

@github-actions[bot] commented on GitHub (May 27, 2025):

false

<!-- gh-comment-id:2910795033 --> @github-actions[bot] commented on GitHub (May 27, 2025): false
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-R1#372
No description provided.