deepseek-ai/DeepSeek-R1

Fork 0

mirror of https://github.com/deepseek-ai/DeepSeek-R1.git synced 2026-11-05 00:47:42 +00:00

[GH-ISSUE #548] 请问官方有提供评测姿势，可以复现评测结果嘛？ #437

New issue

Closed

opened 2026-10-26 21:21:17 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-26 21:21:17 +00:00

Owner

Originally created by @hduyyg on GitHub (Mar 7, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/548

请问官方有提供评测姿势，可以复现评测结果嘛？
网上找了一大圈，没有找到

Originally created by @hduyyg on GitHub (Mar 7, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/548 请问官方有提供评测姿势，可以复现评测结果嘛？网上找了一大圈，没有找到

bsakaguchi

2026-10-26 21:21:17 +00:00

closed this issue
added the
stale

closed-as-stale
labels

bsakaguchi commented

2026-10-26 21:21:19 +00:00

Author

Owner

@pipilia commented on GitHub (Mar 11, 2025):

同求

@pipilia commented on GitHub (Mar 11, 2025): 同求

bsakaguchi commented

2026-10-26 21:21:19 +00:00

Author

Owner

@Keithwwa commented on GitHub (Mar 13, 2025):

同求，个人复现mmlu ceval等数据集，和官方结果有5分以内的差距。

@Keithwwa commented on GitHub (Mar 13, 2025): 同求，个人复现mmlu ceval等数据集，和官方结果有5分以内的差距。

bsakaguchi commented

2026-10-26 21:21:19 +00:00

Author

Owner

@ZCzzzzzz commented on GitHub (Apr 2, 2025):

同求，个人复现mmlu ceval等数据集，和官方结果有5分以内的差距。

你好，请问官方的mmlu、ceval的精度是多少？

@ZCzzzzzz commented on GitHub (Apr 2, 2025): > 同求，个人复现mmlu ceval等数据集，和官方结果有5分以内的差距。你好，请问官方的mmlu、ceval的精度是多少？

bsakaguchi commented

2026-10-26 21:21:19 +00:00

Author

Owner

@Keithwwa commented on GitHub (Apr 8, 2025):

https://gitee.com/aisbench/benchmark
请尝试这个测评工具，我们在部分常用数据集比如math500 aime2024 ceval等数据集上复现了R1的评估结果
Please try this evaluation tool. We have reproduced the evaluation results of R1 on some common datasets such as math500 aime2024 ceval and other datasets.

@Keithwwa commented on GitHub (Apr 8, 2025): https://gitee.com/aisbench/benchmark 请尝试这个测评工具，我们在部分常用数据集比如math500 aime2024 ceval等数据集上复现了R1的评估结果 Please try this evaluation tool. We have reproduced the evaluation results of R1 on some common datasets such as math500 aime2024 ceval and other datasets.

bsakaguchi commented

2026-10-26 21:21:19 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (May 19, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

@github-actions[bot] commented on GitHub (May 19, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

bsakaguchi commented

2026-10-26 21:21:20 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 12, 2025):

false

@github-actions[bot] commented on GitHub (Jun 12, 2025): false

bsakaguchi commented

2026-10-26 21:21:20 +00:00

Author

Owner

@AlonglangQQ commented on GitHub (Aug 21, 2025):

https://gitee.com/aisbench/benchmark 请尝试这个测评工具，我们在部分常用数据集比如math500 aime2024 ceval等数据集上复现了R1的评估结果 Please try this evaluation tool. We have reproduced the evaluation results of R1 on some common datasets such as math500 aime2024 ceval and other datasets.

怎么复现？能给具体命令吗？mmlu有测过吗

@AlonglangQQ commented on GitHub (Aug 21, 2025): > https://gitee.com/aisbench/benchmark 请尝试这个测评工具，我们在部分常用数据集比如math500 aime2024 ceval等数据集上复现了R1的评估结果 Please try this evaluation tool. We have reproduced the evaluation results of R1 on some common datasets such as math500 aime2024 ceval and other datasets. 怎么复现？能给具体命令吗？mmlu有测过吗