[GH-ISSUE #41] For evaluating on the MBPP dataset, any code for the instruction-based model? #34

New issue

Closed

opened 2026-10-26 21:02:28 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-26 21:02:28 +00:00

Owner

Originally created by @wujwyi on GitHub (Nov 21, 2023).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/41

There are different codes for base model and instruction-based model when evaluated on HumanEval dataset. But I didn't find the code for instruction-based model evaluated on MBPP dataset.

Originally created by @wujwyi on GitHub (Nov 21, 2023). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-Coder/issues/41 There are different codes for base model and instruction-based model when evaluated on HumanEval dataset. But I didn't find the code for instruction-based model evaluated on MBPP dataset.

bsakaguchi closed this issue

2026-10-26 21:02:28 +00:00

bsakaguchi commented

2026-10-26 21:02:29 +00:00

Author

Owner

@DejianYang commented on GitHub (Nov 23, 2023):

please refer to https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/MBPP/eval_instruct.py

@DejianYang commented on GitHub (Nov 23, 2023): please refer to https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/Evaluation/MBPP/eval_instruct.py