[GH-ISSUE #20] Can you talk more about the format reward model ? #18

Closed
opened 2026-10-26 21:10:47 +00:00 by bsakaguchi · 3 comments
Owner

Originally created by @zhengyunqq on GitHub (Jan 21, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/20

For example, does the format reward model give higher scores for longer responses between the "think" tags?

Originally created by @zhengyunqq on GitHub (Jan 21, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/20 For example, does the format reward model give higher scores for longer responses between the "think" tags?
bsakaguchi 2026-10-26 21:10:47 +00:00
Author
Owner

@kaiyliu commented on GitHub (Jan 22, 2025):

Same question, how does Format reward determine whether the reasoning process is within the tags, and furthermore, how does it determine which sentence constitutes the reasoning process?

<!-- gh-comment-id:2606577461 --> @kaiyliu commented on GitHub (Jan 22, 2025): Same question, how does Format reward determine whether the reasoning process is within the <think> tags, and furthermore, how does it determine which sentence constitutes the reasoning process?
Author
Owner

@github-actions[bot] commented on GitHub (Feb 22, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

<!-- gh-comment-id:2675891675 --> @github-actions[bot] commented on GitHub (Feb 22, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
Author
Owner

@github-actions[bot] commented on GitHub (Mar 18, 2025):

false

<!-- gh-comment-id:2731238315 --> @github-actions[bot] commented on GitHub (Mar 18, 2025): false
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
deepseek-ai/DeepSeek-R1#18
No description provided.