[GH-ISSUE #74] Fine tune template for distilled /R1 models #61

New issue

Closed

opened 2026-10-26 21:12:43 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-26 21:12:43 +00:00

Owner

Originally created by @kishoretvk on GitHub (Jan 25, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/74

I'm interested in exploring fine-tuning DeepSeek-R1 or its distilled models e.g. DeepSeek-R1-Distill-Qwen to utilize LoRA, DPO, and their combinations for potential enhancements.

Proposed Approaches:

LoRA: Incorporating Low-Rank Adaptation techniques to enhance model efficiency and performance during fine-tuning.
DPO: Utilizing Differentiable Prompt Optimization for refining prompts and improving task performance.
Combined Approach: Applying both LoRA and DPO to leverage the strengths of both methods for comprehensive model improvements.

Could you provide any templates for this

Originally created by @kishoretvk on GitHub (Jan 25, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-R1/issues/74 I'm interested in exploring fine-tuning DeepSeek-R1 or its distilled models \(e.g. DeepSeek-R1-Distill-Qwen\) to utilize **LoRA, DPO, and their combinations** for potential enhancements. #### Proposed Approaches: 1. **LoRA**: Incorporating Low-Rank Adaptation techniques to enhance model efficiency and performance during fine-tuning. 2. **DPO**: Utilizing Differentiable Prompt Optimization for refining prompts and improving task performance. 3. **Combined Approach**: Applying both **LoRA** and **DPO** to leverage the strengths of both methods for comprehensive model improvements. Could you provide any templates for this

bsakaguchi

2026-10-26 21:12:43 +00:00

closed this issue
added the
stale

closed-as-stale
labels

bsakaguchi commented

2026-10-26 21:12:44 +00:00

Author

Owner

@ramisszvq commented on GitHub (Jan 29, 2025):

https://github.com/aws-samples/amazon-sagemaker-llm-fine-tuning-remote-decorator/blob/main/deepseek-r1-distilled-llama-70b-fsdp-qlora-remote-decorator_qa.ipynb

do check this out

@ramisszvq commented on GitHub (Jan 29, 2025): https://github.com/aws-samples/amazon-sagemaker-llm-fine-tuning-remote-decorator/blob/main/deepseek-r1-distilled-llama-70b-fsdp-qlora-remote-decorator_qa.ipynb do check this out

bsakaguchi commented

2026-10-26 21:12:44 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Mar 4, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

@github-actions[bot] commented on GitHub (Mar 4, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

bsakaguchi commented

2026-10-26 21:12:45 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Apr 12, 2025):

false

@github-actions[bot] commented on GitHub (Apr 12, 2025): false