[GH-ISSUE #1865] Fine Tune kimi k2 thinking - INT4 #1161

New issue

Closed

opened 2026-10-29 14:49:25 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:49:25 +00:00

Owner

Originally created by @savitha-suresh on GitHub (Feb 24, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1865

Reminder

I have read the above rules and searched the existing issues.

Problem

Hello.
https://huggingface.co/moonshotai/Kimi-K2-Thinking/blob/main/docs/deploy_guidance.md
I would like to fine tune kimi-k2-thinking model. I have downloaded the int4 weights. I dont have 2TB+ to convert to bf16. How to fine tune using ktransformers and llamafactory using int4 weights instead of using bf16?

System Info

I have only 860GB RAM and 1TB disk and 320GB VRAM.

Reproduction

Others

No response

Originally created by @savitha-suresh on GitHub (Feb 24, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1865 ### Reminder - [x] I have read the above rules and searched the existing issues. ### Problem Hello. https://huggingface.co/moonshotai/Kimi-K2-Thinking/blob/main/docs/deploy_guidance.md I would like to fine tune kimi-k2-thinking model. I have downloaded the int4 weights. I dont have 2TB+ to convert to bf16. How to fine tune using ktransformers and llamafactory using int4 weights instead of using bf16? ### System Info I have only 860GB RAM and 1TB disk and 320GB VRAM. ### Reproduction ```text ``` ### Others _No response_

bsakaguchi closed this issue

2026-10-29 14:49:25 +00:00

bsakaguchi commented

2026-10-29 14:49:25 +00:00

Author

Owner

@ErvinXie commented on GitHub (Feb 28, 2026):

KTransformers does support fine-tuning via ktsft. However, fine-tuning Kimi K2 currently requires approximately 2TB of RAM, which exceeds your available 860GB.

We are actively working on a new version of our fine-tuning feature that will significantly reduce memory requirements. Please stay tuned for updates.

This reply was generated by an AI agent and reviewed by a human.

@ErvinXie commented on GitHub (Feb 28, 2026): KTransformers does support fine-tuning via **ktsft**. However, fine-tuning Kimi K2 currently requires approximately 2TB of RAM, which exceeds your available 860GB. We are actively working on a new version of our fine-tuning feature that will significantly reduce memory requirements. Please stay tuned for updates. --- *This reply was generated by an AI agent and reviewed by a human.*

bsakaguchi commented

2026-10-29 14:49:26 +00:00

Author

Owner

@JimmyPeilinLi commented on GitHub (Mar 5, 2026):

Just as ErvinXie stated, we will release a new version in April to address these issues. However, the current kt-sft version does not support loading from int4.

@JimmyPeilinLi commented on GitHub (Mar 5, 2026): Just as ErvinXie stated, we will release a new version in April to address these issues. However, the current kt-sft version does not support loading from int4.