kvcache-ai/ktransformers

Fork 0

mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-11-05 01:07:28 +00:00

[GH-ISSUE #1779] KTransformers Roadmap (2026 Q1) #1121

New issue

Open

opened 2026-10-29 14:48:46 +00:00 by bsakaguchi · 11 comments

bsakaguchi commented

2026-10-29 14:48:46 +00:00

Owner

Originally created by @KMSorSMS on GitHub (Jan 4, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1779

KTransformers Roadmap - 2026 Q1

Focus

Community: Build official KTransformers website for better user experience and documentation.
Model Coverage: Native support for full-precision models (Xiaomi MiMo, MiniMax, etc.).
Usability: Develop KTransformers CLI tool for streamlined workflow.
Performance: improve performance for amd platform

Community / Website

Build official KTransformers website. Documents @ErvinXie ✅
Add community feedback submission on website @ErvinXie ✅

Coverage

Support fp8 and bf16 moe models in native precision （M2, MiMo, V3.2), (GLM4.7-FP8) with CI @ouqingliang ✅
Support nvfp4 model for 50 series cards.
Intel npu Support https://github.com/kvcache-ai/ktransformers/issues/1701 (community help wanted)
Sglang new PR for FP8 https://github.com/sgl-project/sglang/pull/9754 (community help wanted)

Usability

Develop KTransformers CLI tool. https://github.com/kvcache-ai/ktransformers/tree/ljq-cli-dev @SkqLiao ✅
- Model download and management
- bench: client based bench
- Tune: search params
- Add Support for AMXInt4
- Add Support for gguf
- Add Support for quantization
- Add SFT
Improve installation experience for new users. @ovowei ✅

Performance

Optimized expert offload strategy @ovowei ✅
Optimize Prefill when input is not so long (community help wanted & matmul lib) @ouqingliang

Codebase Refactoring

SFT in kt-kernel @JimmyPeilinLi
Update build system and dependencies.

Contribution / Maintenance

Maintain CI/CD pipelines for merged codebase. @ovowei ✅

Any contribution is welcomed, please email yzwliam@126.com / ervinxie@qq.com if you want to join development WeChat group.

Originally created by @KMSorSMS on GitHub (Jan 4, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1779 # KTransformers Roadmap - 2026 Q1 ## Focus - Community: Build official KTransformers website for better user experience and documentation. - Model Coverage: Native support for full-precision models (Xiaomi MiMo, MiniMax, etc.). - Usability: Develop KTransformers CLI tool for streamlined workflow. - Performance: improve performance for amd platform ## Community / Website - Build official KTransformers website. Documents @ErvinXie ✅ - Add community feedback submission on website @ErvinXie ✅ ## Coverage - Support fp8 and bf16 moe models in native precision （M2, MiMo, V3.2), (GLM4.7-FP8) with CI @ouqingliang ✅ - Support nvfp4 model for 50 series cards. - Intel npu Support https://github.com/kvcache-ai/ktransformers/issues/1701 (community help wanted) - Sglang new PR for FP8 https://github.com/sgl-project/sglang/pull/9754 (community help wanted) ## Usability - Develop KTransformers CLI tool. https://github.com/kvcache-ai/ktransformers/tree/ljq-cli-dev @SkqLiao ✅ - Model download and management - bench: client based bench - Tune: search params - Add Support for AMXInt4 - Add Support for gguf - Add Support for quantization - Add SFT - Improve installation experience for new users. @ovowei ✅ ## Performance - Optimized expert offload strategy @ovowei ✅ - Optimize Prefill when input is not so long (community help wanted & matmul lib) @ouqingliang ## Codebase Refactoring - SFT in kt-kernel @JimmyPeilinLi - Update build system and dependencies. ## Contribution / Maintenance - Maintain CI/CD pipelines for merged codebase. @ovowei ✅ --- Any contribution is welcomed, please email yzwliam@126.com / ervinxie@qq.com if you want to join development WeChat group.

bsakaguchi added the

enhancement

label

2026-10-29 14:48:46 +00:00

bsakaguchi commented

2026-10-29 14:48:47 +00:00

Author

Owner

@KMSorSMS commented on GitHub (Jan 8, 2026):

🙌 Call for Contributions!

We warmly welcome community partners to apply for and contribute to the tasks outlined in this roadmap. Your contributions are vital to the growth of KTransformers. If you are interested in any of the features above (especially those marked "community help wanted"), please email us at yzwliam@126.com or ervinxie@qq.com. We will invite you to join our development WeChat group. Let's build together!

@KMSorSMS commented on GitHub (Jan 8, 2026): 🙌 Call for Contributions! We warmly welcome community partners to apply for and contribute to the tasks outlined in this roadmap. Your contributions are vital to the growth of KTransformers. If you are interested in any of the features above (especially those marked "community help wanted"), please email us at yzwliam@126.com or ervinxie@qq.com. We will invite you to join our development WeChat group. Let's build together!

bsakaguchi commented

2026-10-29 14:48:47 +00:00

Author

Owner

@CodeZ-Hao commented on GitHub (Jan 9, 2026):

我觉得KT的版本更新策略有点太激进了，
0.3版本更新了AMX支持，后面适配了新模型后前缀缓存功能就丢了，
0.4版本合入了sglang，可用性强了很多，但是丢了gguf等量化能力的支持，
0.5版本加入原生fp8支持，性能提升很大，但又丢了AMX支持，
每个大版本都像是一个新产品，用户每次跟随升级都要考虑新特性和老特性之间的取舍，这个是不合理的。
对于会破坏现有特性的新特性，或许可以用preview版本来发布，这个版本就只用来预览这个新特性，新特性无关的功能则继续在正式版本上迭代，直到能平稳合入正式版本为止。像kt的2.x版本，预览一个amx版本，然后在最后合入正式版本，我觉得是合理的

@CodeZ-Hao commented on GitHub (Jan 9, 2026): 我觉得KT的版本更新策略有点太激进了， 0.3版本更新了AMX支持，后面适配了新模型后前缀缓存功能就丢了， 0.4版本合入了sglang，可用性强了很多，但是丢了gguf等量化能力的支持， 0.5版本加入原生fp8支持，性能提升很大，但又丢了AMX支持，每个大版本都像是一个新产品，用户每次跟随升级都要考虑新特性和老特性之间的取舍，这个是不合理的。对于会破坏现有特性的新特性，或许可以用preview版本来发布，这个版本就只用来预览这个新特性，新特性无关的功能则继续在正式版本上迭代，直到能平稳合入正式版本为止。像kt的2.x版本，预览一个amx版本，然后在最后合入正式版本，我觉得是合理的

bsakaguchi commented

2026-10-29 14:48:47 +00:00

Author

Owner

@KMSorSMS commented on GitHub (Jan 9, 2026):

@CodeZ-Hao 丢 gguf 当时是有 bug 迁移没完成，后面加上了，0.5 支持原生 fp8 的时候出现了原来 amx 的小 bug，所以最开始没能支持，但是大概下周就能上线全部支持的版本。后续的更新应该就不会那么多 bug 了，因为变动不会太大，并且引入完整的 CI 测试，我们确实也在注意每次更新带来的 bug，感谢反馈！

@KMSorSMS commented on GitHub (Jan 9, 2026): @CodeZ-Hao 丢 gguf 当时是有 bug 迁移没完成，后面加上了，0.5 支持原生 fp8 的时候出现了原来 amx 的小 bug，所以最开始没能支持，但是大概下周就能上线全部支持的版本。后续的更新应该就不会那么多 bug 了，因为变动不会太大，并且引入完整的 CI 测试，我们确实也在注意每次更新带来的 bug，感谢反馈！

bsakaguchi commented

2026-10-29 14:48:47 +00:00

Author

Owner

@Henryhaohao commented on GitHub (Jan 9, 2026):

辛苦，期待新版上线！

@Henryhaohao commented on GitHub (Jan 9, 2026): 辛苦，期待新版上线！

bsakaguchi commented

2026-10-29 14:48:47 +00:00

Author

Owner

@NukePeace commented on GitHub (Jan 23, 2026):

赶紧支持一下稠密层平均分配至多个GPU吧，拖鞋军块等死了，只有这样才在便宜卡上能支持可用长度的上下文，谢了！！

@NukePeace commented on GitHub (Jan 23, 2026): 赶紧支持一下稠密层平均分配至多个GPU吧，拖鞋军块等死了，只有这样才在便宜卡上能支持可用长度的上下文，谢了！！

bsakaguchi commented

2026-10-29 14:48:48 +00:00

Author

Owner

@mathxie commented on GitHub (Jan 24, 2026):

avx512跑kimi 原生int4，啥时候能修好呢？

@mathxie commented on GitHub (Jan 24, 2026): avx512跑kimi 原生int4，啥时候能修好呢？

bsakaguchi commented

2026-10-29 14:48:48 +00:00

Author

Owner

@KMSorSMS commented on GitHub (Jan 25, 2026):

@mathxie @ouqingliang 还有 bug 吗？kimi int4？

@KMSorSMS commented on GitHub (Jan 25, 2026): @mathxie @ouqingliang 还有 bug 吗？kimi int4？

bsakaguchi commented

2026-10-29 14:48:48 +00:00

Author

Owner

@mathxie commented on GitHub (Jan 31, 2026):

@mathxie @ouqingliang 还有 bug 吗？kimi int4？
前几天刚修好。

@mathxie commented on GitHub (Jan 31, 2026): > [@mathxie](https://github.com/mathxie) [@ouqingliang](https://github.com/ouqingliang) 还有 bug 吗？kimi int4？前几天刚修好。

bsakaguchi commented

2026-10-29 14:48:48 +00:00

Author

Owner

@ehfd commented on GitHub (Feb 15, 2026):

Is anyone willing to take up https://github.com/sgl-project/sglang/pull/9754 to resolve https://github.com/sgl-project/sglang/issues/12887?

@ehfd commented on GitHub (Feb 15, 2026): Is anyone willing to take up https://github.com/sgl-project/sglang/pull/9754 to resolve https://github.com/sgl-project/sglang/issues/12887?

bsakaguchi commented

2026-10-29 14:48:48 +00:00

Author

Owner

@ydch1014 commented on GitHub (Mar 18, 2026):

Are there any plans to support domestic GPUs, such as the MetaX C500 or Moore Threads S5000?
I couldn't find any tutorials. Also, these GPUs require specialized docker images and vLLM plugins provided by the vendor. so I assume they are not supported at the moment, right?

@ydch1014 commented on GitHub (Mar 18, 2026): Are there any plans to support domestic GPUs, such as the MetaX C500 or Moore Threads S5000? I couldn't find any tutorials. Also, these GPUs require specialized docker images and vLLM plugins provided by the vendor. so I assume they are not supported at the moment, right?

bsakaguchi commented

2026-10-29 14:48:48 +00:00

Author

Owner

@libingtao981 commented on GitHub (Mar 23, 2026):

Models with Multi-Token Prediction (MTP) heads are becoming increasingly common now. So, it would be great if MTP support for Speculative Decoding could be implemented soon to improve throughput. Thanks!

@libingtao981 commented on GitHub (Mar 23, 2026): Models with Multi-Token Prediction (MTP) heads are becoming increasingly common now. So, it would be great if MTP support for Speculative Decoding could be implemented soon to improve throughput. Thanks!