25 Commits

Author SHA1 Message Date
mrhaoxx
0f85b2a744 fix seqlen buffer size int32 overflow 2026-03-18 06:05:09 +00:00
mrhaoxx
b1a9691201 online repack 2026-02-24 11:01:53 +00:00
mrhaoxx
8aea4b3e2a ckpt 2026-02-22 09:18:26 +00:00
mrhaoxx
391cb6f79d [fix]: fix memory footprint 2026-02-02 08:34:01 +00:00
mrhaoxx
9efe1317b1 [fix]: fix forward cache (maybe) 2026-01-31 21:19:46 +00:00
mrhaoxx
192a9584f1 [chore]: save 2026-01-25 18:28:40 +00:00
mrhaoxx
8f44a64a7a [fix]: fix memory pool 2026-01-22 19:41:07 +00:00
mrhaoxx
5f6482ff50 [feat]: support skip lora 2026-01-20 05:38:11 +00:00
JimmyPeilinLi
84935a22a6 [feat](kt-sft-refactor): add AMXInt4_1KGroup and AMXInt4_KGroup SFT-MOE support 2026-01-05 16:22:57 +00:00
JimmyPeilinLi
fdf7994677 [feat](kt-sft-refactor): add AMXINT4 AMXINT4_1 quantization mode support to SFT-MOE tests 2026-01-05 13:25:58 +00:00
JimmyPeilinLi
de71edb6a6 [fix](kt-sft-refactor): fix LoRA params sync bug and pass no-tp test 2026-01-04 09:28:23 +00:00
JimmyPeilinLi
1fe15ed6fe [fix](kt-sft-refactor): fix output dtype to bf16 and add pybind config fields 2026-01-02 07:18:28 +00:00
JimmyPeilinLi
1ff825f2e7 [feat](kt-kernel): Add SFT MoE AMX implementation and documentation
- Add SFT MoE AMX operator implementation (sft_moe.hpp, moe-sft-tp.hpp)
- Add architecture documentation for kt-kernel codebase
- Add MoE SFT AMX design and usage documentation
- Add test example for MoE SFT AMX
- Update ext_bindings.cpp and common.hpp for SFT support
2025-12-31 06:24:24 +00:00
Jiaqi Liao
46b0f36980 [feat](kt-kernel): Fix CPU instruction set variants for build & install (#1746)
* [feat]: Enhance CPU feature detection and support for AVX512 extensions

- Added cmake/DetectCPU.cmake for automatic CPU feature detection.
- Updated CMakeLists.txt to include auto-detection logic for AVX512 features.
- Modified install.sh to include new AVX512_VBMI option for FP8 MoE.
- Enhanced _cpu_detect.py to support progressive matching of CPU variants.
- Created scripts/check_cpu_features.py for manual CPU feature checks.
- Updated setup.py to reflect changes in CPU variant building and environment variables.

* [fix](kt-kernel): Add conditional inclusion of FP8 MoE for AVX512 BF16 support

* [chore](kt-kernel): update project version to 0.5.0 in CMakeLists.txt and version.py
2025-12-24 18:57:45 +08:00
ErvinXie
d8046e1bb4 Kt minimax (#1742)
[feat]: fp8 kernel and kt-cli support
2025-12-24 15:39:44 +08:00
Jiaqi Liao
3c134359bc Fix CPU Instruction Set and Installation (#1729)
* [fix](kt-kernel): fix AVX512 cpu instruction set detection

* [feat](kt-kernel): AVX512 fallback kernel for RAW-INT4

* [fix](kt-kernel): fix setup version issue

* [fix](kt-kernel): update install for custom build

* [docs](kt-kernel): new installation guide for various cpu instruction set

* [fix](kt-kernel): fix _mm512_dpbusd_epi32_compat fallback implmentation

* [style](kt-kernel): clang format
2025-12-18 00:11:57 +08:00
Jiaqi Liao
fcf8882075 [Feature] Add avx-based kimi-k2 support (#1656)
* support Kimi-K2-Thinking original weight
fix amx kernel bug

* update k2 avx kernel.

* feat: add CPUInfer write buffer task

* [feat]: add kimi k2 cpu write buffer support

- Implement write_weights_to_buffer function in k2-moe.hpp for extracting GPU expert weights
- Fix down (w2) weight column-wise slicing for different TP configurations
- Support three TP scenarios: cpu_tp == gpu_tp, cpu_tp > gpu_tp, cpu_tp < gpu_tp
- Add comprehensive test cases for weight extraction validation
- Ensure compatibility with Kimi model's MoE architecture

* [fix]: correct write_weight_scale_to_buffer expert offset calculation

Fixed the bug in write_weight_scale_to_buffer_task where expert offsets in GPU buffers were incorrectly calculated. Changed from using per_expert_gpu sizes to using full gpu_tp sizes, ensuring correct memory layout for multi-expert scenarios.

Also added benchmark scripts for k2 moe and write buffer operations, and cleaned up debug output in test files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat]: add write buffer wrapper

* [fix] fix comment

---------

Co-authored-by: ouqingliang <1692110604@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 16:01:07 +08:00
Jiaqi Liao
94c25626dc Fix kt-kernel for new wrapper (#1588)
* update README for kt-kernel

* style: format C++ and Python code in kt-kernel

  - Format C++ files: task_queue, ext_bindings, and MoE operators
  - Format Python utility modules: amx, llamafile, and loader
  - Improve code readability and consistency
2025-11-10 21:47:34 +08:00
KMSorSMS
62d8685699 [fix]: update moe's physical to logical map 2025-11-07 16:29:04 +00:00
KMSorSMS
2641d15383 [fix]: add amx optimization 2025-11-07 15:51:56 +00:00
chenht2022
6fe30af50d Merge branch 'main' into develop-cht 2025-11-03 14:35:44 +00:00
ovowei
f854d03bd7 update kt-kernel 2025-11-03 15:19:52 +08:00
KMSorSMS
164b13adac [build]: fix cmake env settings bug 2025-11-02 04:49:27 +00:00
chenht2022
4a9b6cd99e Rename sync method parameters 2025-10-28 03:28:41 +00:00
Atream
4c5fcf9774 add kt-kernel 2025-10-12 05:13:00 +00:00