Commit Graph

12 Commits

Author SHA1 Message Date
mrhaoxx
07fd9328fa refactor(sft): move SFT logic into kt_kernel.sft submodule
- Create python/sft/ with 11 modules: base, amx, arch, autograd, layer,
  lora, weights, wrapper, dist_utils, config, __init__
- Move BaseSFTMoEWrapper + buffer management into sft/base.py (template
  method pattern: subclass provides _make_forward/backward_task)
- Move AMXSFTMoEWrapper into sft/amx.py (thinner, no buffer logic)
- Move from accelerate kt_moe.py: KTMoEFunction, KTMoELayerWrapper,
  MOEArchConfig, PEFT LoRA adaptation, weight extraction, wrapping
- Add KTConfig dataclass (DeepSpeed pattern: opaque config passthrough)
- Add _get_kt_config() with old→new field name compat conversion
- Rename forward_sft→forward, submit_forward_sft→submit_forward,
  sync_forward_sft→sync_forward (Python only, C++ binding names unchanged)
- Delete dump utilities from sft_moe.hpp (-526) and moe-sft-tp.hpp (-78)
- Delete experts_sft.py and utils/amx_sft.py (moved to sft/)
- Remove SFT stubs from BaseMoEWrapper (experts_base.py)
- Lazy SFT import in __init__.py and experts.py (inference isolation)
- Delete all lifecycle/debug logging (~500 lines)

Verified: Qwen3-235B 4GPU AMXBF16 training, 3 steps loss converges.
2026-04-08 23:07:41 +08:00
mrhaoxx
5f6482ff50 [feat]: support skip lora 2026-01-20 05:38:11 +00:00
JimmyPeilinLi
18ab0cb943 [feat](kt-sft-refactor): add KT-SFT to KTMoEWrapper 2026-01-15 12:29:52 +00:00
ErvinXie
d8046e1bb4 Kt minimax (#1742)
[feat]: fp8 kernel and kt-cli support
2025-12-24 15:39:44 +08:00
Jiaqi Liao
fcf8882075 [Feature] Add avx-based kimi-k2 support (#1656)
* support Kimi-K2-Thinking original weight
fix amx kernel bug

* update k2 avx kernel.

* feat: add CPUInfer write buffer task

* [feat]: add kimi k2 cpu write buffer support

- Implement write_weights_to_buffer function in k2-moe.hpp for extracting GPU expert weights
- Fix down (w2) weight column-wise slicing for different TP configurations
- Support three TP scenarios: cpu_tp == gpu_tp, cpu_tp > gpu_tp, cpu_tp < gpu_tp
- Add comprehensive test cases for weight extraction validation
- Ensure compatibility with Kimi model's MoE architecture

* [fix]: correct write_weight_scale_to_buffer expert offset calculation

Fixed the bug in write_weight_scale_to_buffer_task where expert offsets in GPU buffers were incorrectly calculated. Changed from using per_expert_gpu sizes to using full gpu_tp sizes, ensuring correct memory layout for multi-expert scenarios.

Also added benchmark scripts for k2 moe and write buffer operations, and cleaned up debug output in test files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat]: add write buffer wrapper

* [fix] fix comment

---------

Co-authored-by: ouqingliang <1692110604@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-02 16:01:07 +08:00
ZiWei Yuan
1374b98ee5 [feat](moe_kernel): add amd blis support (int8) (#1600)
* [feat]: init amd adaption

* [feat]: add blis support

* [fix]: fix setup and moe kernel warpper

* [fix](setup.py): support rebuild with cache and import kt_kernel works
fine

* [feat]: add moe_kernel converter for amd and implement the load
method(haven't tested yet)

* [feat](moe_kernel/moe.hpp): delete unused memory when using save

* [fix](moe_kernel): update PLAIN for pack

* [fix](moe_kernel): rm printf debug

* [fix](moe_kernel): skip gpu experts

* [fix](moe_kernel/moe.hpp): update include memory path

* [feat](moe_kernel/moe.hpp): support expert deferral

* [feat]: finish amd

---------

Co-authored-by: mrhaoxx <mr.haoxx@gmail.com>
2025-11-27 12:08:53 +08:00
Jiaqi Liao
9bc00e587b Refactor KTMoEWrapper backend (#1587)
* universal backend for cpu inference
* expert defer
2025-11-10 20:26:15 +08:00
chenht2022
6fe30af50d Merge branch 'main' into develop-cht 2025-11-03 14:35:44 +00:00
ovowei
f854d03bd7 update kt-kernel 2025-11-03 15:19:52 +08:00
chenht2022
dd4377b60b feat: add deferred expert scheduling support 2025-10-31 08:03:37 +00:00
ovowei
28d8663374 fix 2025-10-22 18:14:34 +08:00
Atream
4c5fcf9774 add kt-kernel 2025-10-12 05:13:00 +00:00