Commit Graph

26 Commits

Author SHA1 Message Date
Jiaqi Liao
e69c67713f [refactor] fix third_party issue (#1632)
* [refactor]: relocate third_party directory

* [fix]: fix custom_flashinfer for kt-sft
2025-11-20 13:55:55 +08:00
Jiaqi Liao
57d14d22bc Refactor: restructure repository to focus on kt-kernel and KT-SFT modulesq recon (#1581)
* refactor: move legacy code to archive/ directory

  - Moved ktransformers, csrc, third_party, merge_tensors to archive/
  - Moved build scripts and configurations to archive/
  - Kept kt-kernel, KT-SFT, doc, and README files in root
  - Preserved complete git history for all moved files

* refactor: restructure repository to focus on kt-kernel and KT-SFT modules

* fix README

* fix README

* fix README

* fix README

* docs: add performance benchmarks to kt-kernel section

Add comprehensive performance data for kt-kernel to match KT-SFT's presentation:
- AMX kernel optimization: 21.3 TFLOPS (3.9× faster than PyTorch)
- Prefill phase: up to 20× speedup vs baseline
- Decode phase: up to 4× speedup
- NUMA optimization: up to 63% throughput improvement
- Multi-GPU (8×L20): 227.85 tokens/s total throughput with DeepSeek-R1 FP8

Source: https://lmsys.org/blog/2025-10-22-KTransformers/

This provides users with concrete performance metrics for both core modules,
making it easier to understand the capabilities of each component.

* refactor: improve kt-kernel performance data with specific hardware and models

Replace generic performance descriptions with concrete benchmarks:
- Specify exact hardware: 8×L20 GPU + Xeon Gold 6454S, Single/Dual-socket Xeon + AMX
- Include specific models: DeepSeek-R1-0528 (FP8), DeepSeek-V3 (671B)
- Show detailed metrics: total throughput, output throughput, concurrency details
- Match KT-SFT presentation style for consistency

This provides users with actionable performance data they can use to evaluate
hardware requirements and expected performance for their use cases.

* fix README

* docs: clean up performance table and improve formatting

* add pic for README

* refactor: simplify .gitmodules and backup legacy submodules

- Remove 7 legacy submodules from root .gitmodules (archive/third_party/*)
- Keep only 2 active submodules for kt-kernel (llama.cpp, pybind11)
- Backup complete .gitmodules to archive/.gitmodules
- Add documentation in archive/README.md for researchers who need legacy submodules

This reduces initial clone size by ~500MB and avoids downloading unused dependencies.

* refactor: move doc/ back to root directory

Keep documentation in root for easier access and maintenance.

* refactor: consolidate all images to doc/assets/

- Move kt-kernel/assets/heterogeneous_computing.png to doc/assets/
- Remove KT-SFT/assets/ (images already in doc/assets/)
- Update KT-SFT/README.md image references to ../doc/assets/
- Eliminates ~7.9MB image duplication
- Centralizes all documentation assets in one location

* fix pic path for README
2025-11-10 17:42:26 +08:00
wanghanqingLYT
e7e2c2bd70 merge arm branch for sgemm.cpp, tinyblas_cpu_sgemm.inc and iqk_mul_mat.inc 2025-09-08 14:11:17 +08:00
ouqingliang
3b4a1c7532 add prefix cache support for kvc2. 2025-06-26 04:57:25 +00:00
Atream
753075728c update-custom-flashinfer 2025-04-30 10:45:25 +00:00
qiyuxinlin
74bb7fdcf6 Merge remote-tracking branch 'dev/support-amx-2' 2025-04-28 18:46:51 +00:00
djw
3f9bbf1181 support qwen3, dont speak human language 2025-04-28 08:44:47 +00:00
PC-DOS
5379e68f19 Replaced Chinese comments with English to avoid breaking MSVC compiling 2025-04-26 03:20:23 +08:00
PC-DOS
5b4d9c41ac Replaced Chinese comments with English to avoid breaking MSVC compiling 2025-04-26 03:18:01 +08:00
qiyuxinlin
03a65d6bea roll back ktransformers backend, add max_tokens, max_completion_tokens param 2025-04-21 12:55:37 +00:00
Iwan Kawrakow
99a247e167 Spelling 2025-04-11 10:15:42 +03:00
Iwan Kawrakow
c46b0c59d0 Add missing references to ik_llama.cpp 2025-04-11 09:39:57 +03:00
Atream
80c5cbecdd add nlohmann 2025-04-01 10:38:45 +08:00
Atream
9360d1e3c8 add submodules 2025-03-31 23:20:29 +08:00
liu.shen
26bd889ff8 fix #829: 兼容Intel Cascade Lake架构的CPU 2025-03-09 19:26:12 +08:00
liam
8eeb6dd432 update compile option for avx512vpopcntdq 2025-03-06 12:18:04 +08:00
Azure
dd390835ca Add compile condition 2025-03-06 03:25:39 +00:00
Azure
8068018504 fix gcc compilation 2025-03-05 15:59:56 +00:00
moonshadow-25
d24d369332 iq1s files 2025-03-01 22:44:06 +08:00
moonshadow-25
c513ae59c3 iq1s files 2025-03-01 22:38:04 +08:00
moonshadow-25
9781d1e6f4 iq1s core 2025-03-01 21:48:25 +08:00
godrosev
93c5b75716 rem 2025-03-01 21:25:18 +08:00
godrosev
e6349eb240 iq1s 2025-03-01 21:00:11 +08:00
chenxl
1db4a67dca [feature] add github action for pre compile 2024-08-14 16:54:50 +00:00
chenxl
f5f79f5c0e [ADD] support multi-gpu qlen>1 q5_k 2024-08-12 11:41:26 +00:00
chenxl
18c42e67df Initial commit 2024-07-27 16:06:58 +08:00