[PR #36] [MERGED] Release v0.1.2 #1326

New issue

Closed

opened 2026-10-29 14:51:26 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:51:26 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/36
Author: @UnicornChan
Created: 8/15/2024
Status: ✅ Merged
Merged: 8/15/2024
Merged by: @UnicornChan

Base: main ← Head: develop-0.1.2

📝 Commits (10+)

c1cc7d2 1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.
f293803 Merge pull request #27 from chenht2022/develop-0.1.2
f5f79f5 [ADD] support multi-gpu qlen>1 q5_k
650c368 Merge remote-tracking branch 'upstream/main' into develop-0.1.2
7c4cb52 [feature] support q2_k & q3_k dequantize on gpu
80815db Merge pull request #30 from BITcyman/feature-lc-dequantize_q2k_q3k_gpu
412055d [feature] experts can be injected using CPUInfer
1db4a67 [feature] add github action for pre compile
67043b4 [fix] format classes and files name
c47205d fix name

📊 Changes

69 files changed (+4799 additions, -1961 deletions)

View changed files

➕ .github/workflows/package_wheel_release.yml (+252 -0)
➕ .github/workflows/package_wheel_test.yml (+132 -0)
📝 .gitignore (+4 -1)
📝 README.md (+5 -2)
➕ doc/assets/deepseekv2_structure.png (+0 -0)
➕ doc/assets/model_structure_guild.png (+0 -0)
➕ doc/assets/multi_gpu.png (+0 -0)
📝 doc/en/deepseek-v2-injection.md (+10 -10)
➕ doc/en/injection_tutorial.md (+328 -0)
📝 ktransformers/__init__.py (+1 -1)
📝 ktransformers/ktransformers_ext/CMakeLists.txt (+1 -2)
📝 ktransformers/ktransformers_ext/bench/bench_linear.py (+31 -21)
📝 ktransformers/ktransformers_ext/bench/bench_linear_torch.py (+21 -23)
📝 ktransformers/ktransformers_ext/bench/bench_mlp.py (+31 -21)
📝 ktransformers/ktransformers_ext/bench/bench_mlp_torch.py (+33 -53)
📝 ktransformers/ktransformers_ext/bench/bench_moe.py (+35 -29)
📝 ktransformers/ktransformers_ext/bench/bench_moe_torch.py (+68 -79)
📝 ktransformers/ktransformers_ext/cpu_backend/cpuinfer.h (+30 -6)
📝 ktransformers/ktransformers_ext/cpu_backend/task_queue.h (+1 -1)
📝 ktransformers/ktransformers_ext/cuda/binding.cpp (+8 -2)

...and 49 more files

📄 Description

Support windows native.
Support multiple GPU
Support qlen > 1.
Support new model: mixtral 87B and 822B
Support q2k, q3k dequant on gpu.
Support github action to create pre compile package
Fix some bugs

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/36 **Author:** [@UnicornChan](https://github.com/UnicornChan) **Created:** 8/15/2024 **Status:** ✅ Merged **Merged:** 8/15/2024 **Merged by:** [@UnicornChan](https://github.com/UnicornChan) **Base:** `main` ← **Head:** `develop-0.1.2` --- ### 📝 Commits (10+) - [`c1cc7d2`](https://github.com/kvcache-ai/ktransformers/commit/c1cc7d2cd258af4815536c76f13675b68241c0fa) 1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic. - [`f293803`](https://github.com/kvcache-ai/ktransformers/commit/f2938031567d30e322ab2a76167d04955adbbe64) Merge pull request #27 from chenht2022/develop-0.1.2 - [`f5f79f5`](https://github.com/kvcache-ai/ktransformers/commit/f5f79f5c0ec07ffd0d1b039534c45b8f6b0d939f) [ADD] support multi-gpu qlen>1 q5_k - [`650c368`](https://github.com/kvcache-ai/ktransformers/commit/650c368c187da0acbeb745fd4e0fa0cd5fd3a0d0) Merge remote-tracking branch 'upstream/main' into develop-0.1.2 - [`7c4cb52`](https://github.com/kvcache-ai/ktransformers/commit/7c4cb520bd935f944312604b4e6a79380da6a3ae) [feature] support q2_k & q3_k dequantize on gpu - [`80815db`](https://github.com/kvcache-ai/ktransformers/commit/80815dbc504095ac94fdf2caa9b82c08fa043a12) Merge pull request #30 from BITcyman/feature-lc-dequantize_q2k_q3k_gpu - [`412055d`](https://github.com/kvcache-ai/ktransformers/commit/412055d450fd870d65caf6b17589f46df56b19af) [feature] experts can be injected using CPUInfer - [`1db4a67`](https://github.com/kvcache-ai/ktransformers/commit/1db4a67dca09d0a1d5a8c668502b029f47602192) [feature] add github action for pre compile - [`67043b4`](https://github.com/kvcache-ai/ktransformers/commit/67043b4b5c274e84dca0ee3ace21a7f857721711) [fix] format classes and files name - [`c47205d`](https://github.com/kvcache-ai/ktransformers/commit/c47205dce9b7a64a47fc4cd3cd55f2fd6926f98b) fix name ### 📊 Changes **69 files changed** (+4799 additions, -1961 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/package_wheel_release.yml` (+252 -0) ➕ `.github/workflows/package_wheel_test.yml` (+132 -0) 📝 `.gitignore` (+4 -1) 📝 `README.md` (+5 -2) ➕ `doc/assets/deepseekv2_structure.png` (+0 -0) ➕ `doc/assets/model_structure_guild.png` (+0 -0) ➕ `doc/assets/multi_gpu.png` (+0 -0) 📝 `doc/en/deepseek-v2-injection.md` (+10 -10) ➕ `doc/en/injection_tutorial.md` (+328 -0) 📝 `ktransformers/__init__.py` (+1 -1) 📝 `ktransformers/ktransformers_ext/CMakeLists.txt` (+1 -2) 📝 `ktransformers/ktransformers_ext/bench/bench_linear.py` (+31 -21) 📝 `ktransformers/ktransformers_ext/bench/bench_linear_torch.py` (+21 -23) 📝 `ktransformers/ktransformers_ext/bench/bench_mlp.py` (+31 -21) 📝 `ktransformers/ktransformers_ext/bench/bench_mlp_torch.py` (+33 -53) 📝 `ktransformers/ktransformers_ext/bench/bench_moe.py` (+35 -29) 📝 `ktransformers/ktransformers_ext/bench/bench_moe_torch.py` (+68 -79) 📝 `ktransformers/ktransformers_ext/cpu_backend/cpuinfer.h` (+30 -6) 📝 `ktransformers/ktransformers_ext/cpu_backend/task_queue.h` (+1 -1) 📝 `ktransformers/ktransformers_ext/cuda/binding.cpp` (+8 -2) _...and 49 more files_ </details> ### 📄 Description 1. Support windows native. 2. Support multiple GPU 3. Support qlen > 1. 4. Support new model: mixtral 8*7B and 8*22B 5. Support q2k, q3k dequant on gpu. 6. Support github action to create pre compile package 7. Fix some bugs --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:51:26 +00:00