[PR #27] [MERGED] [Feature] towards 0.1.2 #1322

New issue

Closed

opened 2026-10-29 14:51:25 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:51:25 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/27
Author: @chenht2022
Created: 8/8/2024
Status: ✅ Merged
Merged: 8/9/2024
Merged by: @UnicornChan

Base: develop-0.1.2 ← Head: develop-0.1.2

📝 Commits (1)

c1cc7d2 1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.

📊 Changes

21 files changed (+749 additions, -731 deletions)

View changed files

📝 ktransformers/ktransformers_ext/CMakeLists.txt (+1 -2)
📝 ktransformers/ktransformers_ext/bench/bench_linear.py (+31 -21)
📝 ktransformers/ktransformers_ext/bench/bench_linear_torch.py (+21 -23)
📝 ktransformers/ktransformers_ext/bench/bench_mlp.py (+31 -21)
📝 ktransformers/ktransformers_ext/bench/bench_mlp_torch.py (+33 -53)
📝 ktransformers/ktransformers_ext/bench/bench_moe.py (+35 -29)
📝 ktransformers/ktransformers_ext/bench/bench_moe_torch.py (+68 -79)
📝 ktransformers/ktransformers_ext/cpu_backend/cpuinfer.h (+30 -6)
📝 ktransformers/ktransformers_ext/examples/test_linear.py (+22 -43)
📝 ktransformers/ktransformers_ext/examples/test_mlp.py (+35 -51)
📝 ktransformers/ktransformers_ext/examples/test_moe.py (+73 -65)
📝 ktransformers/ktransformers_ext/ext_bindings.cpp (+118 -203)
📝 ktransformers/ktransformers_ext/operators/llamafile/linear.cpp (+38 -12)
📝 ktransformers/ktransformers_ext/operators/llamafile/linear.h (+11 -7)
📝 ktransformers/ktransformers_ext/operators/llamafile/mlp.cpp (+55 -35)
📝 ktransformers/ktransformers_ext/operators/llamafile/mlp.h (+16 -12)
📝 ktransformers/ktransformers_ext/operators/llamafile/moe.cpp (+32 -63)
📝 ktransformers/ktransformers_ext/operators/llamafile/moe.h (+3 -2)
➕ ktransformers/ktransformers_ext/operators/llamafile/shared_mem_buffer.cpp (+55 -0)
➕ ktransformers/ktransformers_ext/operators/llamafile/shared_mem_buffer.h (+37 -0)

...and 1 more files

📄 Description

Linear and MLP operators support qlen>1.
All operators now share a single memory buffer.
Refactor CPUInfer submit/sync logic.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/27 **Author:** [@chenht2022](https://github.com/chenht2022) **Created:** 8/8/2024 **Status:** ✅ Merged **Merged:** 8/9/2024 **Merged by:** [@UnicornChan](https://github.com/UnicornChan) **Base:** `develop-0.1.2` ← **Head:** `develop-0.1.2` --- ### 📝 Commits (1) - [`c1cc7d2`](https://github.com/kvcache-ai/ktransformers/commit/c1cc7d2cd258af4815536c76f13675b68241c0fa) 1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic. ### 📊 Changes **21 files changed** (+749 additions, -731 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/ktransformers_ext/CMakeLists.txt` (+1 -2) 📝 `ktransformers/ktransformers_ext/bench/bench_linear.py` (+31 -21) 📝 `ktransformers/ktransformers_ext/bench/bench_linear_torch.py` (+21 -23) 📝 `ktransformers/ktransformers_ext/bench/bench_mlp.py` (+31 -21) 📝 `ktransformers/ktransformers_ext/bench/bench_mlp_torch.py` (+33 -53) 📝 `ktransformers/ktransformers_ext/bench/bench_moe.py` (+35 -29) 📝 `ktransformers/ktransformers_ext/bench/bench_moe_torch.py` (+68 -79) 📝 `ktransformers/ktransformers_ext/cpu_backend/cpuinfer.h` (+30 -6) 📝 `ktransformers/ktransformers_ext/examples/test_linear.py` (+22 -43) 📝 `ktransformers/ktransformers_ext/examples/test_mlp.py` (+35 -51) 📝 `ktransformers/ktransformers_ext/examples/test_moe.py` (+73 -65) 📝 `ktransformers/ktransformers_ext/ext_bindings.cpp` (+118 -203) 📝 `ktransformers/ktransformers_ext/operators/llamafile/linear.cpp` (+38 -12) 📝 `ktransformers/ktransformers_ext/operators/llamafile/linear.h` (+11 -7) 📝 `ktransformers/ktransformers_ext/operators/llamafile/mlp.cpp` (+55 -35) 📝 `ktransformers/ktransformers_ext/operators/llamafile/mlp.h` (+16 -12) 📝 `ktransformers/ktransformers_ext/operators/llamafile/moe.cpp` (+32 -63) 📝 `ktransformers/ktransformers_ext/operators/llamafile/moe.h` (+3 -2) ➕ `ktransformers/ktransformers_ext/operators/llamafile/shared_mem_buffer.cpp` (+55 -0) ➕ `ktransformers/ktransformers_ext/operators/llamafile/shared_mem_buffer.h` (+37 -0) _...and 1 more files_ </details> ### 📄 Description - Linear and MLP operators support qlen>1. - All operators now share a single memory buffer. - Refactor CPUInfer submit/sync logic. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:51:25 +00:00