[PR #413] [MERGED] Fix precision mla #1417

New issue

Closed

opened 2026-10-29 14:52:14 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:14 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/413
Author: @Atream
Created: 2/17/2025
Status: ✅ Merged
Merged: 2/18/2025
Merged by: @Atream

Base: main ← Head: fix_precision_MLA

📝 Commits (7)

b845246 support bf16 read
038bc30 fix precision bug imported by position_ids in 0.2.0
c176e51 server mix mla
9f1da18 update install.sh
f029588 fix server warmup
07a0555 ⚡ fix device and add test
592e13d ⚡ add mmlu_pro test

📊 Changes

18 files changed (+899 additions, -51 deletions)

View changed files

📝 .gitignore (+5 -0)
📝 Makefile (+5 -1)
📝 install.sh (+2 -0)
📝 ktransformers/local_chat.py (+11 -3)
📝 ktransformers/models/custom_cache.py (+0 -2)
📝 ktransformers/operators/RoPE.py (+9 -8)
📝 ktransformers/operators/attention.py (+189 -22)
📝 ktransformers/operators/base_operator.py (+5 -2)
📝 ktransformers/operators/experts.py (+5 -1)
➕ ktransformers/operators/flashinfer_wrapper.py (+240 -0)
📝 ktransformers/operators/gate.py (+2 -2)
📝 ktransformers/operators/linear.py (+1 -1)
📝 ktransformers/server/backend/interfaces/ktransformers.py (+11 -3)
📝 ktransformers/server/backend/interfaces/transformers.py (+8 -2)
➕ ktransformers/tests/mmlu_pro_test.py (+195 -0)
➕ ktransformers/tests/mmlu_test.py (+195 -0)
📝 ktransformers/util/custom_gguf.py (+7 -0)
📝 ktransformers/util/utils.py (+9 -4)

📄 Description

No description provided

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/413 **Author:** [@Atream](https://github.com/Atream) **Created:** 2/17/2025 **Status:** ✅ Merged **Merged:** 2/18/2025 **Merged by:** [@Atream](https://github.com/Atream) **Base:** `main` ← **Head:** `fix_precision_MLA` --- ### 📝 Commits (7) - [`b845246`](https://github.com/kvcache-ai/ktransformers/commit/b84524622ef0cbfefb412f4c86eb2b6ec032bfec) support bf16 read - [`038bc30`](https://github.com/kvcache-ai/ktransformers/commit/038bc30888e004ce2409c339d31a9fcfe7aa6cf2) fix precision bug imported by position_ids in 0.2.0 - [`c176e51`](https://github.com/kvcache-ai/ktransformers/commit/c176e516b5d0c5be86111f2942992f37bbcdc53d) server mix mla - [`9f1da18`](https://github.com/kvcache-ai/ktransformers/commit/9f1da18630ebaccefab45b656fb9c5a66b680588) update install.sh - [`f029588`](https://github.com/kvcache-ai/ktransformers/commit/f029588b61b0385c6cffa414438ab47877ae314b) fix server warmup - [`07a0555`](https://github.com/kvcache-ai/ktransformers/commit/07a0555016a967abbddfeda26354b750a745eea5) :zap: fix device and add test - [`592e13d`](https://github.com/kvcache-ai/ktransformers/commit/592e13d4533a592695e8e4cf52a08f4723e4a934) :zap: add mmlu_pro test ### 📊 Changes **18 files changed** (+899 additions, -51 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+5 -0) 📝 `Makefile` (+5 -1) 📝 `install.sh` (+2 -0) 📝 `ktransformers/local_chat.py` (+11 -3) 📝 `ktransformers/models/custom_cache.py` (+0 -2) 📝 `ktransformers/operators/RoPE.py` (+9 -8) 📝 `ktransformers/operators/attention.py` (+189 -22) 📝 `ktransformers/operators/base_operator.py` (+5 -2) 📝 `ktransformers/operators/experts.py` (+5 -1) ➕ `ktransformers/operators/flashinfer_wrapper.py` (+240 -0) 📝 `ktransformers/operators/gate.py` (+2 -2) 📝 `ktransformers/operators/linear.py` (+1 -1) 📝 `ktransformers/server/backend/interfaces/ktransformers.py` (+11 -3) 📝 `ktransformers/server/backend/interfaces/transformers.py` (+8 -2) ➕ `ktransformers/tests/mmlu_pro_test.py` (+195 -0) ➕ `ktransformers/tests/mmlu_test.py` (+195 -0) 📝 `ktransformers/util/custom_gguf.py` (+7 -0) 📝 `ktransformers/util/utils.py` (+9 -4) </details> ### 📄 Description _No description provided_ --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:14 +00:00