[PR #663] [MERGED] [release] 0.2.2rc. #1451

New issue

Closed

opened 2026-10-29 14:52:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:32 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/663
Author: @Azure-Tang
Created: 2/25/2025
Status: ✅ Merged
Merged: 2/25/2025
Merged by: @Azure-Tang

Base: main ← Head: develop-0.2.2

📝 Commits (10+)

7b7c6a6 Add fp8 linear kernel;\n Add empty cache to fit in 16G VRAM; By 'wkGCaSS - 知乎 https://zhuanlan.zhihu.com/p/25491611225'
006e8c6 remove causal mask
e9b1216 Merge branch 'main' into feat-absorb-for-long-prefill
581a524 Add data loader to read special weights for fp8; Add special weight process script
ca7366d Merge remote-tracking branch 'upstream/develop-0.2.2' into support-fp8
4dc5518 update fp8 kernel tutorial
36fbeee Update doc
050b745 Merge pull request #643 from Azure-Tang/support-fp8
f4c198b support absorb for prefill long context
b443c7d Merge pull request #657 from kvcache-ai/feat-absorb-for-long-prefill

📊 Changes

34 files changed (+1313 additions, -83 deletions)

View changed files

➕ .devcontainer/Dockerfile (+19 -0)
➕ .devcontainer/devcontainer.json (+34 -0)
📝 .gitignore (+2 -6)
📝 README.md (+3 -2)
📝 README_ZH.md (+9 -8)
📝 doc/README.md (+1 -0)
📝 doc/SUMMARY.md (+2 -1)
📝 doc/en/DeepseekR1_V3_tutorial.md (+13 -0)
📝 doc/en/FAQ.md (+30 -2)
➕ doc/en/benchmark.md (+43 -0)
➕ doc/en/fp8_kernel.md (+74 -0)
📝 doc/en/injection_tutorial.md (+1 -0)
📝 doc/en/install.md (+1 -1)
➕ ktransformers/ktransformers_ext/triton/fp8gemm.py (+193 -0)
📝 ktransformers/local_chat.py (+7 -7)
📝 ktransformers/operators/attention.py (+37 -18)
📝 ktransformers/operators/experts.py (+10 -1)
📝 ktransformers/operators/flashinfer_wrapper.py (+30 -9)
📝 ktransformers/operators/gate.py (+10 -3)
📝 ktransformers/operators/linear.py (+68 -5)

...and 14 more files

📄 Description

Release note 0.2.2rc

New features:

Support longer context in 24GB VRAM. #657
Support FP8 GPU kernel. #643

Fix some bugs.
Update benchmark scripts and scores. #659

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/663 **Author:** [@Azure-Tang](https://github.com/Azure-Tang) **Created:** 2/25/2025 **Status:** ✅ Merged **Merged:** 2/25/2025 **Merged by:** [@Azure-Tang](https://github.com/Azure-Tang) **Base:** `main` ← **Head:** `develop-0.2.2` --- ### 📝 Commits (10+) - [`7b7c6a6`](https://github.com/kvcache-ai/ktransformers/commit/7b7c6a657d8f4bb7a0e330dd91c03a3e81e802f4) Add fp8 linear kernel;\n Add empty cache to fit in 16G VRAM; By 'wkGCaSS - 知乎 https://zhuanlan.zhihu.com/p/25491611225' - [`006e8c6`](https://github.com/kvcache-ai/ktransformers/commit/006e8c6abc6503921411db935efd80ad7f16032d) remove causal mask - [`e9b1216`](https://github.com/kvcache-ai/ktransformers/commit/e9b1216a9ad5797ab5c3c34d4e087134514692fe) Merge branch 'main' into feat-absorb-for-long-prefill - [`581a524`](https://github.com/kvcache-ai/ktransformers/commit/581a524f65db422011d9a8db99439d658223db6f) Add data loader to read special weights for fp8; Add special weight process script - [`ca7366d`](https://github.com/kvcache-ai/ktransformers/commit/ca7366d2dbb81751e65f1fa8b7b5fba9165f9f42) Merge remote-tracking branch 'upstream/develop-0.2.2' into support-fp8 - [`4dc5518`](https://github.com/kvcache-ai/ktransformers/commit/4dc5518e4d2ae89a687709bcbe05d2f3f80e00ad) update fp8 kernel tutorial - [`36fbeee`](https://github.com/kvcache-ai/ktransformers/commit/36fbeee341e283a93b6befa2a4d9085b7a5dd2b1) Update doc - [`050b745`](https://github.com/kvcache-ai/ktransformers/commit/050b745a6e2d94423bf1138f585a94b661210bff) Merge pull request #643 from Azure-Tang/support-fp8 - [`f4c198b`](https://github.com/kvcache-ai/ktransformers/commit/f4c198bd42f037ccc570eb8f1f1b4ab1ea9c7fa2) support absorb for prefill long context - [`b443c7d`](https://github.com/kvcache-ai/ktransformers/commit/b443c7dfa2baeae84658351898eae90d6c330f61) Merge pull request #657 from kvcache-ai/feat-absorb-for-long-prefill ### 📊 Changes **34 files changed** (+1313 additions, -83 deletions) <details> <summary>View changed files</summary> ➕ `.devcontainer/Dockerfile` (+19 -0) ➕ `.devcontainer/devcontainer.json` (+34 -0) 📝 `.gitignore` (+2 -6) 📝 `README.md` (+3 -2) 📝 `README_ZH.md` (+9 -8) 📝 `doc/README.md` (+1 -0) 📝 `doc/SUMMARY.md` (+2 -1) 📝 `doc/en/DeepseekR1_V3_tutorial.md` (+13 -0) 📝 `doc/en/FAQ.md` (+30 -2) ➕ `doc/en/benchmark.md` (+43 -0) ➕ `doc/en/fp8_kernel.md` (+74 -0) 📝 `doc/en/injection_tutorial.md` (+1 -0) 📝 `doc/en/install.md` (+1 -1) ➕ `ktransformers/ktransformers_ext/triton/fp8gemm.py` (+193 -0) 📝 `ktransformers/local_chat.py` (+7 -7) 📝 `ktransformers/operators/attention.py` (+37 -18) 📝 `ktransformers/operators/experts.py` (+10 -1) 📝 `ktransformers/operators/flashinfer_wrapper.py` (+30 -9) 📝 `ktransformers/operators/gate.py` (+10 -3) 📝 `ktransformers/operators/linear.py` (+68 -5) _...and 14 more files_ </details> ### 📄 Description ## Release note 0.2.2rc New features: - Support longer context in 24GB VRAM. #657 - Support FP8 GPU kernel. #643 Fix some bugs. Update benchmark scripts and scores. #659 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:32 +00:00