[PR #799] [MERGED] Restore CPU offloading capability #1484

New issue

Closed

opened 2026-10-29 14:52:50 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:50 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/799
Author: @aubreyli
Created: 3/5/2025
Status: ✅ Merged
Merged: 5/9/2025
Merged by: @Atream

Base: main ← Head: cpu_offloading

📝 Commits (1)

f4d52d1 Restore CPU offloading capability

📊 Changes

3 files changed (+192 additions, -2 deletions)

View changed files

📝 ktransformers/operators/attention.py (+4 -1)
📝 ktransformers/operators/models.py (+4 -1)
➕ ktransformers/optimize/optimize_rules/DeepSeek-V2-Lite-Chat-gpu-cpu.yaml (+184 -0)

📄 Description

Restore CPU offloading capability.
Provided an example yaml for deepseek-V2-Lite-Chat model.
Patch tested by both default GPU case and GPU+CPU case.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/799 **Author:** [@aubreyli](https://github.com/aubreyli) **Created:** 3/5/2025 **Status:** ✅ Merged **Merged:** 5/9/2025 **Merged by:** [@Atream](https://github.com/Atream) **Base:** `main` ← **Head:** `cpu_offloading` --- ### 📝 Commits (1) - [`f4d52d1`](https://github.com/kvcache-ai/ktransformers/commit/f4d52d1f0c72e9ca14f175c3fe90db65054898f4) Restore CPU offloading capability ### 📊 Changes **3 files changed** (+192 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `ktransformers/operators/attention.py` (+4 -1) 📝 `ktransformers/operators/models.py` (+4 -1) ➕ `ktransformers/optimize/optimize_rules/DeepSeek-V2-Lite-Chat-gpu-cpu.yaml` (+184 -0) </details> ### 📄 Description Restore CPU offloading capability. Provided an example yaml for deepseek-V2-Lite-Chat model. Patch tested by both default GPU case and GPU+CPU case. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:50 +00:00