[PR #62] [MERGED] [Fix] Fix problem that ktransformers cannot offload whole layer in cpu #1339

New issue

Closed

opened 2026-10-29 14:51:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:51:33 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/62
Author: @Azure-Tang
Created: 8/29/2024
Status: ✅ Merged
Merged: 8/29/2024
Merged by: @UnicornChan

Base: main ← Head: main

📝 Commits (2)

6735beb Fix cannot offload whole layer in cpu
8747c09 update yaml example; update version idx; update docker file

📊 Changes

8 files changed (+48 additions, -45 deletions)

View changed files

📝 Dockerfile (+1 -1)
📝 ktransformers/__init__.py (+3 -3)
📝 ktransformers/local_chat.py (+4 -2)
📝 ktransformers/operators/experts.py (+3 -3)
📝 ktransformers/operators/linear.py (+2 -2)
📝 ktransformers/operators/models.py (+5 -4)
📝 ktransformers/optimize/optimize_rules/DeepSeek-V2-Chat-multi-gpu-4.yaml (+28 -28)
📝 ktransformers/optimize/optimize_rules/DeepSeek-V2-Chat-multi-gpu.yaml (+2 -2)

📄 Description

Fix bug that ktransformers cannot offload whole layer in cpu.
Update DeepseekV2‘s multi gpu yaml examples to evenly allocate layers.
Update Docker file.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/62 **Author:** [@Azure-Tang](https://github.com/Azure-Tang) **Created:** 8/29/2024 **Status:** ✅ Merged **Merged:** 8/29/2024 **Merged by:** [@UnicornChan](https://github.com/UnicornChan) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (2) - [`6735beb`](https://github.com/kvcache-ai/ktransformers/commit/6735beb5b6b76688cfb1beb3e9d3ddf03585825f) Fix cannot offload whole layer in cpu - [`8747c09`](https://github.com/kvcache-ai/ktransformers/commit/8747c099f200bc98c35fcf85c627021199629a36) update yaml example; update version idx; update docker file ### 📊 Changes **8 files changed** (+48 additions, -45 deletions) <details> <summary>View changed files</summary> 📝 `Dockerfile` (+1 -1) 📝 `ktransformers/__init__.py` (+3 -3) 📝 `ktransformers/local_chat.py` (+4 -2) 📝 `ktransformers/operators/experts.py` (+3 -3) 📝 `ktransformers/operators/linear.py` (+2 -2) 📝 `ktransformers/operators/models.py` (+5 -4) 📝 `ktransformers/optimize/optimize_rules/DeepSeek-V2-Chat-multi-gpu-4.yaml` (+28 -28) 📝 `ktransformers/optimize/optimize_rules/DeepSeek-V2-Chat-multi-gpu.yaml` (+2 -2) </details> ### 📄 Description 1. Fix bug that ktransformers cannot offload whole layer in cpu. 2. Update DeepseekV2‘s multi gpu yaml examples to evenly allocate layers. 3. Update Docker file. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:51:33 +00:00