[PR #122] [MERGED] [Feat] add support to DeepSeekV3 #1354

New issue

Closed

opened 2026-10-29 14:51:41 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:51:41 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/122
Author: @Azure-Tang
Created: 1/31/2025
Status: ✅ Merged
Merged: 2/10/2025
Merged by: @UnicornChan

Base: main ← Head: feat-DeepSeekV3

📝 Commits (10+)

476b1d8 support deepseekv3; runable but have precition problem
5a50b34 fix hard coding caused by rope dim calculation, load from config now
f873558 update rope calculation; update modeling.py; update gate for moe
f748cd2 fix rope; update moegate
907251c done support deepseekv3
ee24a27 update v3 single gpu rule yaml;
027b112 modify moeinfer param
3dca28d ⚡ fix moe.cpp int overflow problem
0262f95 Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3
c4d9bc6 support KExpertsMarlin backend

📊 Changes

32 files changed (+4460 additions, -159 deletions)

View changed files

📝 .github/workflows/package_wheel_release.yml (+2 -2)
📝 .github/workflows/package_wheel_test.yml (+2 -2)
📝 .gitignore (+4 -1)
📝 Makefile (+1 -1)
📝 README.md (+39 -24)
➕ doc/en/DeepseekR1_V3_tutorial.md (+139 -0)
📝 ktransformers/__init__.py (+3 -3)
📝 ktransformers/configs/config.yaml (+1 -1)
📝 ktransformers/ktransformers_ext/CMakeLists.txt (+21 -0)
📝 ktransformers/ktransformers_ext/cpu_backend/backend.cpp (+17 -0)
📝 ktransformers/ktransformers_ext/cpu_backend/backend.h (+3 -0)
📝 ktransformers/ktransformers_ext/operators/llamafile/moe.cpp (+77 -2)
📝 ktransformers/ktransformers_ext/operators/llamafile/moe.h (+6 -0)
📝 ktransformers/local_chat.py (+218 -39)
➕ ktransformers/models/configuration_deepseek_v3.py (+235 -0)
📝 ktransformers/models/custom_cache.py (+11 -4)
➕ ktransformers/models/modeling_deepseek_v3.py (+1936 -0)
📝 ktransformers/operators/RoPE.py (+189 -1)
📝 ktransformers/operators/attention.py (+202 -0)
📝 ktransformers/operators/experts.py (+144 -20)

...and 12 more files

📄 Description

Add support to DeepSeekV3

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/122 **Author:** [@Azure-Tang](https://github.com/Azure-Tang) **Created:** 1/31/2025 **Status:** ✅ Merged **Merged:** 2/10/2025 **Merged by:** [@UnicornChan](https://github.com/UnicornChan) **Base:** `main` ← **Head:** `feat-DeepSeekV3` --- ### 📝 Commits (10+) - [`476b1d8`](https://github.com/kvcache-ai/ktransformers/commit/476b1d8dc69da6c8823cc7a7d004e574d1cabaf1) support deepseekv3; runable but have precition problem - [`5a50b34`](https://github.com/kvcache-ai/ktransformers/commit/5a50b346271b93a828c3002320be6f14e3426433) fix hard coding caused by rope dim calculation, load from config now - [`f873558`](https://github.com/kvcache-ai/ktransformers/commit/f873558a893d611444ad73ecabae4e0ea21f1713) update rope calculation; update modeling.py; update gate for moe - [`f748cd2`](https://github.com/kvcache-ai/ktransformers/commit/f748cd29f0c63bbe39ab617b701eabe58b6753db) fix rope; update moegate - [`907251c`](https://github.com/kvcache-ai/ktransformers/commit/907251c7432d3f4d61db3c3579443379b5633356) done support deepseekv3 - [`ee24a27`](https://github.com/kvcache-ai/ktransformers/commit/ee24a27001d8eadd798dfea5983b16a4f49fdca8) update v3 single gpu rule yaml; - [`027b112`](https://github.com/kvcache-ai/ktransformers/commit/027b11266c9bb59ca4853fa854aeacbb2370e7c0) modify moeinfer param - [`3dca28d`](https://github.com/kvcache-ai/ktransformers/commit/3dca28d23b1fac0c38c746fd3e15851572a54460) :zap: fix moe.cpp int overflow problem - [`0262f95`](https://github.com/kvcache-ai/ktransformers/commit/0262f954c7c9df82e60b5e94e28794f61bf7aed7) Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3 - [`c4d9bc6`](https://github.com/kvcache-ai/ktransformers/commit/c4d9bc6670c927abb82f2436762da3fc8efb2bf3) support KExpertsMarlin backend ### 📊 Changes **32 files changed** (+4460 additions, -159 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/package_wheel_release.yml` (+2 -2) 📝 `.github/workflows/package_wheel_test.yml` (+2 -2) 📝 `.gitignore` (+4 -1) 📝 `Makefile` (+1 -1) 📝 `README.md` (+39 -24) ➕ `doc/en/DeepseekR1_V3_tutorial.md` (+139 -0) 📝 `ktransformers/__init__.py` (+3 -3) 📝 `ktransformers/configs/config.yaml` (+1 -1) 📝 `ktransformers/ktransformers_ext/CMakeLists.txt` (+21 -0) 📝 `ktransformers/ktransformers_ext/cpu_backend/backend.cpp` (+17 -0) 📝 `ktransformers/ktransformers_ext/cpu_backend/backend.h` (+3 -0) 📝 `ktransformers/ktransformers_ext/operators/llamafile/moe.cpp` (+77 -2) 📝 `ktransformers/ktransformers_ext/operators/llamafile/moe.h` (+6 -0) 📝 `ktransformers/local_chat.py` (+218 -39) ➕ `ktransformers/models/configuration_deepseek_v3.py` (+235 -0) 📝 `ktransformers/models/custom_cache.py` (+11 -4) ➕ `ktransformers/models/modeling_deepseek_v3.py` (+1936 -0) 📝 `ktransformers/operators/RoPE.py` (+189 -1) 📝 `ktransformers/operators/attention.py` (+202 -0) 📝 `ktransformers/operators/experts.py` (+144 -20) _...and 12 more files_ </details> ### 📄 Description Add support to DeepSeekV3 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:51:41 +00:00