[PR #889] [CLOSED] Add support for HYGON DCU #1504

New issue

Closed

opened 2026-10-29 14:53:00 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:53:00 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/889
Author: @south-ocean
Created: 3/14/2025
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (2)

8442a74 Add support HYGON DCU
a48adbb Merge branch 'main' into main

📊 Changes

4 files changed (+112 additions, -10 deletions)

View changed files

➕ install_dcu.sh (+20 -0)
📝 ktransformers/ktransformers_ext/cuda/gptq_marlin/gptq_marlin_dtypes.cuh (+1 -1)
➕ ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-dcu.yaml (+76 -0)
📝 setup.py (+15 -9)

📄 Description

This commit code is submit for KME and BW1000, and we also suppply the Z100 method.

Introduction
Enable ktransformers on HYGON DCU.

Compile ktransformers from sourcecode :

pip3 install fastpt (from das)
bash install_dcu.sh

Limitation

Marlin kernel is not supported on DCU(KME and BW1000) and will be support in future , the yaml change can reference by ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-dcu.yaml to Replace all instances of KLinearMarlin with KLinearTorch.

If you want to use ktransformers on Z100, you should do as follows:

Replace all instances of KLinearMarlin with KLinearTorch
Modify local_chat.py to replace all instances of flash_attention_2 with eager.
change code as pictures:

Validation command:
python3 ./ktransformers/local_chat.py --model_path ./DeepSeek-R1 --gguf_path ./DeepSeek-R1-GGUF/ --cpu_infer 65 --max_new_tokens 1000 --optimize-config-path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-dcu.yaml

Chat: who are you
Greetings! I'm DeepSeek-R1, an artificial intelligence assistant created by DeepSeek. I'm at your service and would be delighted to assist ou with any inquiries or tasks you may have.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/889 **Author:** [@south-ocean](https://github.com/south-ocean) **Created:** 3/14/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (2) - [`8442a74`](https://github.com/kvcache-ai/ktransformers/commit/8442a745206637c59173e23f2a3343c1edece7c5) Add support HYGON DCU - [`a48adbb`](https://github.com/kvcache-ai/ktransformers/commit/a48adbb4d53cbf938eb4848982ccf70fff59c2af) Merge branch 'main' into main ### 📊 Changes **4 files changed** (+112 additions, -10 deletions) <details> <summary>View changed files</summary> ➕ `install_dcu.sh` (+20 -0) 📝 `ktransformers/ktransformers_ext/cuda/gptq_marlin/gptq_marlin_dtypes.cuh` (+1 -1) ➕ `ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-dcu.yaml` (+76 -0) 📝 `setup.py` (+15 -9) </details> ### 📄 Description This commit code is submit for KME and BW1000, and we also suppply the Z100 method. Introduction Enable ktransformers on HYGON DCU. Compile ktransformers from sourcecode : pip3 install fastpt (from das) bash install_dcu.sh Limitation Marlin kernel is not supported on DCU(KME and BW1000) and will be support in future , the yaml change can reference by ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-dcu.yaml to Replace all instances of KLinearMarlin with KLinearTorch. If you want to use ktransformers on Z100, you should do as follows: 1. Replace all instances of KLinearMarlin with KLinearTorch 2. Modify local_chat.py to replace all instances of flash_attention_2 with eager. 3. change code as pictures: ![image](https://github.com/user-attachments/assets/00083a66-79c5-4ef3-83ba-d03cf3a8feeb) ![image](https://github.com/user-attachments/assets/73bfd36d-e667-438d-95ab-ef42d073f40d) ![image](https://github.com/user-attachments/assets/8eb92946-ed84-4e29-8b22-5bd696adcf9b) ![image](https://github.com/user-attachments/assets/b29f2b92-9c40-47e6-9c2a-f224203f97b3) Validation command: python3 ./ktransformers/local_chat.py --model_path ./DeepSeek-R1 --gguf_path ./DeepSeek-R1-GGUF/ --cpu_infer 65 --max_new_tokens 1000 --optimize-config-path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-dcu.yaml Chat: who are you Greetings! I'm DeepSeek-R1, an artificial intelligence assistant created by DeepSeek. I'm at your service and would be delighted to assist ou with any inquiries or tasks you may have. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:53:00 +00:00