[GH-ISSUE #1864] Add Habana/Intel Gaudi support #1160

New issue

Closed

opened 2026-10-29 14:49:23 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 14:49:23 +00:00

Owner

Originally created by @alpha-spirit-uvx on GitHub (Feb 23, 2026).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1864

Reminder

I have read the above rules and searched the existing issues.

Description

I'm running a single Intel Gaudi 2 (96 GB HBM2e) with an EPYC 7742 CPU and 1TB system RAM, and I'd love to see Habana support added for dynamic MoE expert offloading (similar to the CUDA/ROCm/Arc backends that are already present). This would be huge for running giant sparse models (like Qwen3.5-397B-A17B) locally without full multi-card setups.

Pull Request

No response

Originally created by @alpha-spirit-uvx on GitHub (Feb 23, 2026). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1864 ### Reminder - [x] I have read the above rules and searched the existing issues. ### Description I'm running a single Intel Gaudi 2 (96 GB HBM2e) with an EPYC 7742 CPU and 1TB system RAM, and I'd love to see Habana support added for dynamic MoE expert offloading (similar to the CUDA/ROCm/Arc backends that are already present). This would be huge for running giant sparse models (like Qwen3.5-397B-A17B) locally without full multi-card setups. ### Pull Request _No response_

bsakaguchi

2026-10-29 14:49:23 +00:00

closed this issue
added the
enhancement
label

bsakaguchi commented

2026-10-29 14:49:24 +00:00

Author

Owner

@JimmyPeilinLi commented on GitHub (Mar 5, 2026):

We don't have this hardware. If you are willing to provide the machine, we can assist in setting it up for you.

@JimmyPeilinLi commented on GitHub (Mar 5, 2026): We don't have this hardware. If you are willing to provide the machine, we can assist in setting it up for you.

bsakaguchi referenced this issue

2026-10-29 14:53:36 +00:00

[PR #1160] [MERGED] Fix bug when loading Marveric Model #1573