[PR #754] [MERGED] Support for IQ1_S(Dynamic 1.58-bit) #1473

New issue

Closed

opened 2026-10-29 14:52:44 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:52:44 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/ktransformers/pull/754
Author: @moonshadow-25
Created: 3/1/2025
Status: ✅ Merged
Merged: 3/3/2025
Merged by: @Azure-Tang

Base: develop-0.2.3 ← Head: dev023

📝 Commits (5)

e6349eb iq1s
93c5b75 rem
9781d1e iq1s core
c513ae5 iq1s files
d24d369 iq1s files

📊 Changes

3 files changed (+1870 additions, -89 deletions)

View changed files

📝 third_party/llamafile/iqk_mul_mat.inc (+1854 -75)
📝 third_party/llamafile/sgemm.h (+8 -3)
📝 third_party/llamafile/tinyblas_cpu_sgemm.inc (+8 -11)

📄 Description

This PR adds support for IQ1_S.
This can further lower the barrier to entry for KTransformers.
Now you only need 60G of RAM + 16G of VRAM to run 671B's Deepseek smoothly
You can download the model here

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/ktransformers/pull/754 **Author:** [@moonshadow-25](https://github.com/moonshadow-25) **Created:** 3/1/2025 **Status:** ✅ Merged **Merged:** 3/3/2025 **Merged by:** [@Azure-Tang](https://github.com/Azure-Tang) **Base:** `develop-0.2.3` ← **Head:** `dev023` --- ### 📝 Commits (5) - [`e6349eb`](https://github.com/kvcache-ai/ktransformers/commit/e6349eb2404e4bb57a5f459ed9ecf151e1fb5ea6) iq1s - [`93c5b75`](https://github.com/kvcache-ai/ktransformers/commit/93c5b757160a8f613c17596f90e0ad94718ed952) rem - [`9781d1e`](https://github.com/kvcache-ai/ktransformers/commit/9781d1e6f4c83341810ba9165d59a7b1317becf4) iq1s core - [`c513ae5`](https://github.com/kvcache-ai/ktransformers/commit/c513ae59c32a2021aff404c6febcae1245df8069) iq1s files - [`d24d369`](https://github.com/kvcache-ai/ktransformers/commit/d24d3693329b274e99d541b86dc07662391ed38e) iq1s files ### 📊 Changes **3 files changed** (+1870 additions, -89 deletions) <details> <summary>View changed files</summary> 📝 `third_party/llamafile/iqk_mul_mat.inc` (+1854 -75) 📝 `third_party/llamafile/sgemm.h` (+8 -3) 📝 `third_party/llamafile/tinyblas_cpu_sgemm.inc` (+8 -11) </details> ### 📄 Description This PR adds support for IQ1_S. This can further lower the barrier to entry for KTransformers. Now you only need 60G of RAM + 16G of VRAM to run 671B's Deepseek smoothly You can download the model [here](https://unsloth.ai/blog/deepseekr1-dynamic) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:52:44 +00:00