mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-03-13 07:20:15 +00:00
31 lines
770 B
Markdown
31 lines
770 B
Markdown
### 🔀 [#115](https://github.com/ikawrakow/ik_llama.cpp/pull/115) - MMQ for Q6_0
|
|
|
|
| **Author** | `ikawrakow` |
|
|
| :--- | :--- |
|
|
| **State** | ❌ **Closed** |
|
|
| **Created** | 2024-11-20 |
|
|
| **Updated** | 2024-11-21 |
|
|
|
|
---
|
|
|
|
#### Description
|
|
|
|
Add MMQ kernel for `Q6_0`.
|
|
|
|
@Nexesenex
|
|
|
|
---
|
|
|
|
#### 💬 Conversation
|
|
|
|
👤 **Nexesenex** commented the **2024-11-20** at **19:42:01**:<br>
|
|
|
|
Tested successfully on IK_LLame, PPL is 0.1% above Q6_K on a pure quant of Sheared Llama 2.7b.
|
|
Thanks IK. I'll play with the Qwen models in the next days.
|
|
|
|
---
|
|
|
|
👤 **Nexesenex** commented the **2024-11-20** at **19:42:56**:<br>
|
|
|
|
Tested successfully on IK_LLama, PPL is 0.1% above Q6_K on a pure quant of Sheared Llama 2.7b.
|
|
Thanks IK. I'll play with the Qwen models in the next days. |