mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-01-26 17:20:01 +00:00
770 B
770 B
🔀 #115 - MMQ for Q6_0
| Author | ikawrakow |
|---|---|
| State | ❌ Closed |
| Created | 2024-11-20 |
| Updated | 2024-11-21 |
Description
Add MMQ kernel for Q6_0.
@Nexesenex
💬 Conversation
👤 Nexesenex commented the 2024-11-20 at 19:42:01:
Tested successfully on IK_LLame, PPL is 0.1% above Q6_K on a pure quant of Sheared Llama 2.7b. Thanks IK. I'll play with the Qwen models in the next days.
👤 Nexesenex commented the 2024-11-20 at 19:42:56:
Tested successfully on IK_LLama, PPL is 0.1% above Q6_K on a pure quant of Sheared Llama 2.7b. Thanks IK. I'll play with the Qwen models in the next days.