ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-26 09:09:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
ik_llama.cpp/github-data/pull_requests/348 - Fix q4_1 and q5_1 on Arm.md
Thomas eaa2510a28 Add GitHub data: filename sanitization (#640)
2025-07-23 13:31:53 +02:00

821 B
Raw Permalink Blame History

🐛 #348 - Fix q4_1 and q5_1 on Arm

Author ikawrakow
State ❌ Closed
Created 2025-04-25
Updated 2025-04-25

Description

When I changed the vet_dot_type for q8_1_x4 to q8_2_x4 for the quants using q8_1_x4 I forgot to also make the change for the ARM_NEON implementation. As a result q4_1 and q5_1 are currently broken. But because q4_0/q5_0 will use q4_1/q5_1 for a few ffn_down layers, q4_0 and q5_0 are broken as well.

Looking at the implementation, changing to use q8_2_x4 would be too a major change. Hence, just go back to using q8_1_x4 on Arm. If this results in some models not working correctly, then simply don't use legacy quants for those models.