ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-02-26 08:04:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
d89c88e8df227eb8a852c3677ab2ec00f3e7792d
ik_llama.cpp/ggml
History
Iwan Kawrakow d89c88e8df iq4_k: NEON implementation 
For LLaMA-3.1-8B we get PP-512 = 60.7 t/s, TG-128 = 25.0 t/s
on the M2-Max. TG is on par with q4_K_S, PP is ~10% slower.
2024-07-28 08:36:20 +02:00
..
cmake
Merge mainline llama.cpp (#3)
2024-07-27 07:55:01 +02:00
include
iq4_k: basics
2024-07-27 17:05:31 +03:00
src
iq4_k: NEON implementation
2024-07-28 08:36:20 +02:00
.gitignore
Merge mainline llama.cpp (#3)
2024-07-27 07:55:01 +02:00
CMakeLists.txt
Merge mainline llama.cpp (#3)
2024-07-27 07:55:01 +02:00