ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-02-06 22:40:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
61304f5c04c3eedc17bdd506b73bd5ad8daf63c5
ik_llama.cpp/include
History
Kawrakow 61304f5c04 Q4_0_R4 (#119) 
* Adding iq4_0_r4 - q4_0 repacked

We get PP-512(LLaMA-3.1-8B) = 278 t/s on a Ryzen-7950X CPU,
so ~5-6% faster than iq4_nl_x4.

* q4_0_r4: NEON

Here we get 115.8 t/s, so also ~5% better than iq4_nl_x4.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2024-12-02 17:01:48 +01:00
..
llama.h
Q4_0_R4 (#119)
2024-12-02 17:01:48 +01:00