ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-31 11:39:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
a63a96b5aea031f9dabf1e7c8d5ee28af170e9e7
ik_llama.cpp/include
History
Kawrakow a63a96b5ae Q5_K_R4 (#132) 
* q5_k_r4: WIP

* q5_k_r4: Zen4 and AVX2

We get PP-512(LLaMA-3.1-8B) = 248.3 t/s on Zen4.
Q5_K_S has PP-512 = 190 t/s.

* q5_k_r4: NEON

We get PP-512(LLaMA-3.1-8B) = 96.1 t/s.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2024-12-10 18:13:47 +01:00
..
llama.h
Q5_K_R4 (#132)
2024-12-10 18:13:47 +01:00