mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-04-29 10:51:51 +00:00
Faster iq4_xs_r4 on Zen4
The trick is to simply prepare the Q8 block sums for blocks of 32 as floats. This brings PP-512 up to 254.6 t/s from 224 t/s.
This commit is contained in:
@@ -407,6 +407,7 @@ extern "C" {
|
||||
GGML_TYPE_IQ2_KS = 145,
|
||||
GGML_TYPE_IQ4_KSS = 146,
|
||||
GGML_TYPE_Q8_K16 = 147,
|
||||
GGML_TYPE_Q8_K32 = 148,
|
||||
|
||||
GGML_TYPE_Q4_0_R4 = 202,
|
||||
GGML_TYPE_Q5_0_R4 = 206,
|
||||
|
||||
Reference in New Issue
Block a user