Iwan Kawrakow 9f8cd4049a q8_k16: use integer arithmetic to sum row values ...

The existing implementation that just sums up the f32 quantizations
works fine for the original BitNet models and also for the TriLM
ternary models. But for Falcon3 I see a significant difference between
the CPU and the GPU perplexity. If I use the q8_K16 int8_t quants to sum
up the values in a row, then the CPU-GPU PPL difference becomes much
smaller, and we get a lower PPL than Microsoft BitNet, which claims
to be "losless".

2025-01-10 14:01:01 +02:00

..

cmake

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

include

IQ3_S_R4 (#162)

2024-12-23 14:34:23 +01:00

src

q8_k16: use integer arithmetic to sum row values

2025-01-10 14:01:01 +02:00

.gitignore

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

Move to c++17 projectwide (#80)

2024-10-04 14:43:26 +03:00