Iwan Kawrakow 026adac30d iq4_kss: CUDA works ...

TG-128 performance is very decent with 131 t/s for LLaMA-3.1-8B.
In comparison, we have 123 t/s for q4_0 and 128 t/s for iq4_ks.
I.e., the reduced model size more than offsets the additional
bit fiddling required for iq4_kss.

2024-10-16 14:14:00 +03:00

..

cmake

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

include

iq4_kss: WIP

2024-10-16 14:14:00 +03:00

src

iq4_kss: CUDA works

2024-10-16 14:14:00 +03:00

.gitignore

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

Move to c++17 projectwide (#80)

2024-10-04 14:43:26 +03:00