ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-04-29 02:41:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
810285581cefec72ac35b0aa8aefda0beab84555
ik_llama.cpp/ggml/include/ggml.h
Iwan Kawrakow 1b41d792ec iq2_tn: TriLM specific 2.0625 bpw quantization 
Quantize/dequantize/scale dot product.

I get 46 t/s for the TriLM-3.9B with any SIMD!
Finally a compiler doing a decent job auto-vectorizing the
scalar implementation.
2024-08-05 14:22:05 +03:00

90 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink