Iwan Kawrakow 1b41d792ec iq2_tn: TriLM specific 2.0625 bpw quantization ...

Quantize/dequantize/scale dot product.

I get 46 t/s for the TriLM-3.9B with any SIMD!
Finally a compiler doing a decent job auto-vectorizing the
scalar implementation.

2024-08-05 14:22:05 +03:00

..

ggml-alloc.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-backend.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-blas.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-cann.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-cuda.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-kompute.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-metal.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-rpc.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-sycl.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-vulkan.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml.h

iq2_tn: TriLM specific 2.0625 bpw quantization

2024-08-05 14:22:05 +03:00