Kawrakow fdfbd98022 Faster IQ1_BN Metal implementation (#107) ...

* iq1_bn: faster Metal dot product

82 t/s -> 87.9 t/s

* iq1_bn(Metal): 87.9 -> 89.0 t/s for TG-128

* iq1_bn(Metal): 89.0 -> 94.7 t/s for TG-128

So, total improvement is ~15%. Not bad.

* iq1_bn(Metal): 686 -> 702 t/s for PP-512

* iq2_bn(Metal): 710 -> 714 t/s for PP-512

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2024-10-26 10:59:59 +02:00

..

ggml-cann

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-cuda

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

ggml-sycl

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

iqk

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

kompute @ 4565194ed7

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

kompute-shaders

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llamafile

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

vulkan-shaders

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

CMakeLists.txt

Enable q6_0 for flash attention (#101)

2024-10-22 11:34:49 +02:00

ggml-aarch64.c

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-aarch64.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-alloc.c

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-backend-impl.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-backend.c

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

ggml-blas.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-cann.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-common.h

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

ggml-cuda.cu

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

ggml-impl.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-kompute.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-metal.m

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

ggml-metal.metal

Faster IQ1_BN Metal implementation (#107)

2024-10-26 10:59:59 +02:00

ggml-quants.c

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

ggml-quants.h

IQ2_KS: 2.1875 bpw non-linear quantization (#85)

2024-10-13 13:34:30 +03:00

ggml-rpc.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-sycl.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-vulkan.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml.c

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00