Iwan Kawrakow 050bdfa101 iq6_k: CUDA dot product ...

90.2 t/s for LLaMA-3.1-8B. Q6_K gives 91.2 t/s, so we are good.

2024-08-09 16:00:31 +02:00

..

ggml-cann

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-cuda

iq6_k: CUDA dot product

2024-08-09 16:00:31 +02:00

ggml-sycl

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

iqk

iq6_k: CUDA dequantize

2024-08-09 16:00:31 +02:00

kompute @ 4565194ed7

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

kompute-shaders

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llamafile

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

vulkan-shaders

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

Factor out iqk CUDA dot products

2024-08-01 09:38:06 +02:00

ggml-aarch64.c

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-aarch64.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-alloc.c

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-backend-impl.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-backend.c

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-blas.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-cann.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-common.h

iq6_k: CUDA dot product

2024-08-09 16:00:31 +02:00

ggml-cuda.cu

iq6_k: CUDA dequantize

2024-08-09 16:00:31 +02:00

ggml-impl.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-kompute.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-metal.m

Adding IQ2_TN for use with ternary models (#13)

2024-08-07 07:56:09 +02:00

ggml-metal.metal

Adding IQ2_TN for use with ternary models (#13)

2024-08-07 07:56:09 +02:00

ggml-quants.c

iq6_k: WIP (quantize/dequantize)

2024-08-09 16:00:31 +02:00

ggml-quants.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-rpc.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-sycl.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-vulkan.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml.c

iq6_k: WIP (nothing works)

2024-08-09 16:00:31 +02:00