Kawrakow fd20638bbc Allow bf16 kv-cache (#69) ...

On the CPU I get the exact same PPL with and without FA
using bf16 for kv-cache. But on CUDA the bf16 kv-cache
result is about the same as the fp16 kv-cache CPU result,
so I'm missing some conversion somewhere.

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2024-09-29 09:03:52 +03:00

..

ggml-cann

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-cuda

Allow bf16 kv-cache (#69)

2024-09-29 09:03:52 +03:00

ggml-sycl

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

iqk

Adding ability to have meta data per tensor row (#61)

2024-09-27 08:16:06 +03:00

kompute @ 4565194ed7

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

kompute-shaders

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llamafile

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

vulkan-shaders

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

CMakeLists.txt

Fix compiler warnings (#58)

2024-09-17 14:31:29 +03:00

ggml-aarch64.c

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-aarch64.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-alloc.c

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-backend-impl.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-backend.c

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-blas.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-cann.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-common.h

Adding ability to have meta data per tensor row (#61)

2024-09-27 08:16:06 +03:00

ggml-cuda.cu

Allow bf16 kv-cache (#69)

2024-09-29 09:03:52 +03:00

ggml-impl.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-kompute.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-metal.m

Adding SWIGLU unary op (#65)

2024-09-28 13:37:25 +03:00

ggml-metal.metal

Adding SWIGLU unary op (#65)

2024-09-28 13:37:25 +03:00

ggml-quants.c

Adding ability to have meta data per tensor row (#61)

2024-09-27 08:16:06 +03:00

ggml-quants.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-rpc.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-sycl.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-vulkan.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml.c

Adding SWIGLU unary op (#65)

2024-09-28 13:37:25 +03:00