Kawrakow cd7e7b6bbc Allow bf16 kv-cache (#69) ...

On the CPU I get the exact same PPL with and without FA
using bf16 for kv-cache. But on CUDA the bf16 kv-cache
result is about the same as the fp16 kv-cache CPU result,
so I'm missing some conversion somewhere.

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2024-09-29 09:03:52 +03:00

..

CMakeLists.txt

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-grammar.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama-grammar.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-impl.h

Time to fix replace_all (#68)

2024-09-28 17:59:47 +03:00

llama-sampling.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-sampling.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-vocab.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama-vocab.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama.cpp

Allow bf16 kv-cache (#69)

2024-09-29 09:03:52 +03:00

unicode-data.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unicode-data.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unicode.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unicode.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00