Iwan Kawrakow cfee1b68ec WIP: plugging into ggml_compute_forward_flash_attn_ext_f16 ...

OK, if we take into account that the mask is diagonal
and skip further computations once we encounter -INFINITY,
we can speed it up and make it on par with no-FA.
Better than nothing, but still no luck.

2024-08-24 14:31:13 +03:00

..

cmake

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

include

Fused soft cap and SIMD-ified GeLU (#9)

2024-08-20 17:15:47 +03:00

src

WIP: plugging into ggml_compute_forward_flash_attn_ext_f16

2024-08-24 14:31:13 +03:00

.gitignore

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00