Kawrakow 7abcc6cc0b CUDA non-contiguous RoPE (#66) ...

In this way we can avoid the Q, K, V copies being made
after multiplication with the QKV tensor in, e.g., Phi-3.5-mini.
This results in a 6-7% speedup of PP-512(Phi-3.5-mini)
on CUDA (RTX-4080)

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2024-09-28 17:41:21 +03:00

..

cmake

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

include

Adding SWIGLU unary op (#65)

2024-09-28 13:37:25 +03:00

src

CUDA non-contiguous RoPE (#66)

2024-09-28 17:41:21 +03:00

.gitignore

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00