Iwan Kawrakow 147f9606d0 CUDA non-contiguous RoPE ...

In this way we can avoid the Q, K, V copies being made
after multiplication with the QKV tensor in, e.g., Phi-3.5-mini.
This results in a 6-7% speedup of PP-512(Phi-3.5-mini)
on CUDA (RTX-4080)

2024-09-28 14:37:28 +03:00

..

CMakeLists.txt

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-grammar.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama-grammar.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-impl.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama-sampling.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-sampling.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama-vocab.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama-vocab.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

llama.cpp

CUDA non-contiguous RoPE

2024-09-28 14:37:28 +03:00

unicode-data.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unicode-data.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unicode.cpp

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unicode.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00