Kawrakow e57440472e DeepSeek FA support (CPU only) (#200) ...

* Adding support for K head size != V head size

This is relevant for DeepSeek models.
At this point ggml CPU FA works.
Now I need to go and change iqk FA to make it work
with Dk != Dv.

* iqk support for K head size != V head size

To not have compilation time explode, just
Dk = 192, Dv = 128 for now (DeepSeek)

* FA: very slightly faster for nq = 1 (TG)

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2025-02-11 14:46:30 +02:00

..

cmake

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

include

Use Q8_K_128 for IQ1_S_R4 and IQ1_M_R4 matrix multiplications (#194)

2025-02-09 09:14:52 +02:00

src

DeepSeek FA support (CPU only) (#200)

2025-02-11 14:46:30 +02:00

.gitignore

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

FA: Add option to build all FA kernels (#197)

2025-02-09 18:59:33 +02:00