Kawrakow 33646fc409 Fuse MoE up and gate matrix multiplications (#219) ...

* This seems to be a better way

to do the attention matrix multiplications in the TG case.

* Cleanup

* Fuse up and gate gemms in MoE models

Small (~1-2%) but measurable performan ce gain

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2025-02-22 09:41:40 +02:00

..

cmake

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

include

Q8_KV: 8-bit quantization type targeting the KV cache (#208)

2025-02-19 11:47:07 +02:00

src

Fuse MoE up and gate matrix multiplications (#219)

2025-02-22 09:41:40 +02:00

.gitignore

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

CMakeLists.txt

FA: Add option to build all FA kernels (#197)

2025-02-09 18:59:33 +02:00