Kawrakow f7c9a0f036 CUDA: MMQ for IQ4_KS (#374) ...

* WIP

* WIP: still getting illegal memory access

* CUDA: MMQ for iq4_ks now works

~25% faster than dequantize+cuBLAS, ~10% slower than Q4_0 MMQ.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2025-05-04 12:45:00 +03:00

..

ggml-cann

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-cuda

CUDA: MMQ for IQ4_KS (#374)

2025-05-04 12:45:00 +03:00

ggml-sycl

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

iqk

Another attempt to fix #367 (#371)

2025-05-04 09:02:12 +03:00

kompute @ 4565194ed7

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

kompute-shaders

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llamafile

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

vulkan-shaders

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

CMakeLists.txt

CUDA: faster FA TG for GQA models (#370)

2025-05-04 09:17:44 +03:00

ggml-aarch64.c

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-aarch64.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-alloc.c

Fix ARM_NEON build failure due to q8_2 (#303)

2025-04-01 13:48:20 +02:00

ggml-backend-impl.h

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

ggml-backend.c

FlashMLA-2 (CPU): faster and smaller compute buffer size (#253)

2025-03-13 12:07:43 +02:00

ggml-blas.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-cann.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-common.h

Add copyright notices (#317)

2025-04-07 10:43:26 +02:00

ggml-cuda.cu

Allow q8_0 KV cache for head size 256 (#330)

2025-04-15 17:05:31 +02:00

ggml-impl.h

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-kompute.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-metal.m

Metal: FA and FlashMLA (#310)

2025-04-03 17:54:25 +02:00

ggml-metal.metal

Metal: FA and FlashMLA (#310)

2025-04-03 17:54:25 +02:00

ggml-quants.c

Improved IQ1_M quantization (#327)

2025-04-13 10:37:55 +02:00

ggml-quants.h

IQ1_M_R4: better 1.75 bpw quants (#187)

2025-02-06 14:08:52 +02:00

ggml-rpc.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-sycl.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml-vulkan.cpp

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

ggml.c

CPU FA improvements (#351)

2025-04-29 07:19:43 +02:00