Kawrakow 4d2fbde0cb MMQ for Q6_0 (#115) ...

* MMQ for Q6_0

* Add Q6_0 MMQ to template generator

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2024-11-21 07:12:11 +01:00

..

template-instances

MMQ for Q6_0 (#115)

2024-11-21 07:12:11 +01:00

vendors

Adding bf16 support to CUDA (#40)

2024-09-14 20:02:32 +03:00

acc.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

acc.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

arange.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

arange.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

argsort.cu

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

argsort.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

binbcast.cu

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

binbcast.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

clamp.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

clamp.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

common.cuh

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

concat.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

concat.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

conv-transpose-1d.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

conv-transpose-1d.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

convert.cu

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

convert.cuh

Adding bf16 support to CUDA (#40)

2024-09-14 20:02:32 +03:00

cpy.cu

Adding Q6_0 (#77)

2024-10-02 15:22:13 +03:00

cpy.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

dequantize.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

diagmask.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

diagmask.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

dmmv.cu

MMQ for Q6_0 (#115)

2024-11-21 07:12:11 +01:00

dmmv.cuh

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

fattn-common.cuh

Enable q6_0 for flash attention (#101)

2024-10-22 11:34:49 +02:00

fattn-tile-f16.cu

Faster Gemma2 (#27)

2024-08-27 17:40:59 +03:00

fattn-tile-f16.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

fattn-tile-f32.cu

Faster Gemma2 (#27)

2024-08-27 17:40:59 +03:00

fattn-tile-f32.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

fattn-vec-f16.cuh

Enable IQ4_NL for KV-cache in token generation using Flash Attention (#99)

2024-10-21 12:16:54 +02:00

fattn-vec-f32.cuh

Faster Gemma2 (#27)

2024-08-27 17:40:59 +03:00

fattn-wmma-f16.cuh

Faster Gemma2 (#27)

2024-08-27 17:40:59 +03:00

fattn.cu

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

fattn.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

getrows.cu

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

getrows.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

im2col.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

im2col.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

iqk_mmvq.cu

Bitnet CUDA improvements (#109)

2024-10-26 16:26:04 +02:00

iqk_mmvq.cuh

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

mma.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

mmq.cu

MMQ for Q6_0 (#115)

2024-11-21 07:12:11 +01:00

mmq.cuh

MMQ for Q6_0 (#115)

2024-11-21 07:12:11 +01:00

mmvq.cu

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00

mmvq.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

norm.cu

Adding fused rms_norm (#42)

2024-09-08 10:19:21 +03:00

norm.cuh

Adding fused rms_norm (#42)

2024-09-08 10:19:21 +03:00

pad.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

pad.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

pool2d.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

pool2d.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

quantize.cu

Fix quantized k-cache without FA (#105)

2024-10-24 12:20:30 +02:00

quantize.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

rope.cu

CUDA non-contiguous RoPE (#66)

2024-09-28 17:41:21 +03:00

rope.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

scale.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

scale.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

softcap.cu

Fused soft cap and SIMD-ified GeLU (#9)

2024-08-20 17:15:47 +03:00

softcap.cuh

Fused soft cap and SIMD-ified GeLU (#9)

2024-08-20 17:15:47 +03:00

softmax.cu

Faster Gemma2 (#27)

2024-08-27 17:40:59 +03:00

softmax.cuh

Faster Gemma2 (#27)

2024-08-27 17:40:59 +03:00

sumrows.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

sumrows.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

tsembd.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

tsembd.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

unary.cu

Faster MoE inference (#112)

2024-10-31 12:05:27 +01:00

unary.cuh

Faster MoE inference (#112)

2024-10-31 12:05:27 +01:00

upscale.cu

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

upscale.cuh

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

vecdotq.cuh

Bitnet changes (#106)

2024-10-25 13:08:43 +02:00