Kawrakow 1b05210904 IQ4_KSS improvements (#642) ...

* iq4_kss: slightly better quantization

* iq4_kss: CUDA MMQ

* iq4_kss: repack/convert to q8_k_r8 (AVX2)

* iq4_kss: repack/convert to q8_k_r8 (NEON)

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2025-07-23 20:50:57 +02:00

..

fa

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_common.h

IQ2_XXS: much faster CPU prompt processing (#515)

2025-06-11 11:12:30 +03:00

iqk_config.h

Fix termux/android build (#336)

2025-04-21 09:13:46 +02:00

iqk_flash_attn.cpp

Option to enable disable the IQK CPU FA kernels (#429)

2025-05-17 11:21:58 +03:00

iqk_flash_impl.h

CPU FA improvements (#351)

2025-04-29 07:19:43 +02:00

iqk_gemm_1bit.cpp

IQ1_M GEMM for ARM_NEON (#631)

2025-07-20 09:49:59 +02:00

iqk_gemm_1bit.h

Faster iq1_s GEMM via repacking to Q8_0_R8 (#517)

2025-06-11 15:01:34 +03:00

iqk_gemm_floats.cpp

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_gemm_floats.h

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_gemm_iqk_quants.cpp

IQ4_KSS improvements (#642)

2025-07-23 20:50:57 +02:00

iqk_gemm_iqk_quants.h

Much faster CPU prompt processing (part 2) (#533)

2025-06-18 07:29:33 +03:00

iqk_gemm_iquants.cpp

Much faster prompt processing for I-quants (ARM_NEON) (#550)

2025-06-23 15:50:24 +02:00

iqk_gemm_iquants.h

IQ2_XXS: much faster CPU prompt processing (#515)

2025-06-11 11:12:30 +03:00

iqk_gemm_kquants.cpp

Much faster prompt processing for k-quants (ARM_NEON) (#552)

2025-06-24 13:05:01 +02:00

iqk_gemm_kquants.h

Faster CPU prompt processing for Q4_K and Q5_K (#525)

2025-06-13 07:58:15 +03:00

iqk_gemm_ktquants.cpp

Adding IQ1_KT - 1.75 bpw SOTA quants (#616)

2025-07-20 10:05:23 +02:00

iqk_gemm_ktquants.h

Trellis quants: faster CPU prompt processing (#482)

2025-06-01 15:24:05 +03:00

iqk_gemm_legacy_quants.cpp

Faster ARM_NEON GEMM implementation for legacy quants (#546)

2025-06-21 16:35:08 +02:00

iqk_gemm_legacy_quants.h

Much faster CPU prompt processing (part 3) (#534)

2025-06-18 15:30:56 +03:00

iqk_mul_mat.cpp

IQ4_KSS improvements (#642)

2025-07-23 20:50:57 +02:00

iqk_mul_mat.h

Much faster CPU prompt processing (part 1) (#531)

2025-06-17 07:12:48 +03:00

iqk_quantize.cpp

IQ4_KSS improvements (#642)

2025-07-23 20:50:57 +02:00

iqk_quantize.h

Adding IQ1_KT - 1.75 bpw SOTA quants (#616)

2025-07-20 10:05:23 +02:00

iqk_utils.h

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00