Kawrakow b5f2f00106 Much faster prompt processing for IQ1_S and IQ1_M on ARM_NEON (#553) ...

* iq1_s

66.3 t/s -> 168.8 t/s.

* iq1_m

19 t/s -> 163 t/s.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2025-06-24 14:21:37 +02:00

..

fa

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_common.h

IQ2_XXS: much faster CPU prompt processing (#515)

2025-06-11 11:12:30 +03:00

iqk_config.h

Fix termux/android build (#336)

2025-04-21 09:13:46 +02:00

iqk_flash_attn.cpp

Option to enable disable the IQK CPU FA kernels (#429)

2025-05-17 11:21:58 +03:00

iqk_flash_impl.h

CPU FA improvements (#351)

2025-04-29 07:19:43 +02:00

iqk_gemm_1bit.cpp

Much faster prompt processing for IQ1_S and IQ1_M on ARM_NEON (#553)

2025-06-24 14:21:37 +02:00

iqk_gemm_1bit.h

Faster iq1_s GEMM via repacking to Q8_0_R8 (#517)

2025-06-11 15:01:34 +03:00

iqk_gemm_floats.cpp

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_gemm_floats.h

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_gemm_iqk_quants.cpp

Much faster prompt processing for IQK quants (ARM_NEON) (#549)

2025-06-23 11:55:50 +02:00

iqk_gemm_iqk_quants.h

Much faster CPU prompt processing (part 2) (#533)

2025-06-18 07:29:33 +03:00

iqk_gemm_iquants.cpp

Much faster prompt processing for I-quants (ARM_NEON) (#550)

2025-06-23 15:50:24 +02:00

iqk_gemm_iquants.h

IQ2_XXS: much faster CPU prompt processing (#515)

2025-06-11 11:12:30 +03:00

iqk_gemm_kquants.cpp

Much faster prompt processing for k-quants (ARM_NEON) (#552)

2025-06-24 13:05:01 +02:00

iqk_gemm_kquants.h

Faster CPU prompt processing for Q4_K and Q5_K (#525)

2025-06-13 07:58:15 +03:00

iqk_gemm_ktquants.cpp

Perhaps slightly faster trellis quants (#541)

2025-06-21 16:32:16 +02:00

iqk_gemm_ktquants.h

Trellis quants: faster CPU prompt processing (#482)

2025-06-01 15:24:05 +03:00

iqk_gemm_legacy_quants.cpp

Faster ARM_NEON GEMM implementation for legacy quants (#546)

2025-06-21 16:35:08 +02:00

iqk_gemm_legacy_quants.h

Much faster CPU prompt processing (part 3) (#534)

2025-06-18 15:30:56 +03:00

iqk_mul_mat.cpp

Much faster prompt processing for IQ1_S and IQ1_M on ARM_NEON (#553)

2025-06-24 14:21:37 +02:00

iqk_mul_mat.h

Much faster CPU prompt processing (part 1) (#531)

2025-06-17 07:12:48 +03:00

iqk_quantize.cpp

New IQ2_KT, IQ3_KT and IQ4_KT, V2 (#529)

2025-06-18 16:20:54 +03:00

iqk_quantize.h

Trellis quants with CPU inference (#441)

2025-05-23 09:17:52 +03:00

iqk_utils.h

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00