Kawrakow 2572d16399 Fix q8_0 repacking issues on AVX2 (#708) ...

Q8_0 needs Q0_0_X4, but Q8_0_R8 needs Q8_2_X4.
So, if we decide to repack a Q8_0 MoE tensor to Q8_0_R8,
iqk_moe_fused_mul_unary fails because the activations were
prepared as Q0_0_X4, but we now need Q8_2_X4.

For now a simple fix: just take the slow path, do not repack.

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2025-08-19 19:49:58 +03:00

..

fa

Enable CUDA graphs for MoE models + GPT-OSS support (#689)

2025-08-15 09:18:07 +03:00

iqk_common.h

IQ2_XXS: much faster CPU prompt processing (#515)

2025-06-11 11:12:30 +03:00

iqk_config.h

Fix termux/android build (#336)

2025-04-21 09:13:46 +02:00

iqk_flash_attn.cpp

Revert "Better CPU prompt processing performance for SWA models (#696)" (#701)

2025-08-17 15:44:02 +03:00

iqk_flash_impl.h

Enable CUDA graphs for MoE models + GPT-OSS support (#689)

2025-08-15 09:18:07 +03:00

iqk_gemm_1bit.cpp

IQ1_M GEMM for ARM_NEON (#631)

2025-07-20 09:49:59 +02:00

iqk_gemm_1bit.h

Faster iq1_s GEMM via repacking to Q8_0_R8 (#517)

2025-06-11 15:01:34 +03:00

iqk_gemm_floats.cpp

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_gemm_floats.h

Refactor iqk_mul_mat.cpp (#435)

2025-05-22 10:05:51 +03:00

iqk_gemm_iqk_quants.cpp

IQ4_KSS improvements (#642)

2025-07-23 20:50:57 +02:00

iqk_gemm_iqk_quants.h

Much faster CPU prompt processing (part 2) (#533)

2025-06-18 07:29:33 +03:00

iqk_gemm_iquants.cpp

Much faster prompt processing for I-quants (ARM_NEON) (#550)

2025-06-23 15:50:24 +02:00

iqk_gemm_iquants.h

IQ2_XXS: much faster CPU prompt processing (#515)

2025-06-11 11:12:30 +03:00

iqk_gemm_kquants.cpp

Much faster prompt processing for k-quants (ARM_NEON) (#552)

2025-06-24 13:05:01 +02:00

iqk_gemm_kquants.h

Faster CPU prompt processing for Q4_K and Q5_K (#525)

2025-06-13 07:58:15 +03:00

iqk_gemm_ktquants.cpp

Adding IQ1_KT - 1.75 bpw SOTA quants (#616)

2025-07-20 10:05:23 +02:00

iqk_gemm_ktquants.h

Trellis quants: faster CPU prompt processing (#482)

2025-06-01 15:24:05 +03:00

iqk_gemm_legacy_quants.cpp

MXFP4 (#682)

2025-08-09 08:40:18 +03:00

iqk_gemm_legacy_quants.h

Much faster CPU prompt processing (part 3) (#534)

2025-06-18 15:30:56 +03:00

iqk_mul_mat.cpp

Fix q8_0 repacking issues on AVX2 (#708)

2025-08-19 19:49:58 +03:00

iqk_mul_mat.h

Revert "Better CPU prompt processing performance for SWA models (#696)" (#701)

2025-08-17 15:44:02 +03:00

iqk_quantize.cpp

MXFP4 (#682)

2025-08-09 08:40:18 +03:00

iqk_quantize.h

MXFP4 (#682)

2025-08-09 08:40:18 +03:00

iqk_utils.h

Enable CUDA graphs for MoE models + GPT-OSS support (#689)

2025-08-15 09:18:07 +03:00