ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-27 01:29:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
94aa54df763aecde4cc5f528a26f72e9c5f2d4ff
ik_llama.cpp/github-data/pull_requests
History
Thomas 94aa54df76 Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
..
1-Offload Bitnet token embeddings to the GPU.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
2-Offload Bitnet token embeddings to the GPU - the right way.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
3-Merge mainline llama.cpp.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
4-Simdify and multi-thread tanh.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
5-Fusing a mat mul op followed by a scale op on the CPU.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
6-IQ4_K_ SOTA 4-bit quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
7-Adding IQ2_K, IQ3_K and IQ5_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
9-Fused soft cap and SIMD-ified GeLU .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
10-iq4_k_ speedup quantization by a factor of ~2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
11-Faster iq3_k and iq5_k quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
12-q2_K_ allow it to detect ternary nets and quantize accordingly.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
13-Adding IQ2_TN for use with ternary models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
14-Adding IQ6_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
16-Fix Makefile.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
17-Merge mainline - Aug 12 2024.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
19-Skip barriers of noops.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
20-iq2_k_ slightly better bpw - accuracy compromise.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
21-quantize_stats_ print rmse and max error as fraction of _x_.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
22-AVX2 quantization for Q8_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
23-iq4_k tweak.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
24-softcap_ minor improvement.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
27-Faster Gemma2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
28-Binary KQ mask.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
31-Fix build when iqk_mul_mat is disabled.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
32-Zen4 Flash Attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
33-Do not process prompts containing binary data for escapes.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
35-Fix Zen4 Flash Attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
36-Zen4 Flash Attnetion 2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
37-Performance improvements for legacy quants on ARM_NEON.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
38-Zen4 Flash Attention - bf16 support.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
39-Add support for bf16 to iqk_mul_mat.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
40-Adding bf16 support to CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
41-iqk_mul_mat(ARM_NEON)_ adding bf16 support.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
42-Adding fused rms_norm.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
43-iq2_tn_ slightly faster PP on Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
44-Adding IQ1_TN - 1.6875 bpw for TriLM ternary models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
45-Add CUDA support for IQ1_TN.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
46-IQ1_TN Metal implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
47-iq2_tn_ slightly better performance on AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
48-AVX2 Flash Attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
49-ARM_NEON Flash Attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
50-AVX2 Flash Attention 2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
51-Quantized Flash Attention for all supported CPU platforms.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
52-Fix bug and D _ 128 case for Q8_0 k-cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
53-Quantization mixes tweaks.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
54-Improve Q4_0 and Q8_0 performance on AVX2_Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
55-Improve Q5_0 performance on AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
56-BF16 support on Metal.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
57-AVX2_Zen4 horizontal sums .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
58-Fix compiler warnings.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
61-Adding ability to have meta data per tensor row.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
62-Use fp32 for K_Q in Metal FA implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
64-Better sub-3-bit quantization mixes with a qkv tensor.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
65-Adding SWIGLU unary op.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
66-CUDA non-contiguous RoPE.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
68-It is time to fix replace_all.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
69-Allow bf16 kv-cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
70-Fused unary(x)_y.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
71-iqk_mul_mat_ better srategy when nrc_y not divisible by ny.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
72-iqk_mul_mat_ better iq4_nl implementation on Zen4_AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
73-CUDA_ faster float -_ iq4_nl conversion.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
74-IQ4_NL kv-cache on the CPU (Zen4_AVX2_ARM_NEON).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
75-Fix Q5_0 flash attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
76-iq4_nl_ faster quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
77-Adding Q6_0.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
78-q6_0_ Slightly faster Zen4_AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
79-Do not quantize activations if not necessary.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
80-Move to c++17 projectwide.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
81-Cleanup scale fudge factors.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
83-New SOTA quantization_ 4.25 bpw IQ4_KS.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
84-Better model info.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
85-IQ2_KS_ 2.1875 bpw non-linear quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
86-Fix and optimize iq2k Metal implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
87-iq3_k_ fix and optimize Metal dot product.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
89-Adding IQ4_KSS_ 4.0 bpw quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
90-iq4_ks_ faster dot product on Metal.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
91-CLI - Specify GGML_TYPE to quantize for the main tensors..md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
93-Attempt to blindly fix Windows build failure.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
94-Adding @agray3's graph caching approach.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
96-Quant strategies_ attn_q Q4 & attn_v Q6 for Llama 3.1 Q5_K_S.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
97-Bitnet_ make the scale tensors optional.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
98-Avoid rebuild of GGML graph for each token.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
99-Enable IQ4_NL for KV-cache in token generation using Flash Attention .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
101-Enable q6_0 in flash attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
102-Add support for Granite and GraniteMoE models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
105-Fix quantized k-cache without FA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
106-Bitnet changes.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
107-Faster IQ1_BN Metal implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
108-Another Bitnet performance improvement on Metal.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
109-Bitnet CUDA improvements.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
110-Bitnet_ use the fused mul-silu in the FFN network.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
111-Use fused mul - unary op also for MoE models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
112-Faster MoE inference.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
113-Trellis quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
114-MMQ Kernel for Q6_0 (pretty please!).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
115-MMQ for Q6_0.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
116-Use Q6_0 instead of Q5_1 for tensors incompatible with IQ5_K_Q5_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
117-Some minor quant strategies tweaks.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
118-IQ4_NL_X4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
119-Q4_0_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
120-Q8_0_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
121-Q5_0_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
122-Q6_0_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
123-IQ4_XS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
124-iq2_bn_r4_ fastest Bitnet CPU implementation on the planet.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
125-R4 improvements on ARM_NEON.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
126-Rename iq4_nl_x4 to iq4_nl_r4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
127-Q4_0_R4 on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
128-Faster IQ4_XS_R4 on Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
129-Q4_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
130-Q6_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
131-Slightly faster Q4_K_R4 and IQ4_XS_R4 on Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
132-Q5_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
134-Q3_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
135-Better ARM_NEON implementation for R4 quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
136-Q2_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
137-Fix AVX2 implementation of iq4_nl_r4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
138-IQ4_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
139-Faster R4 quants on Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
141-Q8_K_R8_ Fastest quantized matrix multiplications.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
142-BF16_R16 - 16 interleaved bf16 rows .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
143-Slightly faster IQ4_XS_R4 on AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
144-Slightly faster IQ4_K_R4 on AVX2_Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
145-IQ3_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
146-IQ2_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
147-Be able to repack tensors at run time.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
148-Slightly better matrix x vector on Zen4_AVX2 for iq2_k_r4, iq3_k_r4, iq4_k_r4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
149-IQ5_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
150-IQ4_KS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
151-fix typo.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
152-IQ3_XXS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
153-IQ3_XXS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
154-IQ2_XXS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
155-IQ2_XS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
156-IQ2_S_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
157-R4 i-quants improvements.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
158-Faster R4 legacy quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
161-MSVC fixes.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
162-IQ3_S_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
163-q4_0_r4_ Use AVX2 version for matrix x vector.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
168-Falcon3 changes.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
169-Be able to re-quantize MS BitNet I2_S models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
170-MoE fix for R4 quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
171-Fix lower FA performance for even batch sizes.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
172-CPU Flash Attention improvements.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
173-More Flash Attention improvements.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
174-On Zen4 repack fp16 models to bf16_r16.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
175-Better BF16 support on AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
176-Deepseek V3 support added.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
177-Update chat templates.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
178-Interleave 8 rows (Q8_0, IQ4_XS).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
179-Minor performance improvements.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
180-Deepseek MLA Optimizations.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
181-Various.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
182-Faster Q4_K_R4 and Q5_K_R4 on AVX2_Zen4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
184-Deepseek-Lite.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
185-IQ1_S_R4_ better 1.5 bpw quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
186-iq1_s_r4_ slightly faster NEON gemm_gemv.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
187-IQ1_M_R4_ better 1.75 bpw quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
188-Add optional MLA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
189-Rename q4_0_r4, q8_0_r4 and iq4_xs_r4 to _r8.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
190-cuda_ non-contiguous rms norm.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
191-Add additional checks for iq1_s_r4 quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
192-Revert #79.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
193-RPC sync.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
194-Use Q8_K_128 for IQ1_S_R4 and IQ1_M_R4 matrix multiplications.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
195- Deepseek MLA Optimizations V2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
197-FA_ Add option to build all FA kernels.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
198- Load all MoE experts during warmup and make warmup 1 token.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
200-DeepSeek FA support (CPU only).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
202-Fix imatrix overprotectiveness.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
204-Fix iqk_mul_mat on AVX512 systems that are missing BF16 support.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
205-Faster MLA prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
206-MLA_ allow Q8_0 K-cache for MLA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
207-Faster CPU TG for GQA models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
208-Q8_KV_ 8-bit quantization type targeting the KV cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
210-Repack also experts.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
212-Optimized GEMM_GEMV for IQ1_S.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
213-Fix NEON gemm_gemv for legacy quants when row size is not divisible by 128.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
215-Trying to fix confusion betweem HAVE_FANCY_SIMD and AVX512.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
216-Hopefully this really fixes the confusion between AVX512 and FANCY_SIMD.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
218-Better strategy for attention matrix multiplications when generating tokens .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
219-Fuse MoE up and gate matrix multiplications.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
220-Fix #217.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
225- Examples _ Add new sweep-bench benchmark.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
226-Fix compilation error with IQK_FA_ALL_QUANTS enabled.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
229-Fused MoE ffn_up and ffn_gate.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
231-Fix #230.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
232-Give the user the option to override where model weights are stored.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
233-Slightly faster CUDA MLA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
234-Faster MLA on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
235-Option to use MLA without a transposed cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
236-Feat_lock free server.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
237-Reduce size of compute buffers.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
238-A better way to measure the cost of ggml_barrier.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
239-SER - Smart Expert Reduction.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
240-Flash MLA (CPU only).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
241-DeepSeek CUDA Flash Attention .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
243-Better FlashMLA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
244-Custom quantization rules with regular expressions.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
246-Faster FlashMLA prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
247-FlashMLA on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
248-Faster MoE token generation on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
250-DeepSeek imatrix stuff.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
251-Try using fp32 for FlashMLA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
252-MLA-2_ Allow usage of q8_0 for KV cache on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
253-FlashMLA-2 (CPU)_ faster and smaller compute buffer size.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
259-Prepare wk_b tensors of DeepSeek models on the fly.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
260-FlashMLA-2_ reduce compute buffer size (CUDA and CPU).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
261-Compile time option to use bf16 for quants without MMQ kernels.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
262-Fix #261.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
264-Make Q8_0 KV cache work with FlasMLA-2 on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
265-Allow q8_0 cache on the CPU for FlashMLA-2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
268-Prevent FlashMLA-1 from running on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
269-Fix ggml_compute_forward_dup_q.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
270-Honor mmap setting when using tensor overrides.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
272-Convert models to row-interleaved quants using the quantize tool.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
273-FlashMLA-3_ the best of both worlds (CPU only).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
274-Specify tensor name regex for tensors to be repacked.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
275-Fix bug_ missing parentheses in logical expression.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
276-Add Gemma3 support (text only).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
277-Attempt to improve FlashMLA on the CPU.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
278-Test transparent huge pages on Linux.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
279-Fighting with cmake.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
280-Native build ooption for CUDA when GGML_NATIVE is set.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
282-Improve DeepSeek batched processing speed.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
283-CUDA_ better MoE implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
284-llama-bench_ enable having different number of threads for tg and pp.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
287-Is this better for DeepSeek-R1_.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
289-Update sweep bench (depracating .jsonl support).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
290-mmap backed KV cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
291-Disable Zen4 optimizations for Q8_0_Q8_0_R8.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
292-Use bf16 instead of fp16 block scales for q8_1.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
294-Make sure tensor row size is multiple of block size also when quantizing with --pure.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
295-Quantization improvements.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
298-Update gguf-py constants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
299-Additional guards for interleaved quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
301-Fix #300.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
302-Quantization improvements (2).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
303-Fix ARM_NEON build failure due to q8_2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
307-Metal_ much faster MoE prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
309-Fix GCC compilation errors on ARM.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
310-Metal_ FA and FlashMLA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
311-Add -flax-vector-conversions for GCC on ARM.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
312-Improved IQ2_XS quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
313-We need to synchronize before using device to host async memcpy.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
315-Try not repacking q8_0 for FA computations.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
317-Add copyright notices.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
318-Use links for ggml_llama.cpp authors.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
320-Guard against attempts to use MLA for non-MLA models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
321-LlaMA-4 support (text only).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
324-Correct L4 rms_norm.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
325-Fix KLD precision.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
326-WIP Compute per layer LIM Scores during imatrix.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
327-Improved IQ1_M quantization.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
328-imatrix_ collect layer influence statistics.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
329-Add ability to hide imatrix details in llama-quantize.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
330-Allow q8_0 KV cache for head size 256.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
331-Better gemm_gemv on AVX2 fr q4_0_r8.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
332-Better TG performance for GQA models (CPU).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
333-Support GLM-4-0414 models based on piDack's mainline PR.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
336-Fix termux_android build.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
337-Add support for bitnet2b_2501 model.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
338-BitNet adjustments.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
341-Add support for Cohere2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
342-Fix LLaMA-4 attention.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
343-cuda_ use switch in constexpr funcs.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
344-Add GLM-4-0414 Model Support.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
346-Fix FA on ARM CPUs.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
347-Add ability to manually set arch flags.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
348-Fix q4_1 and q5_1 on Arm.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
349-Fix division by zero bug.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
351-CPU FA improvements .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
352-Update README.md.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
355-Apply Qwen3 PR from llama.cpp.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
356-Add missing enum values for qwen3 and qwen3moe.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
360-Fix IQK_FA_ALL_QUANTS on AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
364-Fix FA bug on AVX2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
366-Add support for new Bitnet model architecture name.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
368-Trying to fix iq1_s_r4_iq1_m_r4 quantization failure.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
369-cmake_ force MSVC compiler charset to utf-8.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
370-CUDA_ faster FA TG for GQA models .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
371-Another attempt to fix #367.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
374-CUDA_ MMQ for IQ4_KS.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
375-Add batch warmup to sweep-bench.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
377-Support for Llama-3-Nemotron models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
382-Fix DeepSeek FA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
386-FlashMLA-3 for DeepSeek models on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
390-Fix build for Xeon Gold 6226R.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
391-Fix DeepSeek q8_0 cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
392-fix some MSVC build problem..md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
394-Handle incompatible DeepSeek GGUFs.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
400-Fix CUDA DeepSeek FlashMLA-3 with quantized KV cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
402-Fix missing rope_freqs with convert_hf_to_gguf.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
404-TG improvements for MoE models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
405-GPU offload policy.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
406-Fix race in the CUDA DeepSeek FA kernel.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
408-Faster DeepSeek FA on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
409-Enable faster prompt processing with mainline llama.cpp GGUFs.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
410-Better CPU FA performance for DeepSeek-Lite.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
411-Fix imatrix calculation for MLA models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
413-Fix new CUDA FA on Touring.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
415-Fix SER (CPU).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
416-Fix SER (CUDA).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
417-CUDA_ quantized GEMM for for IQ4_K, IQ5_K, IQ6_K .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
418-CUDA_ quantized GEMM for for IQ2_KS, IQ2_K, IQ3_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
421-Fix standard attention on the CPU.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
422-Adding IQ5_KS - 5.25 bpw quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
424-Adding forgotten template instance for iq5_ks.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
426-IQ5_KS_R4_ row-interleaved IQ5_KS.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
427-Fix AVX2 implementation of IQ4_K, IQ4_KS, IQ5_K, IQ6_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
428-Zen4_ Faster PP for IQ2_KS, IQ4_KS, IQ5_KS.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
429-Option to enable or disable the CPU FA kernels.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
430-Disable multi-add for now.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
431-Forgotten MMQ ref and typo.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
435-Refactor iqk_mul_mat.cpp.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
438-Another attempt to fix the illegal memory access bug.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
439-Bug fixes from mainline.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
441-Trellis quants with CPU inference.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
442-CUDA call tracer.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
443-Streamline a bit the quant strategies.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
444-gguf-split _ update.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
445-Fix typo in non-AVX2 code branch.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
446-Fix bug in MMVQ kernel.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
448-Fix MSVC compilation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
449-Legacy quants conversion schemes in convert_hf_to_gguf.py.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
453-Faster IQ3_KT and IQ4_KT.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
454-Add support for FP8 GGUF creation and re-quantization (WIP).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
457-Remove GGML_IQK_MUL_MAT option.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
458-Add missing gguf-py constants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
460-aarch64 kernels for KT quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
461-CUDA implementation for IQ2_K_R4, IQ3_K_R4, IQ4_K_R4, IQ5_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
462-CUDA GEMM and GEMV for IQ4_KS_R4 and IQ5_KS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
465-Set cache_prompt default to true.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
468-Minor (~2%) iq2_ks TG performance improvement on CUDA.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
469-Replace MLA-specific KV cache with the standard KV cache.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
470-Send [DONE] for OAI compatibility.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
471-NEON implementation for trellis quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
473-Replace MLA-specific KV cache with the standard KV cache V2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
475-Metal implementatio for the trellis quants..md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
478-forgotten refs and typo.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
480-Rpc improvement.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
481-Webui improvement.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
482-Trellis quants_ faster CPU prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
483- convert_hf_to_gguf.py _ conversion from hf weights to Q6_0.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
484-BF16 Trellis implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
486-Adding the XTC sampler.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
487-Make sure MMVQ is supported before using it.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
488-Faster CPU prompt processing for Trellis quants and MoE models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
489-Adding top-n-sigma sampler.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
492-CUDA implementation for IQ1_S_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
493-MMQ implementation for IQ4_KS_R4 and IQ5_KS_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
494-IQ1_M_R4 CUDA implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
495-Check if ffn_up and ffn_gate are of the same type before using fmoe.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
496-Quick hack_ add the MLA flag to llama_hparams.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
497-Make prompt cache saving and restoring MLA aware.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
501-Fix #499.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
502-Add an endpoint that lists all the saved prompt caches to server.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
504-Add DRY and fix the server to use other new samplers..md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
505-New IQ4_KT trellis implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
506-Fix non rpc build error.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
508-Fix Compile error (C2668).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
509-Docs update.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
510-Update News section of readme.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
511-New IQ2_KT.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
512-Add top n sigma sampler in webui and other webui fix.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
513-add dry sampler.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
515-IQ2_XXS_ much faster CPU prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
516-Much faster iq3_xxs GEMM via repacking to q8_0_r8 (AVX2).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
517-IQ1_S_ much faster CPU prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
518-IQ3_S_ much faster CPU prompt processing.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
520-Better strategy for GPU offload.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
524-Perhaps a slightly better GEMV version for IQ2_XXS, IQ3_XXS, IQ3_S.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
525-Faster CPU prompt processing for Q4_K and Q5_K.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
528-Fix bug introduced in #524_#525.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
529-New IQ2_KT, IQ3_KT and IQ4_KT, V2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
531-Much faster CPU prompt processing (part 1).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
533-Much faster CPU prompt processing (part 2).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
534-Much faster CPU prompt processing (part 3).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
535-Minor readme update.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
536-Fix KT Neon _ ARM typo.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
537-Update CMakeLists.txt to fix NDEBUG handling.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
540-Fix missed block_q8_x2 bf16 -_ i16 change.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
541-Perhaps slightly faster trellis quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
542-Fix NEON build.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
544-New integer trellis on ARM_NEON .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
546-Faster ARM_NEON GEMM implementation for legacy quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
547-build_ add script to simplify build&test workflow for Android.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
549-Much faster prompt processing for IQK quants (ARM_NEON).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
550-Much faster prompt processing for I-quants (ARM_NEON).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
552-Much faster prompt processing for k-quants (ARM_NEON).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
553-Much faster prompt processing for IQ1_S and IQ1_M on ARM_NEON.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
554-Update README.md to add quickstart section.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
555-Add Falcon-Edge support.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
557-CUDA_ MMQ for iqX_r4 quants .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
558-Add mikupad to ik_llama as an alternative WebUI.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
559-Use cuBLAS for large batches and quants with block size 16.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
560-Remove what appears to be unnecessary asserts in ggml_cuda_cpy.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
563-Merge vulkan code from mainline up to commit of 6_28_2025.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
565-add hunyuan moe support for 561.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
566-Adding IQ3_KS quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
567-Minor CUDA PP speed improvement.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
569-Conditionally disable fused ops when building with Vulkan enabled.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
570-Remove duplicate_misplaced cmake find_package for Vulkan.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
571-Fix CMakeLists.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
573-Support for dots.llm1 models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
574-Change KQ mask padding to 64.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
577-Vulkan_ fused rms norm.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
578-Do not crash when there is no DRY sampler.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
579-Fix debug build failure with RPC off.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
580-Vulkan_ add GGML_OP_FUSED_MUL_UNARY.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
581-Vulkan_ Disable multi-add for now.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
582-Vulkan_ adding GGML_OP_MULTI_ADD implementation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
583-Adding forgotten file.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
584-Vulkan_ flash attention for DeepSeek models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
585-Special handling of Seed Coder FIM tokens.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
587-Fix crash when there is no DRY sampler.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
588-Fix server crash when there is no DRY sampler.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
589-CUDA_ small PP performance improvement for MoE models.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
592-Another minor readme update.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
593-Faster prompt processing for IQ2_KS, IQ2_K, IQ2_K_R4.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
595-CUDA_ Faster prompt processing for several quantization types .md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
598-Vulkan_ iquants and flash attention split_k_reduce improvement.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
602-Adding IQ2_KL.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
603-Check if MMQ should be used before using it.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
604-Fix attn_v conditionality when quantizing..md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
606-Add iq3_ks to constants.py.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
607-vulkan_ support softmax_FA batch and broadcast.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
608-Vulkan_ a fresh start.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
609-Added kimi-k2 support (ported from llama.cpp).md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
610-q8_k_r8_ experimental AVX512 version.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
611-Bump GGML_MAX_CONTEXTS to allow loading more shards.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
612-kimi-k2 convert script and chat template.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
616-Adding IQ1_KT - 1.75 bpw SOTA quants.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
617-Fixup kimi-k2 convert indentation.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
618-Webui_ New Features for Conversations, Settings, and Chat Messages.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
620-Bump Windows max open files from 512 to 2048.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
622-Add GGML_MAX_CONTEXTS definition in CMakeLists.txt.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
624-Quantization tweaks.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
628-[Draft] Function calling support for Kimi-K2.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00
630-GEMM for IQ1_M.md
Add GitHub data (#637)
2025-07-22 18:18:40 +02:00