ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-04-26 17:39:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Command line option to enable fused MoE up*unary(gate)

Browse Source
This commit is contained in:
Iwan Kawrakow
2025-02-23 11:36:46 +02:00
parent c229183737
commit 5bf5467c21
4 changed files with 26 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
include/llama.h
View File

@@ -377,6 +377,7 @@ extern "C" {
bool offload_kqv; // whether to offload the KQV ops (including the KV cache) to GPU
bool flash_attn; // whether to use flash attention [EXPERIMENTAL]
bool mla_attn; // whether to use MLA attention [EXPERIMENTAL]
bool fused_moe_up_gate; // whether to use fused MoE up/down op [EXPERIMENTAL]
// Abort callback
// if it returns true, execution of llama_decode() will be aborted