ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-26 17:20:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

add split-mode-graph-scheduling parameter (#1068)

Browse Source 
Use -smgs or --split-mode-graph-scheduling in CLI to bypass the disabling of split mode graph scheduling when tensor overrides is used.

Co-authored-by: Kawrakow <iwankawrakow@gmail.com>
This commit is contained in:
Nexes the Elder
2025-12-17 07:58:19 +01:00
committed by GitHub
parent 5585ac2aa8
commit d1dd45b4b9
7 changed files with 21 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
common/common.h
View File

@@ -279,6 +279,7 @@ struct gpt_params {
bool only_active_exps = true; // if true, offload only active experts (relevant only for hybrid CPU/GPU)
bool merge_qkv = false; // if true, merge separate Q, K, V tensors into a single, contiguous tensor
bool k_cache_hadamard = false; // if true, use Hadamard transform for the K-cache (only makes sense with quantized cache)
bool split_mode_graph_scheduling = false; // if true, force split mode graph scheduling
std::string cache_type_k = "f16"; // KV cache data type for the K
std::string cache_type_v = "f16"; // KV cache data type for the V