ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-26 17:20:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

server : smart slot selection using Longest Common Prefix (#7728)

Browse Source 
* server : Smart selection of available slot using Longest Common Substring

* add usage

* remove trailing whitespaces

* Use Longest Common Prefix (LCP) instead of LCS

* Rename argument
This commit is contained in:
sasha0552
2024-06-08 07:50:31 +00:00
committed by GitHub
parent b28de435a1
commit 66217bbac6
4 changed files with 140 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
common/common.h
View File

@@ -203,6 +203,8 @@ struct gpt_params {
std::string slot_save_path;
float slot_prompt_similarity = 0.5f;
// batched-bench params
bool is_pp_shared = false;