2 Commits

Author	SHA1	Message	Date
g2mt	06bed7e01b	Port universal assisted decoding to llama-server (#699) ... * port universal assisted decoding to server * fix calls * fix LOG_INFO * fix llama_detokenize call * use emplace_back	2025-08-18 09:22:23 +03:00
g2mt	b6bc5eedad	Port speculative decoding from upstream to llama-server (#645) ... * server : integrate speculative decoding * server: Fix field names * server: fix include, whitespace * fix compile errors in speculative.cpp * add llama_sampling_sample_and_accept_n to sampling * finish porting speculative decoding in server * port functions from common/speculative, common/sampling * remove arg * fix function names * init params_dft to none * correct value for n_ctx * prefix kv cache tensors with model name to avoid conflict * fix call arguments * fix spec decoding args * correct slot.id * use n_max * port the rest of sampling funcs * fix func arguments * slot.id starts at 1? * Revert "prefix kv cache tensors with model name to avoid conflict" This reverts commit fbd5dfd866. * disable draft logging * disable logging in speculative.cpp in mainline, these would be LOG_DEBUG, but since ik_llama doesnt support it, logging is disabled entirely * add more draft model parameters * fix * pass flash_attn * add speculative params for parity * set speculative params in launch_slot_with_task instead	2025-08-16 07:26:44 +03:00