firecoperana bb358223cd server: cache prompt to host memory (#954) ...

* server : host-memory prompt caching

change similarity calculation and prompt save conditions

Remove unneeded token limit

rename variable

Separate prompt save and load logic

change default values

change log

remove truncate prompt logic

* add description

* bug fixes

* remove token limit in init

---------

Co-authored-by: firecoperana <firecoperana>

2025-11-14 18:40:13 +02:00

..

baby-llama

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

batched

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

batched-bench

MoE fix for R4 quants (#170)

2025-01-12 13:19:14 +02:00

batched.swift

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

benchmark

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

convert-llama2c-to-ggml

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

cvector-generator

CUDA: set compute parameters via command line arguments (#910)

2025-11-07 07:11:23 +02:00

deprecation-warning

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

embedding

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

eval-callback

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

export-lora

Merge vulkan code from mainline up to commit of 6/28/2025 (#563)

2025-07-02 08:49:42 +02:00

gbnf-validator

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

gguf

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

gguf-hash

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

gguf-split

gguf-split : update (#444)

2025-05-23 08:07:42 +03:00

gritlm

llama : allow pooled embeddings on any model (#7477)

2024-06-21 08:38:22 +03:00

imatrix

Fix imatrix calculation for MLA models (#411)

2025-05-13 17:53:38 +03:00

infill

Tool calls support from mainline (#723)

2025-09-01 08:38:49 +03:00

jeopardy

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

llama-bench

Graph reuse (#947)

2025-11-14 06:58:19 +02:00

llama.android

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llama.swiftui

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

llava

add dry sampler (#513)

2025-06-19 10:24:53 +03:00

lookahead

add dry sampler (#513)

2025-06-19 10:24:53 +03:00

lookup

add dry sampler (#513)

2025-06-19 10:24:53 +03:00

main

Port mdmd from mainline + Qwen2/2.5-VL support (#798)

2025-09-27 08:45:29 +02:00

main-cmake-pkg

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

mtmd

Add vision support in llama-server (#901)

2025-11-05 10:43:46 +02:00

parallel

Tool calls support from mainline (#723)

2025-09-01 08:38:49 +03:00

passkey

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

perplexity

More informative PPL readout line (#914)

2025-11-07 16:41:24 +02:00

quantize

Allow quantization of ffn_gate_inp (#896)

2025-11-05 10:44:32 +02:00

quantize-stats

Disable experimental code that causes issues with MSVC (#707)

2025-08-19 18:09:49 +03:00

retrieval

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

rpc

Fix cuda init error in rpc (#957)

2025-11-14 06:59:54 +02:00

save-load-state

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

server

server: cache prompt to host memory (#954)

2025-11-14 18:40:13 +02:00

simple

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

speculative

Support --device and --device-draft parameter (#866)

2025-10-27 18:13:28 +02:00

sweep-bench

sweep-bench: be able to set TG tokens via -n (#897)

2025-11-04 14:39:30 +02:00

sycl

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

tokenize

Merge mainline - Aug 12 2024 (#17)

2024-08-12 15:14:32 +02:00

base-translate.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

chat-13B.bat

Create chat-13B.bat (#592)

2023-03-29 20:21:09 +03:00

chat-13B.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

chat-persistent.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

chat-vicuna.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

chat.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

CMakeLists.txt

Port mdmd from mainline + Qwen2/2.5-VL support (#798)

2025-09-27 08:45:29 +02:00

convert_legacy_llama.py

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

json_schema_pydantic_example.py

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

json_schema_to_grammar.py

Tool calls support from mainline (#723)

2025-09-01 08:38:49 +03:00

llama.vim

llama.vim : added api key support (#5090)

2024-01-23 08:51:27 +02:00

llm.vim

llm.vim : stop generation at multiple linebreaks, bind to <F2> (#2879)

2023-08-30 09:50:55 +03:00

Miku.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

pydantic_models_to_grammar_examples.py

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

pydantic_models_to_grammar.py

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

reason-act.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

regex_to_grammar.py

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

server_embd.py

Merge mainline llama.cpp (#3)

2024-07-27 07:55:01 +02:00

server-llama2-13B.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

ts-type-to-grammar.sh

JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length (#6555)

2024-04-12 19:43:38 +01:00