ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-03-13 23:40:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
01e2261e5f1770cf9d6f110c22af52c7cd707d61
ik_llama.cpp/llama.cpp
Christian Falch 35405e6856 Added api for getting/setting the kv_cache (#685) 
The api provides access methods for retrieving the current memory buffer for the kv_cache and its token number.
It also contains a method for setting the kv_cache from a memory buffer.

This makes it possible to load/save history - maybe support --cache-prompt paramater as well?

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
2023-04-02 12:23:04 +02:00

59 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink