ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-02-02 12:39:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,048 Commits 684 Branches 1 Tag
2bce5a7e2c5dfd3a8dd66fa07cc456ce2930e921
Commit Graph

2 Commits

Author SHA1 Message Date
Georgi Gerganov
32e84d74f1 sync : ggml 2024-01-27 17:00:24 +02:00
Xuan Son Nguyen
88bd8be65e server : refactored the task processing logic (#5065) 
* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"
 2024-01-26 14:42:20 +02:00