Xuan Son Nguyen 48c857aa10 server : refactored the task processing logic (#5065) ...

* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"

2024-01-26 14:42:20 +02:00

554 B

Raw Blame History

View Raw