kingbri d2c6ae2d35 API: Back to async ...

According to FastAPI docs, if you're using a generic function, running
it in async will make it more performant (which makes sense since
running def functions for routes will automatically run the caller
through a threadpool).

Tested and everything works fine.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-03-04 23:21:40 -05:00

..

args.py

Config: Add experimental torch cuda malloc backend

2024-02-14 21:45:56 -05:00

auth.py

API: Back to async

2024-03-04 23:21:40 -05:00

config.py

Launch: Make exllamav2 requirement more friendly

2024-02-02 23:36:17 -05:00

gen_logging.py

Tree: Refactor code organization

2024-01-25 00:15:40 -05:00

generators.py

API: Fix issues with concurrent requests and queueing

2024-03-04 23:21:40 -05:00

logger.py

Tree: Refactor code organization

2024-01-25 00:15:40 -05:00

sampling.py

Model: Add EBNF grammar support

2024-02-24 23:40:11 -05:00

templating.py

API: Add template switching and unload endpoints

2024-01-25 00:15:40 -05:00

utils.py

API: Add more methods to semaphore

2024-03-04 23:21:40 -05:00