kingbri 06ff47e2b4 Model: Use true async jobs and add logprobs ...

The new async dynamic job allows for native async support without the
need of threading. Also add logprobs and metrics back to responses.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-05-25 21:16:14 -04:00

..

OAI

Model: Use true async jobs and add logprobs

2024-05-25 21:16:14 -04:00

server.py

API: Move OAI to APIRouter

2024-04-06 01:25:31 -04:00