kingbri 07d9b7cf7b Model: Add abort on generation ...

When the model is processing a prompt, add the ability to abort
on request cancellation. This is also a catch for a SIGINT.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-03-20 15:21:37 -04:00

..

types

API: Fix finish_reason returns

2024-03-18 15:59:28 -04:00

utils

Model: Add abort on generation

2024-03-20 15:21:37 -04:00

app.py

API: Cleanup permission endpoint

2024-03-18 15:13:26 -04:00