kingbri 2961c5f3f9 API: Handle request disconnect on non-streaming gens ...

Works the same way as streaming gens. If the request is cancelled,
it will log an error to the user and release the semaphore if it's
holding anything.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-03-21 23:12:59 -04:00

..

args.py

Tree: Format

2024-03-13 00:02:55 -04:00

auth.py

API: Cleanup permission endpoint

2024-03-18 15:13:26 -04:00

concurrency.py

API: Fix blocking iterator execution

2024-03-16 23:23:31 -04:00

config.py

Tree: Update to cleanup globals

2024-03-12 23:59:30 -04:00

gen_logging.py

Tree: Format

2024-03-13 23:33:18 -04:00

logger.py

Logging: Escape rich markup sequences

2024-03-11 00:28:48 -04:00

model.py

Model: Fix load if model didn't load properly

2024-03-16 23:23:31 -04:00

sampling.py

Model: Auto-scale max_tokens by default

2024-03-18 22:54:59 -04:00

signals.py

Signal: Fix signal handlers for uvicorn

2024-03-16 23:23:31 -04:00

templating.py

Tree: Format

2024-03-13 00:02:55 -04:00

utils.py

API: Handle request disconnect on non-streaming gens

2024-03-21 23:12:59 -04:00