kingbri 06ff47e2b4 Model: Use true async jobs and add logprobs ...

The new async dynamic job allows for native async support without the
need of threading. Also add logprobs and metrics back to responses.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-05-25 21:16:14 -04:00

..

args.py

Tree: Format

2024-03-13 00:02:55 -04:00

auth.py

API: Cleanup permission endpoint

2024-03-18 15:13:26 -04:00

concurrency.py

Concurrency: Remove release_semaphore method

2024-05-19 10:42:26 -04:00

config.py

Tree: Update to cleanup globals

2024-03-12 23:59:30 -04:00

downloader.py

Downloader: Cleanup on exception

2024-04-30 23:26:22 -04:00

gen_logging.py

Model: Use true async jobs and add logprobs

2024-05-25 21:16:14 -04:00

logger.py

API: Add HuggingFace downloader

2024-04-29 01:15:02 -04:00

model.py

Common: Migrate request utils to networking

2024-03-21 23:21:57 -04:00

networking.py

Concurrency: Remove release_semaphore method

2024-05-19 10:42:26 -04:00

sampling.py

Sampling: Copy over iterable overrides

2024-05-17 21:38:28 -04:00

signals.py

Signal: Fix signal handlers for uvicorn

2024-03-16 23:23:31 -04:00

templating.py

Templates: Migrate to class

2024-04-21 23:28:14 -04:00

transformers_utils.py

Tree: Add transformers_utils

2024-04-20 00:07:39 -04:00

utils.py

Sampling: Copy over iterable overrides

2024-05-17 21:38:28 -04:00