kingbri c7428f0bcd API: Add logprobs for chat completions ...

Adds chat completion logprob support using OAI's spec. Tokens are
not converted to tiktoken here since that will add an extra dependency
for no real reason.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-02-08 21:26:53 -05:00

..

model.py

API: Add logprobs for chat completions

2024-02-08 21:26:53 -05:00

utils.py

Requirements: Update exllamav2, torch, and FA2

2024-02-02 23:53:42 -05:00