kingbri fb1d2f34c1 OAI: Add response_prefix and fix BOS token issues in chat completions ...

response_prefix is used to add a prefix before generating the next
message. This is used in many cases such as continuining a prompt
(see #96).

Also if a template has BOS token specified, add_bos_token will
append two BOS tokens. Add a check which strips a starting BOS token
from the prompt if it exists.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-04-25 00:54:43 -04:00

..

auth.py

Endpoints: Add key permission checker

2024-03-18 00:53:27 -04:00

chat_completion.py

OAI: Add response_prefix and fix BOS token issues in chat completions

2024-04-25 00:54:43 -04:00

common.py

OAI: Add response_format parameter

2024-04-09 21:33:31 -04:00

completion.py

API: Fix finish_reason returns

2024-03-18 15:59:28 -04:00

lora.py

Tree: Format

2024-03-13 00:02:55 -04:00

model.py

Model: Fix chunk size handling

2024-04-07 18:39:19 -04:00

sampler_overrides.py

API: Split into separate folder

2024-03-12 23:59:30 -04:00

template.py

API: Split into separate folder

2024-03-12 23:59:30 -04:00

token.py

OAI: Tokenize chat completion messages

2024-04-15 14:17:16 -04:00