kingbri
d0b6b11068
OAI: Make freq and presence pen floats
...
Also rename the completions typing file.
Signed-off-by: kingbri <bdashore3@proton.me >
2023-11-15 00:55:15 -05:00
kingbri
126afdfdc2
Model: Fix gpu split params
...
GPU split auto is a bool and GPU split is an array of integers for
GBs to allocate per GPU.
Signed-off-by: kingbri <bdashore3@proton.me >
2023-11-15 00:55:15 -05:00
kingbri
ea91d17a11
Api: Add ban_eos_token and add_bos_token support
...
Adds the ability for the client to specify whether to add the BOS
token and ban the EOS token.
Signed-off-by: kingbri <bdashore3@proton.me >
2023-11-15 00:55:15 -05:00
kingbri
8fea5391a8
Api: Add token endpoints
...
Support for encoding and decoding with various parameters.
Signed-off-by: kingbri <bdashore3@proton.me >
2023-11-15 00:55:15 -05:00
kingbri
4670a77c26
API: Don't use response_class
...
This arg in routes caused many errors and isn't even needed for
responses.
Signed-off-by: kingbri <bdashore3@proton.me >
2023-11-14 22:09:26 -05:00
kingbri
b625bface9
OAI: Add API-based model loading/unloading and auth routes
...
Models can be loaded and unloaded via the API. Also add authentication
to use the API and for administrator tasks.
Both types of authorization use different keys.
Also fix the unload function to properly free all used vram.
Signed-off-by: kingbri <bdashore3@proton.me >
2023-11-14 01:17:19 -05:00
