API: Fix load exception handling

Models do not fully unload if an exception is caught in load. Therefore,
leave it to the client to unload on cancel.

Also add handlers in the event a SSE stream is cancelled. These packets
can't be sent back to the client since the client has severed the
connection, so print them in terminal.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2023-12-05 00:23:15 -05:00
parent 7c92968558
commit 8ba3bfa6b3
2 changed files with 18 additions and 15 deletions

View File

@@ -14,9 +14,9 @@ class TabbyGeneratorErrorMessage(BaseModel):
class TabbyGeneratorError(BaseModel):
error: TabbyGeneratorErrorMessage
def get_generator_error(exception: Exception):
def get_generator_error(message: str):
error_message = TabbyGeneratorErrorMessage(
message = str(exception),
message = message,
trace = traceback.format_exc()
)