kingbri 5e54911cc8 API: Fix semaphore handling and chat completion errors ...

Chat completions previously always yielded a final packet to say that
a generation finished. However, this caused errors that a yield was
executed after GeneratorExit. This is correctly stated because python's
garbage collector can't clean up the generator after exiting due to the
finally block executing.

In addition, SSE endpoints close off the connection, so the finish packet
can only be yielded when the response has completed, so ignore yield on
exception.

Signed-off-by: kingbri <bdashore3@proton.me>

2023-12-04 15:51:25 -05:00

11 KiB

Raw Blame History

View Raw