kingbri d45e847c7a API: Fix disconnect handling on streaming responses ...

Starlette's StreamingResponse has an issue where it yields after
a request has disconnected. A bugfix to starlette will fix this
issue, but FastAPI uses starlette <= 0.36 which isn't ideal.

Therefore, switch back to sse-starlette which handles these disconnects
correctly.

Also don't try yielding after the request is disconnected. Just return
out of the generator instead.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-03-10 17:43:13 -04:00

..

args.py

Config: Add experimental torch cuda malloc backend

2024-02-14 21:45:56 -05:00

auth.py

Logging: Switch to loguru

2024-03-08 01:00:48 -05:00

config.py

Logging: Switch to loguru

2024-03-08 01:00:48 -05:00

gen_logging.py

Logging: Switch to loguru

2024-03-08 01:00:48 -05:00

generators.py

API: Fix issues with concurrent requests and queueing

2024-03-04 23:21:40 -05:00

logger.py

Logger: Escape tags

2024-03-10 01:24:50 -05:00

sampling.py

Logging: Switch to loguru

2024-03-08 01:00:48 -05:00

templating.py

API: Add template switching and unload endpoints

2024-01-25 00:15:40 -05:00

utils.py

API: Fix disconnect handling on streaming responses

2024-03-10 17:43:13 -04:00