turboderp 79d581e1f5 OAI endpoints: More rework ...

- remove disconnect_task
- move disconnect logic to a per-request handler that wraps cleanup operation and directly polls the request state with throttling
- exclusively signal disconnect with CancelledError
- rework completions endpoint to follow same approach as chat completions, share some code
- refactor OAI endpoints a bit
- correct behavior for batched completion requests
- make sure logprobs work for completion and streaming completion requests
- more tests

2026-04-02 01:26:44 +02:00

..

_common.py

OAI endpoints: More rework

2026-04-02 01:26:44 +02:00

req_completion.py

OAI endpoints: More rework

2026-04-02 01:26:44 +02:00

req_logprob.py

OAI endpoints: More rework

2026-04-02 01:26:44 +02:00

req_long.py

OAI endpoints: More rework

2026-04-02 01:26:44 +02:00

req_tool_calls.py

Add inference test scripts

2026-03-30 00:23:25 +02:00

wheel_test.py

Tree: Format

2025-05-17 00:46:40 -04:00