kingbri b944f8d756 OAI: Add "n" for non-streaming generations ...

This adds the ability to add multiple choices to a generation. This
is only available for non-streaming gens for now, it requires some
more work to port over to streaming.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-05-28 00:52:30 -04:00

..

types

OAI: Add "n" for non-streaming generations

2024-05-28 00:52:30 -04:00

utils

OAI: Add "n" for non-streaming generations

2024-05-28 00:52:30 -04:00

router.py

OAI: Fix request cancellation behavior

2024-05-26 13:00:33 -04:00