kingbri c8371e0f50 OAI: Copy gen params for "n" ...

For multiple generations in the same request, nested arrays kept their
original reference, resulting in duplications. This will occur with
any collection type.

For optimization purposes, a deepcopy isn't run for the first iteration
since original references are created.

This is not the most elegant solution, but it works for the described
cases.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-05-28 00:52:30 -04:00

..

chat_completion.py

OAI: Copy gen params for "n"

2024-05-28 00:52:30 -04:00

completion.py

OAI: Copy gen params for "n"

2024-05-28 00:52:30 -04:00

lora.py

Tree: Switch to async generators

2024-03-16 23:23:31 -04:00

model.py

API + Model: Add blocks and checks for various load requests

2024-05-25 21:16:14 -04:00