kingbri 69e41e994c Model: Fix generation with non-streaming and logprobs ...

Finish_reason was giving an empty offset. Fix this by grabbing the
finish reason first and then handling the static generation as normal.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-03-21 00:47:24 -04:00

..

grammar.py

Logging: Switch to loguru

2024-03-08 01:00:48 -05:00

model.py

Model: Fix generation with non-streaming and logprobs

2024-03-21 00:47:24 -04:00

utils.py

Dependencies: Update Exllamav2

2024-03-20 15:21:37 -04:00