kingbri cce97deea5 Model: Switch logprobs to use post-sampling ...

Previously, pre-sampling logprobs were used from the raw logits,
but newer versions of exl2 allow for returning token probs post-sampling.
Convert these to logprobs and send to the user.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-02-14 21:51:25 -05:00

..

model.py

Model: Switch logprobs to use post-sampling

2024-02-14 21:51:25 -05:00

utils.py

Packages: Fix exllamav2 version check

2024-02-10 14:00:26 -05:00