kingbri 0af6a38af3 Model: Add logprobs support ...

Returns token offsets, selected tokens, probabilities of tokens
post-sampling, and normalized probability of selecting a token
pre-sampling (for efficiency purposes).

Only for text completions. Chat completions in a later commit.

Signed-off-by: kingbri <bdashore3@proton.me>

2024-02-08 21:26:53 -05:00

..

model.py

Model: Add logprobs support

2024-02-08 21:26:53 -05:00

utils.py

Requirements: Update exllamav2, torch, and FA2

2024-02-02 23:53:42 -05:00