kingbri 94696543bc Model: Warn user if context > max_seq_len ...

Unlike other backends, tabby attempts to generate even if the context
is greater than the max sequence length via truncation of the given
context.

Rather than artifically erroring out, give a warning that outputted
console metrics are going to be incorrect and to make sure that
context <= max_seq_len.

Signed-off-by: kingbri <bdashore3@proton.me>

2023-11-29 01:35:32 -05:00

17 KiB

Raw Blame History

View Raw