DocShotgun 5da335eb3d Model: Robust request length checking in generator ...

* Ensure that length of positive/negative prompt + max_tokens does not exceed max_seq_len
* Ensure that total required pages for CFG request does not exceed allocated cache_size

2024-12-17 09:34:43 -08:00

..

exllamav2

Model: Robust request length checking in generator

2024-12-17 09:34:43 -08:00

infinity

Infinity: Use a runtime type hint for engine

2024-11-22 18:06:08 -05:00