kingbri bdc5189a4b Exl3: Add chunk size, cache size, and model info ...

Use the same algorithm for estimating and adjusting cache size based
on multiples of 256 and above max seq len.

Same applies for chunk size.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>

2025-05-02 21:33:25 -04:00

..

grammar.py

Grammar: Cache the engine vocabulary

2024-12-05 21:36:37 -08:00

model.py

Exl3: Add chunk size, cache size, and model info

2025-05-02 21:33:25 -04:00

utils.py

Exl3: Add chunk size, cache size, and model info

2025-05-02 21:33:25 -04:00

version.py

Dependencies: Update torch, exllamav2, and flash-attn

2025-02-09 01:27:48 -05:00

vision.py

Dependencies: Fix OpenAPI generation

2024-11-22 17:59:20 -05:00