DocShotgun 55d979b7a5 Update dependencies, support Python 3.12, update for exl2 0.1.5 (#134) ...

* Dependencies: Add wheels for Python 3.12

* Model: Switch fp8 cache to Q8 cache

* Model: Add ability to set draft model cache mode

* Dependencies: Bump exllamav2 to 0.1.5

* Model: Support Q6 cache

* Config: Add Q6 cache and draft_cache_mode to config sample

2024-06-09 17:27:39 +02:00

6.5 KiB

Raw Blame History

View Raw