theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-20 06:19:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
eca403a0e4877e46431709a052d12035aa933cb7
tabbyAPI/backends/exllamav2/model.py
kingbri bdc5189a4b Exl3: Add chunk size, cache size, and model info 
Use the same algorithm for estimating and adjusting cache size based
on multiples of 256 and above max seq len.

Same applies for chunk size.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
2025-05-02 21:33:25 -04:00

52 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink