theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-20 14:28:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
47c42a23d409c017235ad588b9b08b0fb6ca37d0
tabbyAPI/main.py
kingbri 9a007c4707 Model: Add support for Q4 cache 
Add this in addition to 8bit cache and 16bit cache. Passing "Q4" with
the cache_mode request parameter will set this on model load.

Signed-off-by: kingbri <bdashore3@proton.me>
2024-03-06 00:59:28 -05:00

25 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink