theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-05-11 16:30:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
46304ce875a6814c68b3cd45f617bc45196e49af
tabbyAPI/backends
History
kingbri 46304ce875 Model: Properly pass in max_batch_size from config 
The override wasn't being passed in before. Also, the default is now
none since Exl2 can automatically calculate the max batch size.

Signed-off-by: kingbri <bdashore3@proton.me>
2024-07-30 18:42:25 -04:00
..
exllamav2
Model: Properly pass in max_batch_size from config
2024-07-30 18:42:25 -04:00