theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-20 06:19:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Tree: Remove fasttensors

Browse Source 
Now a noop in upstream.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-09-30 00:18:32 -04:00
parent 6726014d35
commit 126a44483c
4 changed files with 0 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
backends/exllamav2/model.py
View File

@@ -251,9 +251,6 @@ class ExllamaV2Container:
else:
self.config.scale_alpha_value = rope_alpha
# Enable fasttensors loading if present
self.config.fasttensors = unwrap(kwargs.get("fasttensors"), False)
# Set max batch size to the config override
self.max_batch_size = unwrap(kwargs.get("max_batch_size"))