theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-29 18:51:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix exception unloading models. (#138)

Browse Source 
self.generator is None if a model load fails or is cancelled.
This commit is contained in:
Glenn Maynard
2024-06-15 16:44:29 -05:00
committed by GitHub
parent 85387d97ad
commit 8da7644571
1 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
backends/exllamav2/model.py
View File

@@ -739,8 +739,9 @@ class ExllamaV2Container:
self.tokenizer = None self.tokenizer = None
# Cleanup the generator from any pending jobs # Cleanup the generator from any pending jobs
await self.generator.close() if self.generator is not None:
self.generator = None await self.generator.close()
self.generator = None
# Set all model state variables to False # Set all model state variables to False
self.model_is_loading = False self.model_is_loading = False