theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Model: Fix draft model loading

Browse Source 
Use draft_config to find the path instead of kwargs.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2023-11-19 02:04:02 -05:00
parent 698b0b1976
commit 4cddd0400c
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
model.py
View File

@@ -95,7 +95,7 @@ class ModelContainer:
if self.draft_enabled:
self.draft_config = ExLlamaV2Config()
draft_model_path = pathlib.Path(kwargs.get("draft_model_dir") or "models")
draft_model_path = pathlib.Path(draft_config.get("draft_model_dir") or "models")
draft_model_path = draft_model_path / draft_model_name
self.draft_config.model_dir = str(draft_model_path.resolve())