theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-20 14:28:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

OAI: Add fasttensors to model load endpoint

Browse Source 
Also fix logging when loading prompt templates.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-01-25 01:01:29 -05:00
parent fc4570220c
commit 751627e571
2 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
backends/exllamav2/model.py
View File

@@ -243,7 +243,7 @@ class ExllamaV2Container:
def find_prompt_template(self, prompt_template_name, model_directory):
"""Tries to find a prompt template using various methods"""
logger.info("Loading prompt template with name " f"{prompt_template_name}")
logger.info("Attempting to load a prompt template if present.")
find_template_functions = [
lambda: get_template_from_model_json(