mirror of
https://github.com/theroyallab/tabbyAPI.git
synced 2026-03-14 15:57:27 +00:00
API: Don't fallback to default values on model load request
It's best to pass them down the config stack. API/User config.yml -> model config.yml -> model config.json -> fallback. Doing this allows for seamless flow and yielding control to each member in the stack. Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
@@ -361,6 +361,8 @@ class ExllamaV2Container:
|
||||
self.draft_config.max_attention_size = chunk_size**2
|
||||
|
||||
def set_model_overrides(self, **kwargs):
|
||||
"""Sets overrides from a model folder's config yaml."""
|
||||
|
||||
override_config_path = self.model_dir / "tabby_config.yml"
|
||||
|
||||
if not override_config_path.exists():
|
||||
|
||||
Reference in New Issue
Block a user