theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-27 09:41:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Launch: Make exllamav2 requirement more friendly

Browse Source 
Add the ability to use an unsafe config flag if needed and migrate
the exl2 check to a different file within the exl2 backend code.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-02-02 22:04:35 -05:00
parent b827bcbb44
commit 1919bf7705
5 changed files with 72 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
common/config.py
View File

@@ -55,6 +55,11 @@ def override_config_from_args(args: dict):
**{k.replace("log_", ""): logging_override[k] for k in logging_override},
}
developer_override = args.get("developer")
if developer_override:
developer_config = get_developer_config()
GLOBAL_CONFIG["developer"] = {**developer_config, **developer_override}
def get_sampling_config():
"""Returns the sampling parameter config from the global config"""
@@ -86,3 +91,8 @@ def get_network_config():
def get_gen_logging_config():
"""Returns the generation logging config from the global config"""
return unwrap(GLOBAL_CONFIG.get("logging"), {})
def get_developer_config():
"""Returns the developer specific config from the global config"""
return unwrap(GLOBAL_CONFIG.get("developer"), {})