Launch: Make exllamav2 requirement more friendly

Add the ability to use an unsafe config flag if needed and migrate
the exl2 check to a different file within the exl2 backend code.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-02-02 22:04:35 -05:00
parent b827bcbb44
commit 1919bf7705
5 changed files with 72 additions and 22 deletions

View File

@@ -35,6 +35,13 @@ sampling:
# WARNING: Using this can result in a generation speed penalty
#override_preset:
# Options for development
developer:
# Skips exllamav2 version check (default: False)
# It's highly recommended to update your dependencies rather than enabling this flag
# WARNING: Don't set this unless you know what you're doing!
#unsafe_launch: False
# Options for model overrides and loading
model:
# Overrides the directory to look for models (default: models)