mirror of
https://github.com/theroyallab/tabbyAPI.git
synced 2026-03-15 00:07:28 +00:00
Tree: Unify sampler parameters and add override support
Unify API sampler params into a superclass which should make them easier to manage and inherit generic functions from. Not all frontends expose all sampling parameters due to connections with OAI (that handles sampling themselves with the exception of a few sliders). Add the ability for the user to customize fallback parameters from server-side. In addition, parameters can be forced to a certain value server-side in case the repo automatically sets other sampler values in the background that the user doesn't want. Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
@@ -27,6 +27,14 @@ logging:
|
||||
# Enable generation parameter logging (default: False)
|
||||
generation_params: False
|
||||
|
||||
# Options for sampling
|
||||
sampling:
|
||||
# Override preset name. Find this in the sampler-overrides folder (default: None)
|
||||
# This overrides default fallbacks for sampler values that are passed to the API
|
||||
# Server-side overrides are NOT needed by default
|
||||
# WARNING: Using this can result in a generation speed penalty
|
||||
#override_preset:
|
||||
|
||||
# Options for model overrides and loading
|
||||
model:
|
||||
# Overrides the directory to look for models (default: models)
|
||||
|
||||
Reference in New Issue
Block a user