theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-26 01:08:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Model: Add CFG support

Browse Source 
CFG, or classifier-free guidance helps push a model in different
directions based on what the user provides.

Currently, CFG is ignored if the negative prompt is blank (it shouldn't
be used in that way anyways).

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-01-02 01:09:26 -05:00
committed by Brian Dashore
parent bb7a8e4614
commit b378773d0a
6 changed files with 96 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
OAI/types/model.py
View File

@@ -83,6 +83,7 @@ class ModelLoadRequest(BaseModel):
cache_mode: Optional[str] = "FP16"
prompt_template: Optional[str] = None
num_experts_per_token: Optional[int] = None
use_cfg: Optional[bool] = None
draft: Optional[DraftModelLoadRequest] = None