theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

OAI: adding optional draft model properties for draft_rope alpha and scale (#28)

Browse Source 
* OAI: adding optional draft model properties for draft_rope alpha and scale

* Forgot to set the properties to None
This commit is contained in:
Veden
2023-12-17 11:23:45 -08:00
committed by GitHub
parent 528d58f841
commit 3e57125025
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
OAI/types/model.py
View File

@@ -25,8 +25,8 @@ class ModelList(BaseModel):
class DraftModelLoadRequest(BaseModel):
draft_model_name: str
draft_rope_alpha: float = 1.0
draft_rope_scale: float = 1.0
draft_rope_alpha: Optional[float] = None
draft_rope_scale: Optional[float] = None
# TODO: Unify this with ModelCardParams
class ModelLoadRequest(BaseModel):