mirror of
https://github.com/theroyallab/tabbyAPI.git
synced 2026-03-15 00:07:28 +00:00
Model: Read scale_pos_emb from config
In newer versions of exllamav2, this value is read from the model's config.json. This value will still default to 1.0 anyways. Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
@@ -60,6 +60,7 @@ model:
|
||||
# Rope scale (default: 1.0)
|
||||
# Same thing as compress_pos_emb
|
||||
# Only use if your model was trained on long context with rope (check config.json)
|
||||
# Leave blank to pull the value from the model
|
||||
rope_scale: 1.0
|
||||
|
||||
# Rope alpha (default: 1.0)
|
||||
|
||||
Reference in New Issue
Block a user