theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-26 01:08:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Config: Switch to YAML and add load progress

Browse Source 
YAML is a more flexible format when it comes to configuration. Commandline
arguments are difficult to remember and configure especially for
an API with complicated commandline names. Rather than using half-baked
textfiles, implement a proper config solution.

Also add a progress bar when loading models in the commandline.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2023-11-12 00:21:16 -05:00
parent 5d32aa02cd
commit a10c14d357
6 changed files with 38 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
config_sample.yml Normal file
View File

@@ -0,0 +1,8 @@
model_dir: "D:/models"
model_name: "this_is_a_exl2_model"
max_seq_len: 4096
gpu_split: "auto"
rope_scale: 1.0
rope_alpha: 1.0
no_flash_attention: False
low_mem: False