theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Config: Update Q4 in comments

Browse Source 
Wasn't present when the option was added.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-03-17 01:04:12 -04:00
parent 14d8ec2007
commit 7abbac098a
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
config_sample.yml
View File

@@ -103,7 +103,8 @@ model:
# Disable Flash-attention 2. Set to True for GPUs lower than Nvidia's 3000 series. (default: False)
#no_flash_attention: False
# Enable 8 bit cache mode for VRAM savings (slight performance hit). Possible values FP16, FP8. (default: FP16)
# Enable 8 bit cache mode for VRAM savings (slight performance hit).
# Possible values FP16, FP8, Q4. (default: FP16)
#cache_mode: FP16
# Set the prompt template for this model. If empty, chat completions will be disabled. (default: Empty)