theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Model: Fix no flash attention

Browse Source 
Was being called wrong from config.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2023-12-17 23:31:58 -05:00
parent ad8807a830
commit 95fd0f075e
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
model.py
View File

@@ -94,7 +94,7 @@ class ModelContainer:
)
# Turn off flash attention?
self.config.no_flash_attn = unwrap(kwargs.get("no_flash_attn"), False)
self.config.no_flash_attn = unwrap(kwargs.get("no_flash_attention"), False)
# low_mem is currently broken in exllamav2. Don't use it until it's fixed.
"""