theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-04-22 07:19:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Dependencies: Update torch, exllamav2, and flash-attn

Browse Source 
Torch - 2.6.0
ExllamaV2 - 0.2.8
Flash-attn - 2.7.4.post1

Cuda wheels are now 12.4 instead of 12.1, feature names need to be
migrated over.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri
2025-02-09 01:25:47 -05:00
parent dcbf2de9e5
commit 0dcbb7a722
3 changed files with 39 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
backends/exllamav2/utils.py
View File

@@ -52,8 +52,6 @@ def supports_paged_attn():
"using the following command:\n\n"
"For CUDA 12.1:\n"
"pip install --upgrade .[cu121]\n\n"
"For CUDA 11.8:\n"
"pip install --upgrade .[cu118]\n\n"
"NOTE: Windows users must use CUDA 12.x to use flash-attn."
)