theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update README

Browse Source 
Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2023-11-17 22:02:21 -05:00
parent d627d14385
commit 6f2078cbe4
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@@ -18,7 +18,7 @@ To get started, make sure you have the following installed on your system:
- Python 3.x (preferably 3.11) with pip
- CUDA 12.1 or 11.8
- CUDA 12.1 or 11.8 (or ROCm 5.6)
NOTE: For Flash Attention 2 to work on Windows, CUDA 12.1 **must** be installed!
@@ -92,6 +92,12 @@ All routes require an API key except for the following which require an **admin*
- `/v1/model/unload`
## Common Issues
- AMD cards will error out with flash attention installed, even if the config option is set to False. Run `pip uninstall flash_attn` to remove the wheel from your system.
- See [#5](https://github.com/theroyallab/tabbyAPI/issues/5)
## Contributing
If you have issues with the project: