From 6f2078cbe483e9c48a78f4ab066bfc49ee64a37e Mon Sep 17 00:00:00 2001 From: kingbri Date: Fri, 17 Nov 2023 22:02:21 -0500 Subject: [PATCH] Update README Signed-off-by: kingbri --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c8c7ae8..b6b137f 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ To get started, make sure you have the following installed on your system: - Python 3.x (preferably 3.11) with pip -- CUDA 12.1 or 11.8 +- CUDA 12.1 or 11.8 (or ROCm 5.6) NOTE: For Flash Attention 2 to work on Windows, CUDA 12.1 **must** be installed! @@ -92,6 +92,12 @@ All routes require an API key except for the following which require an **admin* - `/v1/model/unload` +## Common Issues + +- AMD cards will error out with flash attention installed, even if the config option is set to False. Run `pip uninstall flash_attn` to remove the wheel from your system. + + - See [#5](https://github.com/theroyallab/tabbyAPI/issues/5) + ## Contributing If you have issues with the project: