From b364de1005b23326e190968a24d10deb36a28ee5 Mon Sep 17 00:00:00 2001 From: kingbri Date: Sun, 10 Dec 2023 16:05:46 -0500 Subject: [PATCH] Update README Add alternatives if the user doesn't agree with the focus of TabbyAPI. Signed-off-by: kingbri --- README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/README.md b/README.md index f8f2235..ddd486b 100644 --- a/README.md +++ b/README.md @@ -146,6 +146,26 @@ All routes require an API key except for the following which require an **admin* - Restart TabbyAPI and launching should work again. +## Supported Model Types + +TabbyAPI uses Exllamav2 as a powerful and fast backend for model inference, loading, etc. Therefore, the following types of models are supported: + +- Exl2 (Highly recommended) + +- GPTQ + +- FP16 (using Exllamav2's loader) + +#### Alternative Loaders/Backends + +If you want to use a different model type than the ones listed above, here are some alternative backends with their own APIs: + +- GGUF + GGML - [KoboldCPP](https://github.com/lostruins/KoboldCPP) + +- AWQ - [Aphrodite Engine](https://github.com/PygmalionAI/Aphrodite-engine) + +- [Text Generation WebUI](https://github.com/oobabooga/text-generation-webui) + ## Contributing If you have issues with the project: