From 72dff0b6d5f357cfa1d53451636a16316764b628 Mon Sep 17 00:00:00 2001 From: kingbri Date: Sun, 28 Apr 2024 11:26:20 -0400 Subject: [PATCH] Update README Signed-off-by: kingbri --- README.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 98a1e08..64eafd8 100644 --- a/README.md +++ b/README.md @@ -12,10 +12,16 @@ A FastAPI based application that allows for generating text using an LLM (large ## Disclaimer -This API is considered as rolling release. There may be bugs and changes down the line. Please be aware that you might need to reinstall dependencies if needed. +This project is marked rolling release. There may be bugs and changes down the line. Please be aware that you might need to reinstall dependencies if needed. + +TabbyAPI is a hobby project solely for a small amount of users. It is not meant to run on production servers. For that, please look at other backends that support those workloads. ## Getting Started +> [!IMPORTANT] +> +> This README is not for getting started. Please read the Wiki. + Read the [Wiki](https://github.com/theroyallab/tabbyAPI/wiki/1.-Getting-Started) for more information. It contains user-facing documentation for installation, configuration, sampling, API usage, and so much more. ## Supported Model Types @@ -30,19 +36,23 @@ TabbyAPI uses Exllamav2 as a powerful and fast backend for model inference, load #### Alternative Loaders/Backends -If you want to use a different model type than the ones listed above, here are some alternative backends with their own APIs: +If you want to use a different model type or quantization method than the ones listed above, here are some alternative backends with their own APIs: - GGUF + GGML - [KoboldCPP](https://github.com/lostruins/KoboldCPP) -- AWQ - [Aphrodite Engine](https://github.com/PygmalionAI/Aphrodite-engine) +- Production ready + Many other quants + batching [Aphrodite Engine](https://github.com/PygmalionAI/Aphrodite-engine) + +- Production ready + batching [VLLM](https://github.com/vllm-project/vllm) - [Text Generation WebUI](https://github.com/oobabooga/text-generation-webui) ## Contributing +Use the template when creating issues or pull requests, otherwise the developers may not look at your post. + If you have issues with the project: -- Describe the issues in detail +- Describe the issue in detail - If you have a feature request, please indicate it as such.