mirror of
https://github.com/theroyallab/tabbyAPI.git
synced 2026-03-14 15:57:27 +00:00
Update README
Add alternatives if the user doesn't agree with the focus of TabbyAPI. Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
20
README.md
20
README.md
@@ -146,6 +146,26 @@ All routes require an API key except for the following which require an **admin*
|
||||
|
||||
- Restart TabbyAPI and launching should work again.
|
||||
|
||||
## Supported Model Types
|
||||
|
||||
TabbyAPI uses Exllamav2 as a powerful and fast backend for model inference, loading, etc. Therefore, the following types of models are supported:
|
||||
|
||||
- Exl2 (Highly recommended)
|
||||
|
||||
- GPTQ
|
||||
|
||||
- FP16 (using Exllamav2's loader)
|
||||
|
||||
#### Alternative Loaders/Backends
|
||||
|
||||
If you want to use a different model type than the ones listed above, here are some alternative backends with their own APIs:
|
||||
|
||||
- GGUF + GGML - [KoboldCPP](https://github.com/lostruins/KoboldCPP)
|
||||
|
||||
- AWQ - [Aphrodite Engine](https://github.com/PygmalionAI/Aphrodite-engine)
|
||||
|
||||
- [Text Generation WebUI](https://github.com/oobabooga/text-generation-webui)
|
||||
|
||||
## Contributing
|
||||
|
||||
If you have issues with the project:
|
||||
|
||||
Reference in New Issue
Block a user