theroyallab/tabbyAPI
1
0
Fork 0
mirror of https://github.com/theroyallab/tabbyAPI.git synced 2026-03-14 15:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Docs: Update

Browse Source 
Update getting started and server options

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri
2025-03-12 00:40:15 -04:00
parent 4196bb6bc8
commit de77955428
2 changed files with 97 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/01.-Getting-Started.md
View File

@@ -6,7 +6,7 @@ To get started, make sure you have the following installed on your system:
- Do NOT install python from the Microsoft store! This will cause issues with pip.
- Alternatively, you can use miniconda or uv if it's present on your system.
> [!NOTE]
> Prefer a video guide? Watch the step-by-step tutorial on [YouTube](https://www.youtube.com/watch?v=03jYz0ijbUU)
@@ -40,16 +40,16 @@ To get started, make sure you have the following installed on your system:
### For Advanced Users
5. Follow steps 1-2 in the [For Beginners](#for-beginners) section
6. Create a python environment through venv:
1. Follow steps 1-2 in the [For Beginners](#for-beginners) section
2. Create a python environment through venv:
1. `python -m venv venv`
2. Activate the venv
1. On Windows: `.\venv\Scripts\activate`
2. On Linux: `source venv/bin/activate`
7. Install the pyproject features based on your system:
3. Install the pyproject features based on your system:
1. Cuda 12.x: `pip install -U .[cu121]`
2. ROCm 5.6: `pip install -U .[amd]`
8. Start the API by either
4. Start the API by either
1. Run `start.bat/sh`. The script will check if you're in a conda environment and skip venv checks.
2. Run `python main.py` to start the API. This won't automatically upgrade your dependencies.
@@ -78,19 +78,19 @@ You can also access the configuration parameters under [2. Configuration](https:
## Where next?
9. Take a look at the [usage docs](https://github.com/theroyallab/tabbyAPI/wiki/03.-Usage)
10. Get started with [community projects](https://github.com/theroyallab/tabbyAPI/wiki/09.-Community-Projects): Find loaders, UIs, and more created by the wider AI community. Any OAI compatible client is also supported.
1. Take a look at the [usage docs](https://github.com/theroyallab/tabbyAPI/wiki/03.-Usage)
2. Get started with [community projects](https://github.com/theroyallab/tabbyAPI/wiki/09.-Community-Projects): Find loaders, UIs, and more created by the wider AI community. Any OAI compatible client is also supported.
## Updating
There are a couple ways to update TabbyAPI:
11. **Update scripts** - Inside the update_scripts folder, you can run the following scripts:
1. **Update scripts** - Inside the update_scripts folder, you can run the following scripts:
1. `update_deps`: Updates dependencies to their latest versions.
2. `update_deps_and_pull`: Updates dependencies and pulls the latest commit of the Github repository.
These scripts exit after running their respective tasks. To start TabbyAPI, run `start.bat` or `start.sh`.
12. **Manual** - Install the pyproject features and update dependencies depending on your GPU:
2. **Manual** - Install the pyproject features and update dependencies depending on your GPU:
1. `pip install -U .[cu121]` = CUDA 12.x
2. `pip install -U .[amd]` = ROCm 6.0
@@ -113,11 +113,11 @@ NOTE:
Here are ways to install exllamav2:
13. From a [wheel/release](https://github.com/turboderp/exllamav2#method-2-install-from-release-with-prebuilt-extension) (Recommended)
1. From a [wheel/release](https://github.com/turboderp/exllamav2#method-2-install-from-release-with-prebuilt-extension) (Recommended)
1. Find the version that corresponds with your cuda and python version. For example, a wheel with `cu121` and `cp311` corresponds to CUDA 12.1 and python 3.11
14. From [pip](https://github.com/turboderp/exllamav2#method-3-install-from-pypi): `pip install exllamav2`
2. From [pip](https://github.com/turboderp/exllamav2#method-3-install-from-pypi): `pip install exllamav2`
2. This is a JIT compiled extension, which means that the initial launch of tabbyAPI will take some time. The build may also not work due to improper environment configuration.
15. From [source](https://github.com/turboderp/exllamav2#method-1-install-from-source)
3. From [source](https://github.com/turboderp/exllamav2#method-1-install-from-source)
## Other installation methods
@@ -128,15 +128,15 @@ These are short-form instructions for other methods that users can use to instal
### Conda
16. Install [Miniconda3](https://docs.conda.io/projects/miniconda/en/latest/miniconda-other-installer-links.html) with python 3.11 as your base python
17. Create a new conda environment `conda create -n tabbyAPI python=3.11`
18. Activate the conda environment `conda activate tabbyAPI`
19. Install optional dependencies if they aren't present
1. Install [Miniconda3](https://docs.conda.io/projects/miniconda/en/latest/miniconda-other-installer-links.html) with python 3.11 as your base python
2. Create a new conda environment `conda create -n tabbyAPI python=3.11`
3. Activate the conda environment `conda activate tabbyAPI`
4. Install optional dependencies if they aren't present
1. CUDA via
1. CUDA 12 - `conda install -c "nvidia/label/cuda-12.4.1" cuda`
2. Git via `conda install -k git`
20. Clone TabbyAPI via `git clone https://github.com/theroyallab/tabbyAPI`
21. Continue installation steps from:
5. Clone TabbyAPI via `git clone https://github.com/theroyallab/tabbyAPI`
6. Continue installation steps from:
1. [For Beginners](#for-beginners) - Step 3. The start scripts detect if you're in a conda environment and skips the venv check.
2. [For Advanced Users](#For-advanced-users) - Step 3
@@ -170,4 +170,4 @@ volumes:
    # Comment this to build a docker image from source
    image: ghcr.io/theroyallab/tabbyapi:latest
```
7. Run `docker compose -f docker/docker-compose.yml up` to build the dockerfile and start the server.
7. Run `docker compose -f docker/docker-compose.yml up` to build the dockerfile and start the server.