mirror of
https://github.com/kvcache-ai/sglang.git
synced 2026-06-30 19:57:52 +00:00
c4db64c16b
Co-authored-by: Zijie Xia <zijie_xia@icloud.com> Co-authored-by: Zijie Xia <zijiexia@users.noreply.github.com> Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
52 lines
1.7 KiB
Markdown
52 lines
1.7 KiB
Markdown
# Set Up Self-Hosted Runners for GitHub Actions
|
|
|
|
## Add a Runner
|
|
|
|
### Step 1: Start a docker container.
|
|
|
|
**You can mount a folder for the shared huggingface model weights cache. **
|
|
The command below uses `/tmp/huggingface` as an example.
|
|
|
|
```
|
|
docker pull nvidia/cuda:12.9.1-devel-ubuntu22.04
|
|
# Nvidia
|
|
docker run --shm-size 128g -it -v /tmp/huggingface:/hf_home --gpus all nvidia/cuda:12.9.1-devel-ubuntu22.04 /bin/bash
|
|
# AMD
|
|
docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video --shm-size 128g -it -v /tmp/huggingface:/hf_home lmsysorg/sglang:v0.5.8-rocm700-mi30x /bin/bash
|
|
# AMD just the last 2 GPUs
|
|
docker run --rm --device=/dev/kfd --device=/dev/dri/renderD176 --device=/dev/dri/renderD184 --group-add video --shm-size 128g -it -v /tmp/huggingface:/hf_home lmsysorg/sglang:v0.5.8-rocm700-mi30x /bin/bash
|
|
```
|
|
|
|
### Step 2: Configure the runner by `config.sh`
|
|
|
|
Run these commands inside the container.
|
|
|
|
```
|
|
apt update && apt install -y curl python3-pip git
|
|
pip install --upgrade pip
|
|
export RUNNER_ALLOW_RUNASROOT=1
|
|
```
|
|
|
|
Then follow https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners to run `config.sh`
|
|
|
|
**Notes**
|
|
- Do not need to specify the runner group
|
|
- Give it a name (e.g., `test-sgl-gpu-0`) and some labels (e.g., `1-gpu-h100`). The labels can be edited later in Github Settings.
|
|
- Do not need to change the work folder.
|
|
|
|
### Step 3: Run the runner by `run.sh`
|
|
|
|
- Set up environment variables
|
|
```
|
|
export HF_HOME=/hf_home
|
|
export SGLANG_IS_IN_CI=true
|
|
export HF_TOKEN=hf_xxx
|
|
export OPENAI_API_KEY=sk-xxx
|
|
export CUDA_VISIBLE_DEVICES=0
|
|
```
|
|
|
|
- Run it forever
|
|
```
|
|
while true; do ./run.sh; echo "Restarting..."; sleep 2; done
|
|
```
|