mirror of
https://github.com/kvcache-ai/sglang.git
synced 2026-07-18 01:37:57 +00:00
Co-authored-by: AdityaVKochar <adityavardhankochar@gmail.com> Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> Co-authored-by: adhyan-jain <adhyanjain2006@gmail.com> Co-authored-by: Adhyan Jain <71976554+adhyan-jain@users.noreply.github.com> Co-authored-by: Maitri-shah29 <maitrirajivshah@gmail.com> Co-authored-by: Adarsh Shirawalmath <114558126+adarshxs@users.noreply.github.com> Co-authored-by: Maitri Shah <shah29maitri@gmail.com> Co-authored-by: Aditya Vardhan Kochar <80113212+AdityaVKochar@users.noreply.github.com> Co-authored-by: Rishit Shivam <164783543+pokymono@users.noreply.github.com> Co-authored-by: Rishitshivam <164783543+Rishitshivam@users.noreply.github.com> Co-authored-by: IshhanKheria <ishhankheria06@gmail.com> Co-authored-by: Ishita Joshi <ishitata.joshi@gmail.com> Co-authored-by: Richard Chen <104477092+Richardczl98@users.noreply.github.com> Co-authored-by: longGGGGGG <553746008@qq.com> Co-authored-by: Richard <richardchen@radixark.ai> Co-authored-by: Nakul Sinha <nakul.new4socials@gmail.com> Co-authored-by: Divyam Agrawal <ludicrouslytrue@gmail.com> Co-authored-by: Richardczl98 <Zhenlinc@stanford.edu> Co-authored-by: Krishang Zinzuwadia <krishangzinzuwadia@gmail.com> Co-authored-by: nimeshas <nimesha.s106@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jignas Paturu <86356085+JignasP@users.noreply.github.com> Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
111 lines
3.2 KiB
Plaintext
111 lines
3.2 KiB
Plaintext
---
|
|
title: Install SGLang Diffusion
|
|
description: Install SGLang Diffusion on NVIDIA, AMD, MUSA, and Ascend platforms.
|
|
---
|
|
|
|
You can install SGLang Diffusion using one of the methods below.
|
|
|
|
## Standard installation (NVIDIA GPUs)
|
|
|
|
**Platform:** NVIDIA GPUs (CUDA)
|
|
|
|
<Tabs>
|
|
<Tab title="Pip or uv">
|
|
Use `uv` for faster installation:
|
|
|
|
```bash
|
|
pip install --upgrade pip
|
|
pip install uv
|
|
uv pip install "sglang[diffusion]" --prerelease=allow
|
|
```
|
|
</Tab>
|
|
|
|
<Tab title="From source">
|
|
```bash
|
|
git clone https://github.com/sgl-project/sglang.git
|
|
cd sglang
|
|
pip install --upgrade pip
|
|
pip install -e "python[diffusion]"
|
|
```
|
|
|
|
Or with `uv`:
|
|
|
|
```bash
|
|
uv pip install -e "python[diffusion]" --prerelease=allow
|
|
```
|
|
</Tab>
|
|
|
|
<Tab title="Docker">
|
|
The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags), built from the [Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile). Replace `<secret>` below with your HuggingFace Hub [token](https://huggingface.co/docs/hub/en/security-tokens).
|
|
|
|
```bash
|
|
docker run --gpus all \
|
|
--shm-size 32g \
|
|
-p 30000:30000 \
|
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
|
--env "HF_TOKEN=<secret>" \
|
|
--ipc=host \
|
|
lmsysorg/sglang:dev \
|
|
zsh -c '\
|
|
echo "Installing diffusion dependencies..." && \
|
|
pip install -e "python[diffusion]" && \
|
|
echo "Starting SGLang-Diffusion..." && \
|
|
sglang generate \
|
|
--model-path black-forest-labs/FLUX.1-dev \
|
|
--prompt "A logo With Bold Large text: SGL Diffusion" \
|
|
--save-output \
|
|
'
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Platform-specific installs
|
|
|
|
Use the tab that matches your accelerator.
|
|
|
|
<Tabs>
|
|
<Tab title="ROCm (AMD GPUs)">
|
|
**Platform:** AMD Instinct GPUs (ROCm)
|
|
|
|
For AMD Instinct GPUs (for example, MI300X), use the ROCm-enabled Docker image:
|
|
|
|
```bash
|
|
docker run --device=/dev/kfd --device=/dev/dri --ipc=host \
|
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
|
--env HF_TOKEN=<secret> \
|
|
lmsysorg/sglang:v0.5.9-rocm700-mi30x \
|
|
sglang generate --model-path black-forest-labs/FLUX.1-dev --prompt "A logo With Bold Large text: SGL Diffusion" --save-output
|
|
```
|
|
|
|
For detailed ROCm system configuration and installation from source, see [AMD GPUs](../hardware-platforms/amd-gpus).
|
|
</Tab>
|
|
|
|
<Tab title="MUSA (Moore Threads GPUs)">
|
|
**Platform:** Moore Threads GPUs (MUSA)
|
|
|
|
For Moore Threads GPUs (MTGPU) with the MUSA software stack:
|
|
|
|
```bash
|
|
git clone https://github.com/sgl-project/sglang.git
|
|
cd sglang
|
|
pip install --upgrade pip
|
|
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
|
|
pip install -e "python[all_musa]"
|
|
```
|
|
</Tab>
|
|
|
|
<Tab title="Ascend NPU">
|
|
**Platform:** Ascend NPU
|
|
|
|
For Ascend NPU, follow the [NPU installation guide](../hardware-platforms/ascend-npus/SGLang-installation-with-NPUs-support).
|
|
|
|
Quick test:
|
|
|
|
```bash
|
|
sglang generate --model-path black-forest-labs/FLUX.1-dev \
|
|
--prompt "A logo With Bold Large text: SGL Diffusion" \
|
|
--save-output
|
|
```
|
|
</Tab>
|
|
</Tabs>
|