mirror of
https://github.com/kvcache-ai/sglang.git
synced 2026-07-08 00:07:10 +00:00
14c95d255c
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
99 lines
3.1 KiB
Markdown
99 lines
3.1 KiB
Markdown
# SGLang Diffusion
|
|
|
|
SGLang Diffusion is an inference framework for accelerated image and video generation using diffusion models. It provides an end-to-end unified pipeline with optimized kernels and an efficient scheduler loop.
|
|
|
|
## Key Features
|
|
|
|
- **Broad Model Support**: Wan series, FastWan series, Hunyuan, Qwen-Image, Qwen-Image-Edit, Flux, Z-Image, GLM-Image, and more
|
|
- **Fast Inference**: Optimized kernels, efficient scheduler loop, and Cache-DiT acceleration
|
|
- **Ease of Use**: OpenAI-compatible API, CLI, and Python SDK
|
|
- **Multi-Platform**: NVIDIA GPUs (H100, H200, A100, B200, 4090), AMD GPUs (MI300X, MI325X) and Ascend NPU (A2, A3)
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
uv pip install "sglang[diffusion]" --prerelease=allow
|
|
```
|
|
|
|
See [Installation Guide](installation.md) for more installation methods and ROCm-specific instructions.
|
|
|
|
### Basic Usage
|
|
|
|
Generate an image with the CLI:
|
|
|
|
```bash
|
|
sglang generate --model-path Qwen/Qwen-Image \
|
|
--prompt "A beautiful sunset over the mountains" \
|
|
--save-output
|
|
```
|
|
|
|
Or start a server with the OpenAI-compatible API:
|
|
|
|
```bash
|
|
sglang serve --model-path Qwen/Qwen-Image --port 30010
|
|
```
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
### Getting Started
|
|
|
|
- **[Installation](installation.md)** - Install SGLang Diffusion via pip, uv, Docker, or from source
|
|
- **[Compatibility Matrix](compatibility_matrix.md)** - Supported models and optimization compatibility
|
|
|
|
### Usage
|
|
|
|
- **[CLI Documentation](api/cli.md)** - Command-line interface for `sglang generate` and `sglang serve`
|
|
- **[OpenAI API](api/openai_api.md)** - OpenAI-compatible API for image/video generation and LoRA management
|
|
|
|
### Performance Optimization
|
|
|
|
- **[Performance Overview](performance/index.md)** - Overview of all performance optimization strategies
|
|
- **[Attention Backends](performance/attention_backends.md)** - Available attention backends (FlashAttention, SageAttention, etc.)
|
|
- **[Caching Strategies](performance/cache/)** - Cache-DiT and TeaCache acceleration
|
|
- **[Profiling](performance/profiling.md)** - Profiling techniques with PyTorch Profiler and Nsight Systems
|
|
|
|
### Reference
|
|
|
|
- **[Environment Variables](environment_variables.md)** - Configuration via environment variables
|
|
- **[Support New Models](support_new_models.md)** - Guide for adding new diffusion models
|
|
- **[Contributing](contributing.md)** - Contribution guidelines and commit message conventions
|
|
- **[CI Performance](ci_perf.md)** - Performance baseline generation script
|
|
|
|
---
|
|
|
|
## CLI Quick Reference
|
|
|
|
### Generate (one-off generation)
|
|
|
|
```bash
|
|
sglang generate --model-path <MODEL> --prompt "<PROMPT>" --save-output
|
|
```
|
|
|
|
### Serve (HTTP server)
|
|
|
|
```bash
|
|
sglang serve --model-path <MODEL> --port 30010
|
|
```
|
|
|
|
### Enable Cache-DiT acceleration
|
|
|
|
```bash
|
|
SGLANG_CACHE_DIT_ENABLED=true sglang generate --model-path <MODEL> --prompt "<PROMPT>"
|
|
```
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
- [SGLang GitHub](https://github.com/sgl-project/sglang)
|
|
- [Cache-DiT](https://github.com/vipshop/cache-dit)
|
|
- [FastVideo](https://github.com/hao-ai-lab/FastVideo)
|
|
- [xDiT](https://github.com/xdit-project/xDiT)
|
|
- [Diffusers](https://github.com/huggingface/diffusers)
|