kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-07 07:47:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,985 Commits 157 Branches 0 Tags
a45e0e5df4e6e2d765afcaa00b16a67694f5170e
Commit Graph

5794 Commits

Author SHA1 Message Date
YAMY
a45e0e5df4 [SPEC_V2] Enable cudagraph draft_extend for trtllm_mla_backend and Acclen Fix for DP under cudagraph mode (#16974) 2026-01-18 15:56:21 +08:00
Changyi Yang
8fd3399880 [diffusion] fix: set guidance_scale default to None (#17182) 2026-01-18 15:04:11 +08:00
Yuan Luo
6d29d8ab16 [VLM][Reland] Refactor load_mm_data to improve performance (#16152) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-18 14:11:17 +08:00
Ke Bao
e499258e97 Use swa radix cache and memory pool for gpt-oss model (#17261) 2026-01-18 13:47:07 +08:00
Mick
09491a9bcd cli: support sglang version (#17250) 2026-01-18 13:20:24 +08:00
Lancer
e486a4dac1 [diffusion] feat: support default 4-step inference for Flux2-Klein distilled models (#17225) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-01-18 10:15:50 +08:00
Hudson Xing
90399cbc07 fix(ci): recover from corrupted MMMU parquet cache (#17256) 2026-01-17 17:32:01 -08:00
Michael
53609e5e5b Revert "[Diffusion] Move diffusion time embedding to jit kernel" (#17257) 2026-01-17 21:29:22 +08:00
Zehuan Li
d2c863878c [DLLM] Implement initial dynamic batching for diffusion LLM (#14883) 2026-01-17 16:48:15 +08:00
Yi Zhang
737a1183d6 [BUGFIX] fix radix cache memory consumption to avoid OOM (#17191) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-17 16:47:37 +08:00
Mick
dc743fe4ba [diffusion] chore: clean srt imports (#17252) 2026-01-17 15:47:49 +08:00
Nan Jiang
dd99f818e0 fix: fix regression and unclear pattern (#16561) 2026-01-16 23:21:42 -08:00
Hudson Xing
8ce64aa155 fix(ci): skip offline mode for LoRA scenarios (#17248) 2026-01-16 23:06:20 -08:00
Mick
eb768189ee [diffusion] doc: add instruction for adding performance baseline of new model (#17249) 2026-01-17 12:24:59 +08:00
Xiaoyu Zhang
2cdd4370bc [Diffusion] Move diffusion time embedding to jit kernel (#16879) 2026-01-17 12:21:22 +08:00
siyu
e18e0057a4 Add an env var to allow transferring small metadata via TCP for PD (#16951) 2026-01-16 18:32:22 -08:00
Stefan He
43779f27b7 Disable PCG for draft worker (#16354) 2026-01-17 09:56:56 +08:00
Baizhou Zhang
8b9e9357fe [2/n] deepseek_v2.py Refactor: Migrate MHA forward method in deepseek_v2.py (#16817) 2026-01-17 09:36:25 +08:00
b8zhong
d36f6f043c [Fix] flashinfer_trtllm intermediate_size assertion with Qwen3 + TP=8 (#16824) 2026-01-17 09:24:05 +08:00
Lingjun Wen
7f8353aff3 [BugFix]: Fix sglang.bench_one_batch (#16925) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-01-16 13:15:34 -08:00
Baizhou Zhang
a04675892e Update flashinfer to 0.6.1 (#15551) 2026-01-17 00:48:30 +08:00
Yongfei Xu
82a1b645ba [DeepSeek V3.1/V3.2] Optimize fused moe configs for H20 & H20-3E based on swapab (#17133) 2026-01-17 00:10:52 +08:00
billishyahao
6f10e17b4a [bugfix] fix qwen3-next alt_stream none issue (#17016) 2026-01-16 22:40:25 +08:00
Nicolas Castet
9d8bbd4223 Add clear error message when OOM with symmetric memory (#17038) 2026-01-16 20:44:37 +08:00
Raghav Ravishankar
daea51385d Add AFMoE model implementation (#13216) 2026-01-16 20:35:42 +08:00
StonyPort
3355b6e21b feat: add request queued timeout (#17143) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-01-16 17:55:09 +08:00
R0CKSTAR
a1dd3d48ac [diffusion] hardware: support diffusion (single GPU, 3/N) (#17105) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-01-16 17:01:09 +08:00
Zhiqiang Xie
d9ed80b9f1 fix AMD CI failure of NUMA binding (#17184) 2026-01-16 00:21:04 -08:00
Adarsh Shirawalmath
7c39ea68f3 [diffusion] model: support flux Klein (#17173) 2026-01-16 16:16:17 +08:00
YAMY
daa4841e86 [ConfigArgumentMerger] Improve ConfigArgumentMerger compatibility with external callers (#17051) 2026-01-16 15:32:42 +08:00
Hudson Xing
21ee597e4a ci: enable offline mode when local cache is complete to avoid HF Hub … (#16121) 2026-01-15 20:15:33 -08:00
Xiaoyu Zhang
6ee970a365 [Diffusion] Hot fix broken output_path default value (#17180) 2026-01-16 12:14:09 +08:00
shuwenn
8ec160ed46 feature: support uvicorn access log filter(disable logging /metrics) (#15513) 2026-01-15 20:00:06 -08:00
YAMY
2740ed1ae7 [eval] GSM8k support for run_eval (#17041) 2026-01-16 11:10:17 +08:00
Lianmin Zheng
e7dc85c50b Fix grammar sync across TP ranks (#17100) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-15 18:38:01 -08:00
Ratish P
c81bad1bf7 [diffusion] feat: add cloud storage support for API (#14579) 2026-01-16 10:29:38 +08:00
hlu1
0e86de7c0b Remove deepseek-r1 from THINKING_MODE_CHOICES in run_eval.py (#17178) 2026-01-15 16:53:06 -08:00
PiteXChen
8b22deef5b fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 15:14:38 -08:00
JinYan Su
72e2f70ef7 feat(hicache): support numa detect to reduce long tail latency (#11028) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 14:11:49 -08:00
huangtingwei
77fc4c4a53 Add mooncake store read/write bandwidth logs (#10598) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Lei Huang <lei.huang@novita.ai>
 2026-01-15 12:15:41 -08:00
cctry
dda35ccbd8 Fix gid calculation in per_tensor_absmax_kernel (#17126) 2026-01-15 23:21:41 +08:00
wxy
d11e2dc6f4 [diffusion] chore: improve the output_path config and enable the server to return inference duration (#16965) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-01-15 22:31:50 +08:00
Mick
16831ab6d7 [diffusion] fix: fix using upstream flash_attn on blackwell (#17111) 2026-01-15 22:30:48 +08:00
R0CKSTAR
c9a45b7e3c [diffusion] fix: fix UMA detection (#17113) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-01-15 22:29:46 +08:00
HuangJi
e7df8bdc5c [diffusion] refactor: move SLA to attention_backend folder (#17020) 2026-01-15 21:36:48 +08:00
Lancer
e997995037 [diffusion] fix: optimize text encoder CPU offload initialization to address OOM (#17064) 
Signed-off-by: Lancer <maruxiang6688@gmail.com>
Co-authored-by: Lancer <maruxiang6688@gmail.com>
 2026-01-15 21:28:57 +08:00
Mick
68e8d0f68d [diffusion] CI: add testcase for cfg parallel (#17056) 2026-01-15 13:13:31 +08:00
sglang-bot
000ad42225 chore: bump sgl-kernel version to 0.3.21 (#17075) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-01-15 12:41:17 +08:00
Hanming Lu
9d5f16d456 [SWA] fix swa radix cache match_len_since_tombstone update when hits swa_tombstone (#17061) 2026-01-15 11:28:33 +08:00
Ke Bao
7f8a58fffb Refactor prefix cache type checking (#17028) 2026-01-15 11:28:13 +08:00