kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 08:17:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,885 Commits 157 Branches 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

6437 Commits

Author SHA1 Message Date
Prozac614
3778c2fc6d [diffusion] CI: fix CI test case skip problem (#15874) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-26 19:42:20 +08:00
fzyzcjy
5d421db883 Tiny log warn users when tracing is automatically disabled (#15889) 2025-12-26 19:00:37 +08:00
shuwenn
fe3d47fc9d fix: warn once per env var key (#15846) 2025-12-26 18:41:15 +08:00
Yi Zhang
ef92b4eb88 [BUGFIX] fix edge case for qwen3-next (#14209) 2025-12-26 17:14:37 +08:00
Annis
73c0c66ff8 [diffusion] ZImage support Tensor Parallel (#15849) 2025-12-26 14:58:14 +08:00
chhnb
a1e9b4edfa diffusion: support Qwen-Image-Layered (#15817) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
 2025-12-26 14:57:01 +08:00
Liangsheng Yin
e75657c839 [Bug] fix piggyback load report return None bug (#15870) 2025-12-26 14:48:45 +08:00
Ke Bao
c28c536c91 Fix swa available memory check (#15867) 2025-12-26 13:19:01 +08:00
Yuan Luo
086813ae8a [VLM] refactor: refactor load_mm_data to improve performance (#14644) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-26 13:17:53 +08:00
Hexq0210
3fd232ad96 [NPU] update Mixed chunk op to FIA (#15518) 2025-12-26 12:17:40 +08:00
Liangsheng Yin
cb1812954a Introduce ModelRunnerKVCacheMixin to simplify the code. (#15821) 2025-12-26 11:17:33 +08:00
Kangyan-Zhou
a91e072f33 Use allow auto truncate in the OpenAI API endpoint (#15369) 2025-12-25 18:45:57 -08:00
fzyzcjy
0271fc3456 Fix prefill num tokens metrics (#15858) 2025-12-26 10:26:42 +08:00
fzyzcjy
68bece8caa Super tiny move last_prefill_tokens to metrics mixin (#15857) 2025-12-26 10:26:10 +08:00
Ke Bao
7b7e357f61 Separate swa and local attention chunk cache eviction (#15820) 2025-12-26 09:34:22 +08:00
Ke Bao
2f66b0671b Fix chunk_kda_fwd missing argument (#15851) 2025-12-26 09:32:35 +08:00
Alison Shao
4805118122 ci: add continue-on-error for scheduled PR tests (#15701) 2025-12-25 13:26:14 -08:00
Hudson Xing
9d878c1f3e Optimize FP8 MLA KV cache writes with Triton kernel (#15522) 2025-12-25 12:35:39 -08:00
Lianmin Zheng
8087ef126f Clean up the __init__ of TokenizerManager and DetokenizerManager (#15796) 2025-12-25 10:30:27 -08:00
Mick
a355911909 [diffusion] improve: improve post-processing by moving compute-intensive tasks to GPU (#15822) 2025-12-26 01:29:04 +08:00
Yuxuan Zhang
f3ba711662 fix: change class name of GLM-ASR (#15772) 2025-12-26 00:15:16 +08:00
Yuwei An
5c243ba588 Custom All Reduce for Piecewise Cuda Graph (#15356) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2025-12-25 23:55:15 +08:00
StonyPort
b6702d72cf feat: log request when e2e latency exceeds the specified value (#15759) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-12-25 23:12:35 +08:00
fzyzcjy
c12567274a Tiny change bench-serving to use routing key header (#15827) 2025-12-25 21:29:49 +08:00
Yingchun Lai
bb9e6cdf9e [MiMoV2Flash] fix: respect --swa-full-tokens-ratio arg (#15488) 2025-12-25 21:02:56 +08:00
roikoren755
de03b0cd30 [Nemotron 3 Nano] Add triton MoE configs (#15815) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-25 19:31:20 +08:00
Mick
2a8a785634 [diffusion] log: avoid logging in hot path if unnecessary (#15818) 2025-12-25 18:42:52 +08:00
Liangsheng Yin
f4e835af2f Cleanup ModelRunner (#15802) 2025-12-25 18:13:30 +08:00
Xiaoyu Zhang
e6ce16a4c2 [diffusion] feat: support TP for Flux.1.dev (#15666) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-25 18:00:44 +08:00
Xiaoyu Zhang
de2f2880b5 [JIT sgl-kernel] Jit support per tensor quant (#15709) 2025-12-25 16:24:37 +08:00
zhangheng
a89e85e739 [1/N][Sparse With Hicache]: Add Sparse Interface (#14741) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: MagicYang1573 <1328657938@qq.com>
 2025-12-25 00:12:47 -08:00
Ke Bao
cbf9f13493 Adjust server args for Mimo-v2-flash model (#15803) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-25 16:04:39 +08:00
Yi Zhang
eb3da9c1dd fuse ssm state store into chunk_gated_delta_rule_fwd_h (#15409) 2025-12-25 16:03:15 +08:00
Yuan Luo
b9af8d2eb9 [VLM] Support apply qk norm in multi cuda streams (#15720) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-25 14:35:07 +08:00
Prozac614
10a9573efd [diffusion] CI: add tests to validate the size, extension, and format of output images/videos. (#15736) 2025-12-25 14:00:22 +08:00
Prozac614
49ab72f837 [diffusion] fix: remove unused LoRA application logic from RowParallelLinearWithLoRA class in linear.py (#15801) 2025-12-25 13:43:38 +08:00
R0CKSTAR
8865424f14 [diffusion] refactor: refactor attention backend checking to use backend enum (#15555) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2025-12-25 11:51:40 +08:00
Liangsheng Yin
b311c43d13 Clarify None handling in sglang's environ (#15770) 2025-12-25 11:40:17 +08:00
Huaixin Chang
0c39730b18 DP: support piggyback server load report (#11469) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
 2025-12-25 11:35:05 +08:00
satyamk7054
38dd4fbb66 Add overlap scheduling for embeddings code path (#14032) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-12-24 18:24:18 -08:00
Lianmin Zheng
92ddc46824 [Auto Sync] Update server_args.py (20251223) (#15700) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-12-24 14:55:14 -08:00
Chenxi Li
17b38f88bc Add LoRA metrics for potential auto scaling (#15149) 2025-12-24 12:13:52 -08:00
Xinyuan Tong
ae434f7821 Move limit-mm-data-per-request to make code clean (#15775) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-24 10:59:39 -08:00
Douglas Yang
17e65466de fix: nightly fix b200 gpqa (#15745) 2025-12-24 10:29:45 -08:00
Simo Lin
b27b5a834e [bug] fix code formatting which blocks ci (#15780) 2025-12-24 09:14:13 -08:00
Mayyyy
2fb3160598 [NPU] Bug fix in device detect (#14137) 2025-12-25 00:18:39 +08:00
Liwansi
8bf7f240b6 [NPU]qwen3 pp bugfix (#15390) 2025-12-25 00:14:40 +08:00
Mick
2c5679f314 [diffusion] refactor: unify the profiling api for all executors (#15718) 2025-12-24 23:26:09 +08:00
Liangsheng Yin
159b128357 Tiny add flush for CI crash locating (#15769) 2025-12-24 22:47:10 +08:00
fzyzcjy
9338f63f7d Add request counter in addition to existing response counter (#15768) 2025-12-24 22:22:38 +08:00