kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 16:27:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,627 Commits 157 Branches 0 Tags
d874c8bba4c2d2e5d65c1b95e005bffff7dbf980
Commit Graph

5598 Commits

Author SHA1 Message Date
fzyzcjy
d874c8bba4 Tiny support http headers in bench serving (#16606) 2026-01-07 10:15:17 +08:00
fzyzcjy
9a21d89c5b Tiny add metrics for prefill delayer (#16603) 2026-01-07 09:53:52 +08:00
Kangyan-Zhou
18e2ef09d7 Add v1/models endpoint to diffusion model APIs so that they can be discovered by model gateway (#16425) 2026-01-06 14:28:24 -08:00
Alison Shao
0cbd8f3247 ci(test): migrate OpenAI server tests to registered CI system (#16326) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-01-06 11:09:37 -08:00
fzyzcjy
ed307a40bf Tiny add --gsp-ordered (#16575) 2026-01-06 22:43:52 +08:00
Ke Bao
02722b9113 Fix evict swa for overlap scheduler and page size > 1 (#16507) 2026-01-06 22:29:40 +08:00
fzyzcjy
f959250f76 Tiny support sglang_routing_keys_active in engine (#16570) 2026-01-06 22:10:10 +08:00
Junrong Lin
bc2f40bebc [test] Add mamba cache release/resume memory test (#14215) 2026-01-06 15:51:10 +08:00
lw9527
2724b1100b fix double Unicode escape issue in streaming tool_calls parameters (#13518) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-01-05 23:47:26 -08:00
Xiaoyu Zhang
5e5b1183ed [Diffusion] Ring Attention support sage backend (#16496) 2026-01-06 14:45:53 +08:00
jiapingW
1d7ad4afcc [diffusion] fix: fix bench_serving always use prompt fixed prompts (#16201) 
Co-authored-by: jiapingW <root@sgl-training-ray.datacrunch.io>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-01-06 14:39:07 +08:00
siyu
fba785c459 use aync load for encoder_server (#15456) 
Signed-off-by: liuanqi <liuanqi6@xiaomi.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: liuanqi <liuanqi6@xiaomi.com>
 2026-01-06 14:31:52 +08:00
Alison Shao
73398e22d6 ci: migrate VLM tests to test/registered/vlm/ (#16415) 2026-01-05 21:25:29 -08:00
Ke Bao
3aa11ca722 Remove hybrid_kvcache_ratio in server args (#16399) 2026-01-06 13:13:13 +08:00
Adarsh Shirawalmath
7be1a8c70c [diffusion] feat: support diffusers backend - run any model supported by diffusers (#14112) 2026-01-06 12:30:57 +08:00
Raayan Dhar
84d13c54bb feat: only add input vision tokens in bench_serving result if vision dataset is used (#15492) 
Signed-off-by: Raayan Dhar raayan.dhar@gmail.com <raayan.dhar@gmail.com>
 2026-01-06 12:14:42 +08:00
fzyzcjy
c105a3124b Support multi-round conversations in bench_serving (#6135) 2026-01-06 11:59:39 +08:00
Mick
ca922d4b05 [diffusion] feat: support warmup with resolutions (#16434) 2026-01-06 10:32:18 +08:00
zhangheng
2d02c150dc [3/N][Sparse With Hicache]: Init sparse coordinator (#16086) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-05 17:51:04 -08:00
ShirakSyouya
ce694b2b9e Fix the problem where Qwen3VL raises an "object has no attribute 'mod… (#15677) 2026-01-06 08:45:13 +08:00
Lianmin Zheng
6c0fb189f8 [Auto Sync] Update tokenizer_manager.py (20260105) (#16477) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Wangfan Fu <wangfan@x.ai>
 2026-01-05 14:06:03 -08:00
Binyao Jiang
9a9f996f8d [FP8] Fix weight_scale shape to match with x_scale shape for per-tensor quant under torch.compile (#16356) 2026-01-05 13:55:26 -08:00
Yi Zhang
a3914e3b3f [Bugfix] fix some memory computation bugs for qwen3next with mtp (#16138) 2026-01-05 23:24:21 +08:00
Raayan Dhar
4397cda7dc feat: raise error in PD when page sizes are mismatched (#14474) 
Signed-off-by: Raayan Dhar raayan.dhar@gmail.com <raayan.dhar@gmail.com>
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-01-05 19:54:01 +08:00
Lianmin Zheng
c35eb0fdee [Auto Sync] Update loader.py (20260105) (#16478) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: xiuyuli-xai <xiuyu@x.ai>
 2026-01-05 01:34:30 -08:00
fzyzcjy
7f35c46efb Tiny add sglang:http_requests_active metric (#16479) 2026-01-05 17:20:31 +08:00
DarkSharpness
da2f8cc33f [minor] reduce 1 unnecessary add (#16474) 2026-01-05 01:00:56 -08:00
fzyzcjy
4d737db857 Tiny fix prefill delayer not support non-fcfs schedule policy (#16471) 2026-01-05 16:28:54 +08:00
fzyzcjy
9d6029fb92 Fix TokenizerManager bottleneck for offline generation (#16456) 2026-01-05 16:28:25 +08:00
Qi Yuhang
dcfb92ddbc [diffusion] fix: fix using the sub-optimal fa kernel (#16382) 2026-01-05 16:25:41 +08:00
Netanel Haber
bebd625ba1 EVS Framework: Support NemotronH_Nano_VL_V2 (#14051) 2026-01-05 16:18:07 +08:00
Lianmin Zheng
012dc5866d [Auto Sync] Update scheduler.py (20260104) (#16424) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
Co-authored-by: Jie Ren <no-replay@github.com>
 2026-01-04 23:32:56 -08:00
Yuhao Yang
10174e1114 Revert "[grpc] update api to scheduler in grpc request manager" (#16387) 2026-01-04 22:05:39 -08:00
Yuhao Yang
2138ff48c6 Revert "[FEAT] optimize tensor zmq transfer for multimodal inputs" (#16386) 2026-01-04 22:05:26 -08:00
Xiaoyu Zhang
520c048d55 [diffusion] CI: add script for automatically generation ci perf baseline (#16389) 2026-01-05 13:18:35 +08:00
yudian0504
561a3e04f6 remove redundant mamba_cache clearing actions (#16180) 2026-01-05 13:02:07 +08:00
YAMY
12cde0df99 [SPEC_V2] Fix Acclen drop when enabling DP Attention for Spec-Overlap (#16310) 2026-01-05 11:39:23 +08:00
Liangsheng Yin
a7fd810842 Allow editable install without .git with add fallback version in pyproject.toml (#16435) 2026-01-05 11:17:20 +08:00
Kangyan-Zhou
ca80c19b55 Revert "[diffusion] feat: support warmup with resolutions" (#16433) 2026-01-04 18:44:05 -08:00
fzyzcjy
1e7b326482 Super tiny fix main code (#16432) 2026-01-05 10:34:25 +08:00
CHEN Xi
e267ca0beb [diffusion] doc: document LoRA support in CLI (#16375) 2026-01-05 10:20:30 +08:00
Mick
9a8ba3c189 [diffusion] feat: support warmup with resolutions (#16330) 2026-01-05 10:16:26 +08:00
Xiaoyu Zhang
0fee6bc632 [JIT kernel] Apply jit per_tensor_quant_fp8 kernel (#15836) 2026-01-05 10:15:00 +08:00
fzyzcjy
12df16607b Tiny speed up kimi detokenizer by 10x (#16427) 2026-01-05 09:12:05 +08:00
Douglas Yang
87699d48eb fix: only publish trace from tp 0 (#16411) 2026-01-04 12:17:08 -08:00
Alison Shao
52c604342c chore: print test list at beginning and end of run_suite.py (#16334) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-01-04 11:52:52 -08:00
Ke Bao
76bc07a335 Move swa memory pool to a seperate file (#16347) 2026-01-04 22:39:30 +08:00
Ke Bao
b328cd20bb Skip local attn init metadata for mimo swa model (#16349) 2026-01-04 22:38:36 +08:00
Douglas Yang
16e00651dd fix: health check adding test delays (#16381) 2026-01-04 00:28:32 -08:00
fzyzcjy
1b2b95d8d1 Super tiny cleanup unused function (#16376) 2026-01-04 15:08:12 +08:00