kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-13 19:07:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,824 Commits 157 Branches 0 Tags
cfd49e233ceb94de3f702f70aef06e299836aede
Commit Graph

7855 Commits

Author SHA1 Message Date
fzyzcjy
a2960bdd6b Super tiny allow millisecond precision in logging (#14183) 2025-12-01 14:46:09 +08:00
fzyzcjy
487c8d4df3 Tiny add several args to bench serving (#14181) 2025-12-01 14:45:47 +08:00
fzyzcjy
f87b8eab23 Tiny fix transform_scale_ue8m0 wrong output in some scenarios (#14003) 2025-12-01 14:45:27 +08:00
Minglei Zhu
e8542db558 [piecewise] move piecewise_cuda_graph_runner init to model_runner initialize (#14034) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
 2025-11-30 22:16:04 -08:00
Lianmin Zheng
6df1e8d628 [Auto Sync] Update backend.py (20251130) (#14153) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-11-30 22:15:02 -08:00
qichu-yun
bd0e690857 [Feature] Enable PTPC FP8 for compressed tensors moe (aiter kernel) (#12181) 2025-11-30 21:54:28 -08:00
Byron Hsu
0825d7f4c6 [piecewise] Refactor VLM to support input embed buffer and remove external embedder hack (#14155) 2025-11-30 21:43:09 -08:00
Yuhao Yang
0b9dbea593 [diffusion] chore: improve z-image (#14104) 2025-12-01 12:26:17 +08:00
Uranus
982db4ebac Feat: GLM-4.6 supports shared experts fusion (#13873) 
Signed-off-by: UranusSeven <109661872+UranusSeven@users.noreply.github.com>
Co-authored-by: Kevin-XiongC <kevin_xiong1997@outlook.com>
Co-authored-by: Mingyi Jin <jinmingyi1998@sina.cn>
 2025-12-01 11:33:18 +08:00
Teng Ma
f5f3a5d98c [PD] Support json file configuration for Transfer Engine (#14059) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-01 10:47:33 +08:00
YAMY
decb48965d [DeepSeekV3.2] Enable pure TP & Partial DP Attention (#13646) 2025-11-30 15:59:23 -08:00
Fan Yin
c72f0756d2 Fix: fix flashmla fp8 kv cache acc error (#13841) 
Co-authored-by: ybyang <ybyang7@iflytek.com>
 2025-11-30 13:38:19 -08:00
Baizhou Zhang
f1115cf58d Revert "[Minor]Raise Error when deepep num dispatch token per rank is smaller than cuda graph bs" (#14171) 2025-11-30 12:49:46 -08:00
Baizhou Zhang
7b03cc6482 [Minor]Raise Error when deepep num dispatch token per rank is smaller than cuda graph bs (#14065) 2025-11-30 10:11:42 -08:00
Dongjoo Seo
c15c864b6f Fix LMCache unit test and init bug (#14005) 
Signed-off-by: DongDongJu <commisori28@gmail.com>
 2025-11-30 23:57:32 +08:00
PiteXChen
dc7bdc7329 bugfix[schedule]: Excessive preemption occurs when preempting running requests to schedule new prefill requests. (#12494) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
 2025-11-30 22:29:26 +08:00
Liangsheng Yin
0a9d64530d Support grammar + spec + reasoning (#14163) 2025-11-30 21:19:57 +08:00
fzyzcjy
340c613ab5 Support numactl bind for CPU and memory before process starts (#14156) 2025-11-30 17:00:33 +08:00
fzyzcjy
36b729c2b8 Implement profiler v2 and fix stage mixture bug (#14148) 2025-11-30 16:59:52 +08:00
Tianhao Zhou
67e6ef4b2d feat: longcat flash add aux layers capture for eagle3 (#14161) 2025-11-30 00:50:55 -08:00
strgrb
65ba5ab8b1 add cpp files for cpp_radix_tree to pyproject.toml. (#14052) 2025-11-30 13:05:04 +08:00
WenhaoZhang
990023e59b [diffusion] lora: Fix LoRA weight merging for torch.nn.Linear layers from diffusers modules (#14150) 
Co-authored-by: niehen6174 <niehen.6174@gmail.com>
 2025-11-30 12:44:12 +08:00
fzyzcjy
0ae4b1ad81 Show errors when misusing env variables (#14154) 2025-11-30 10:57:35 +08:00
fzyzcjy
94cd64a7b0 Support checking fp8 params in weight_checker (#14147) 2025-11-30 09:08:59 +08:00
fzyzcjy
b870271a50 Fix spec v2 does not support RL update weights from tensor (#14146) 2025-11-30 09:08:05 +08:00
fzyzcjy
22ee9b0111 Super tiny add more info in dumper (#14145) 2025-11-30 09:07:39 +08:00
fzyzcjy
9d0e5f1f74 Tiny fix DeepGEMM precompile rank check (#14136) 2025-11-30 09:07:17 +08:00
Kangyan-Zhou
1d3d8b3418 Fix Minimax M2 loading issue (#13956) 2025-11-29 17:07:19 -05:00
Lianmin Zheng
155a9e7237 Fix condition for streaming output_ids in tokenizer manager (#13759) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Chang Su <chang.s.su\n@oracle.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-29 13:56:15 -08:00
gongwei-130
3339c81072 fix RuntimeError: RMSNorm failed with error code an illegal memory access was encountered (#14135) 2025-11-29 12:17:41 -08:00
Yuhao Yang
f03ea34a3d add runtime check for PyTorch 2.9.1 + CuDNN < 9.15 to prevent Conv3d performance issues (#14119) 2025-11-29 10:05:54 -05:00
fzyzcjy
4cafc835d3 Super tiny fix typo (#14131) 2025-11-29 21:08:31 +08:00
Mick
c6a52f4411 [diffusion] chore: add resolution shortcuts for sampling params (#14129) 2025-11-29 18:00:21 +08:00
elvischenv
848ee57067 feat: support flashinfer kernel autotune (#12306) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2025-11-29 00:05:37 -08:00
Cheng Wan
0fe74af563 Remove incorrect deep_gemm assertions from server_args.py (#14113) 
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
 2025-11-28 20:25:39 -08:00
Mick
0a362d653f [diffusion] log: unify generation performance logging (#14117) 2025-11-29 12:21:59 +08:00
fjybiocs
143b57b805 enable piecewise cuda graph for prefill server (#13377) 
Co-authored-by: serverance.fu <serverance.fu@temu.com>
 2025-11-29 12:09:26 +08:00
Yan Ru Pei
f446b51c41 fix: malformed KV events for NVIDIA Dynamo (#13488) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-11-28 14:55:20 -08:00
Tomer Shmilovich
11b6217aee Fix NIXL OBJ desciptors (#10712) 
Co-authored-by: Tomer Shmilovich <tshmilovich@login-eos01.eos.clusters.nvidia.com>
 2025-11-28 11:32:07 -08:00
Yuhao Yang
841eb29d3d [diffusion] model: support z-image (#14067) 2025-11-28 21:48:31 +08:00
fzyzcjy
45cf575852 Fix overlap scheduler not take effect when outputing logprobs (#14096) 2025-11-28 18:15:56 +08:00
Mick
0e8ce1e832 [diffusion] refactor: clean useless files (#14094) 2025-11-28 18:14:00 +08:00
Lzhang-hub
ea1e9f6b3c feat: support qwen3_vl vision model dp (#13724) 2025-11-28 17:29:07 +08:00
Lzhang-hub
f6e37d3edb [Bugfix] qwen2.5-vl spec decode accept_len low (#13904) 
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
 2025-11-28 17:26:32 +08:00
vipwangerxiao
ab9a46d462 Support configuring the request limit per receiving poll (#14076) 
Co-authored-by: Peng Wang <peng_wang@linux.alibaba.com>
Co-authored-by: Feng Su <225349073+sufeng-buaa@users.noreply.github.com>
 2025-11-28 16:14:21 +08:00
shuwenn
621061f017 [Bugfix] input prompt was not logged (#13936) 2025-11-28 16:00:51 +08:00
Aleksandr Krotov
7daddcdb58 Fix structural_tag tool call with null schema (#14006) 2025-11-27 23:04:16 -08:00
Mick
951028968c [diffusion] refactor: refactor ComponentLoader and support loading native models from diffusers and transformers (#13205) 2025-11-28 14:17:32 +08:00
Mick
3543a04a48 [diffusion] refactor: refactor condition image resize logic (#14079) 2025-11-28 14:06:34 +08:00
fzyzcjy
21af8e73ad Super tiny add comments to SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK (#14048) 2025-11-27 22:16:43 +08:00