kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-12 02:16:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,241 Commits 157 Branches 0 Tags
9305f0e58dca327bbb3dbd7622405e64d31d4449
Commit Graph

6724 Commits

Author SHA1 Message Date
Brayden Zhong
9305f0e58d Support triton_kernels for GPT-OSS on SM120 (#19718) 
Co-authored-by: amittell 1388680+amittell@users.noreply.github.com
 2026-03-03 14:14:01 -08:00
Sam (Kesen Li)
5b2e2750b5 Enable XQA for SM90 and SM120 (#17115) 
Co-authored-by: Xiaowei Wang <100599594+xiaoweiw-nv@users.noreply.github.com>
 2026-03-03 14:09:44 -08:00
Kangyan-Zhou
dc92f88a21 Enhance bench_multiturn.py with OpenAI API support and richer metrics (#19724) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-03 13:48:04 -08:00
Guy Stone
f749802402 [Score API][18132] return token usage in Score API response (#18381) 2026-03-03 13:45:35 -08:00
almaslof
b0f26698f5 feat(benchmark script): add similar to vllm --ready-check-timeout-sec parameter (#15466) 2026-03-03 13:44:38 -08:00
Karthik Koralla
85ab6a7f54 cli: Add lazy imports and fail-fast config validation (RFC #9853) (#19368) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-03-03 13:03:49 -08:00
xrwang8
cedb86a950 Feature:Reserve HTTP server port before model loading to immediately detect port conflicts instead of failing after several minutes of model loading. (#17754) 
Signed-off-by: xrwang8 <xrwang8@gmail.com>
 2026-03-03 11:59:18 -08:00
doujiang24
2e1b9e2547 Fix routed_dp_rank boundary validation (#19762) 
Signed-off-by: doujiang24 <doujiang24@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-03-03 11:55:15 -08:00
yefei12
85f7a0aa30 feat: support Kimi K2.5 for Eagle3 (#19689) 
Co-authored-by: chenyefei.cyf <chenyefei.cyf@U-9V5T77LW-2356.local>
Co-authored-by: GeLee-Q <865038696@qq.com>
Co-authored-by: Gao016 <yngao016@163.com>
Co-authored-by: sxl1993 <1218197792@qq.com>
 2026-03-03 13:41:15 -05:00
xutizhou
c6377bbbca feat(gdn): add FlashInfer K-last SSM layout support for GDN prefill and decode for Hopper (#18361) 
Co-authored-by: HongliMi <106042350+HongliMi@users.noreply.github.com>
Co-authored-by: xiaozhoupy <181108106+zhou9402@users.noreply.github.com>
Co-authored-by: Jinyan Chen <93358689+liz-badada@users.noreply.github.com>
Co-authored-by: Avery Yingyi Huang <averyh@nvidia.com>
Co-authored-by: eigen <52445717+yyihuang@users.noreply.github.com>
 2026-03-03 20:30:48 +08:00
Jasonzhang517
d939e26585 [model gateway][0/N] router EPD support: add encoder grpc server backend support (#16552) 
Co-authored-by: Zongyao Chen <ZongYao.Chen@linux.alibaba.com>
Co-authored-by: Zongyao Chen <solar1s@163.com>
 2026-03-03 19:38:15 +08:00
Shangming Cai
facde4c6d3 [PD] Enable all CP ranks for KVCache transfer (#19765) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-03 19:35:21 +08:00
shengzhaotian
365ca1edb5 [NPU] bugs fix: fix a condition bug when using speculative inference on Qwen3 and Qwen3 moe (#19532) 2026-03-03 17:59:25 +08:00
Muqi Li
666caaf9ce [Tool Call] Stream DeepSeek-V3.2 function call parameters in JSON format. (#16091) 
Co-authored-by: Huixxi <uestc.hugo@gmail.com>
 2026-03-03 01:46:29 -08:00
Shaun Kotek
4c95953b77 Fix/nemotron mtp quantaized (#19433) 2026-03-03 01:07:46 -08:00
Charles Chen
af0d35b224 Fix: Reject requests with a duplicate request ID which can cause server crash/hang (#19035) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-03-03 00:33:25 -08:00
Muqi Li
6af0448cc9 [Bugfix] Catch errors when DeepSeek-V3.2 generates malformed JSON (#18174) 2026-03-03 00:10:07 -08:00
Liangsheng Yin
7a2d3df96f Apply default stream to priority 0 in scheduling. (#16438) 2026-03-03 00:05:27 -08:00
Zack Yu
07b8d763ef feat: Add FP8 KV cache support for Triton attention backend (#18882) 2026-03-02 23:38:34 -08:00
赵晨阳
62480ebb1b [SGLang-Diffusion] Fix custom op fake impl missing eps default for torch.compile (#19725) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-03-03 15:24:36 +08:00
1StepForever
3c01b44700 [Fix] NPU deepep hccl buffer and fix IPC safe check (#17804) 2026-03-03 14:56:06 +08:00
Xinyuan Tong
dbf1247fe0 Add KimiK2Detector with tool interruption support (#19696) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-03-03 14:04:49 +08:00
Yuzhen Zhou
63003a39cf [BUG] Support tuple hidden_states from fused MXFP4/FP8 quantization (#19643) 2026-03-02 20:39:06 -08:00
Alison Shao
fe9d85d93c Fix CompressedTensorsMxInt4MoE abstract method and relax GPQA baseline (#19726) 
Co-authored-by: Alison Shao <alisonshao@Mac.attlocal.net>
 2026-03-02 19:03:21 -08:00
huangtingwei
8dfb6e1684 [HiCache] fix compatibility bugs with eagle and HiCacheStorage (#19570) 
Co-authored-by: maoyuhan <susanmao1997@outlook.com>
 2026-03-02 18:29:51 -08:00
KnightLTC
1041f240c0 [NPU]grok2 model support (#17119) 
Co-authored-by: cy <chenyang08056032@163.com>
 2026-03-03 10:24:10 +08:00
Ratish P
e6e02ec938 [diffusion]: Add model detectors and warning for quantized diffusion models (#18041) 2026-03-03 09:46:25 +08:00
Xiaoyu Zhang
145ae518ac [Diffusion] Revert 18619 (#19510) 2026-03-03 08:15:15 +08:00
Mohammad Miadh Angkad
6822941514 [FlashInfer] Bump FlashInfer version from 0.6.3 to 0.6.4 (#19005) 2026-03-02 16:12:09 -08:00
Mohammad Miadh Angkad
3f9fc8b848 [Qwen3.5] Fix missing quant_config in Qwen3VL (#19291) 2026-03-02 14:07:51 -08:00
Glen Liu
cc860a2198 [TestFix] change LoRA tests to use NVIDIA adapter instead of Nutanix (#19642) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-02 12:55:41 -08:00
Xiaoyu Zhang
51ee17ce44 [diffusion] move skills dir (#19697) 2026-03-03 02:51:29 +08:00
Makcum888e
05950853bc [Diffusion] [NPU] Add CI tests for FLUX (#19001) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-03-02 20:40:22 +03:00
Yuwei An
c64274c746 Piecewise Cuda Graph set default (#16331) 2026-03-02 23:18:07 +08:00
hlu1
468e3dc56b [Qwen3.5] Set full attn_backend to trtllm_mha on SM100 by default when possible (#19030) 2026-03-02 23:14:53 +08:00
0xNullPath
2d183c4e6d [Feat] add PP Support for minimax-m2 series (#19577) 2026-03-02 23:13:59 +08:00
Ruihang Li
5833ea684d [diffusion] fix: make input/output file save paths configurable and disableable (#19580) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-02 23:02:33 +08:00
Xiaoyu Zhang
53de53fb53 [jit_kernel] Tiny unify jit_kernel tests style (#19694) 2026-03-02 21:33:59 +08:00
Hexq0210
714c53d609 [NPU] support PD disaggregation on ascend when using PP (#14908) 
Co-authored-by: iridiumine <42236072+iridiumine@users.noreply.github.com>
 2026-03-02 21:33:16 +08:00
Bi Xue
eaf18ebe8d [sgl]add pin_mem to avoid cpu->gpu copy sync point (#19590) 2026-03-02 21:08:31 +08:00
JiaruiChang5268
b3718982a1 [Feature] add feature mla_ag_after_qlora for dsv3.2 (#19428) 
Co-authored-by: JiaruiChang5268 <changjiarui1@huawei.com>
 2026-03-02 20:00:31 +08:00
Shangming Cai
3f36f27eae [Bugfix] Fix nixl and mori backend for missing decode tp size in PD module (#19690) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-02 19:55:19 +08:00
AichenF
8df9b8dce9 [diffusion] fix: skip USP for cross-attention with replicated KV for wan (#19419) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-02 19:52:08 +08:00
Leoyzen
da2a0240f7 Add GLM45 tool interruption support (#17714) 
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-03-02 19:34:12 +08:00
fzyzcjy
7579ab3f33 Enhance error resilience in dump comparator (#19685) 2026-03-02 19:08:35 +08:00
fzyzcjy
e5ef845cad Support multiple verbosity in dump comparator (#19684) 2026-03-02 18:47:30 +08:00
fzyzcjy
3dd4649b42 Beautify text output in dump comparator (#19683) 2026-03-02 18:47:01 +08:00
fzyzcjy
5bf3deb4bc Trace execution information in dump comparator (#19682) 2026-03-02 18:46:27 +08:00
fzyzcjy
abdc0ee71f Support directory detection in dump comparator (#19680) 2026-03-02 18:45:35 +08:00
fzyzcjy
6980416149 Support non orthogonal parallel axes and explicit replication annotation in dump comparator (#19679) 2026-03-02 18:44:33 +08:00