kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-09 08:47:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,908 Commits 157 Branches 0 Tags
3e4d431a44b37e2ffcd6f0bb7aecc2a4f8ac6f3a
Commit Graph

5160 Commits

Author SHA1 Message Date
ゆり
3e4d431a44 [Feature] Add AIME25 dataset support for SGLang simple_eval (#14990) 
Co-authored-by: zkexorability <zkexorability@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-12-15 21:59:40 -08:00
Yibo Cai
22587bc0b4 [BugFix] Fix CPU inference failure (#15231) 2025-12-15 21:32:41 -08:00
Muqi Li
1da5cd638f [Bugfix][Tool Call] Add null system prompt to support tool system prompt (#15092) 
Signed-off-by: Muqi Li <muqi1029@gmail.com>
 2025-12-15 19:40:10 -08:00
XDaoHong
4733fcff1f [Feature] npu support enable_torch_compile for torchair backend (#13410) 
Co-authored-by: ZhengdQin <zhengdqin@gmail.com>
 2025-12-16 09:23:51 +08:00
Liwansi
30da2f0598 [NPU][eagle3] support qwen eagle3 on NPU (#14820) 2025-12-16 02:25:13 +08:00
Xiaoyu Zhang
4901693110 [diffusion] perf: support FFN pack gate and up proj for Z-Image(#15201) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-16 01:18:47 +08:00
ratish
3d484be547 fix(attention): Prevent trtllm_mha auto-selection with eagle3 speculative decoding (#15127) 2025-12-15 16:42:26 +00:00
Xiaoyu Zhang
c0d94440b7 [diffusion] perf: support pack qkv for Z-Image (#15191) 2025-12-16 00:22:24 +08:00
Mick
1dedb63860 [diffusion] chore: minor code cleanups (#15190) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-15 23:57:02 +08:00
Xiaoyu Zhang
7bc8b1532e [diffusion] fix: fix AttributeError in _build_parallelism_config when accessing tp_group.device_group (#15196) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 23:41:06 +08:00
roikoren755
9003a4369d Add missing assertion in NemotronH path (#15193) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-15 06:17:21 -08:00
Mick
b098b1ae24 [diffusion] fix: fix video model sp when resolution is not specified (#15047) 
Co-authored-by: Brain97 <Brain97@users.noreply.github.com>
 2025-12-15 20:25:43 +08:00
Lancer
abd3e048f2 [diffusion] fix: fix pytorch non-writable array warning (#15017) 2025-12-15 20:18:54 +08:00
Xiaoyu Zhang
92c29d43ac [diffusion] fix: cache dit with parallel (#15163) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 19:15:51 +08:00
DarkSharpness
f03bfa4ce3 [Feature] Fuse mrope all in 1 kernel (#14906) 2025-12-15 02:50:55 -08:00
kk
2ea844ec81 Fused two elementwise kernels for k_nope and k_pe concat (#14862) 
1 TC failure to check, but irrelevant to this code change
 2025-12-15 01:33:33 -08:00
blzheng
d16ff357db [CPU] Add Gemma3RMSNorm kernel in sgl-kernel and add ut (#9324) 2025-12-15 00:24:02 -08:00
Yuhao Yang
01b955ac3d [diffusion] model: support mutli-image input and qwen-image-edit-2509 (#15005) 2025-12-15 16:17:10 +08:00
mmdbhs
16e6bc20b0 fix CompressedTensorsW8A8Int8 min_capability (#13914) 
Co-authored-by: Fan Yin <1106310035@qq.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
 2025-12-15 15:50:39 +08:00
Liangsheng Yin
372507649d Tiny improve summary text in bench_one_batch_server.py (#15158) 2025-12-15 14:59:11 +08:00
Lianmin Zheng
702426b06a Fix import warnings (#15144) 2025-12-14 21:25:18 -08:00
Douglas Yang
9e9a61691e ci: adding errors to Github summary (#14778) 2025-12-14 21:08:16 -08:00
Yuan Luo
3912ee4991 [VLM] feat: support chunked vit attention (#14907) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-15 12:11:02 +08:00
b8zhong
1ab9b8e0a3 Enable TRT AllReduce Fusion by default (#14764) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-14 20:01:49 -08:00
Hanming Lu
e61dabf5e4 [Qwen3-next] support mamba radix cache for overlap scheduler (#14792) 2025-12-14 18:54:16 -08:00
Xiaoyu Zhang
4513f549ee [diffusion] fix: fix default resolution 720p width from 1080 to 1280 (#15058) 2025-12-15 09:16:47 +08:00
YAMY
c96903074c [NSA] Fix NSA backend assertion error when running DeepSeek-V3.2 PP with radix-cache (#15086) 2025-12-14 17:13:18 -08:00
Hudson Xing
99cb2ed988 [refactor] Move trtllm_fp8_kv_kernel to triton_ops directory (#15044) 2025-12-14 15:22:37 -08:00
danielafrimi
8102e36b5d Add NanoV3 reasoning parser support (#15113) 2025-12-14 12:09:44 -08:00
roikoren755
3f0482174a Fix Mamba2-based models' default attention backend (#15117) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-14 11:08:48 -08:00
Ziming Huang
997ea57eaf Fix tensor mismatch error in sepc + topk > 1 + page_size > 1 (#14874) 2025-12-15 01:10:58 +08:00
Xiaoyu Zhang
64b5c3ab90 [diffusion] refactor: refactor fuse qkv with QKVParallelLinear linear (#15090) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 00:33:29 +08:00
Tianyu Guo
9acb21ae27 feat: support EPD disaggregation (#12263) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: Nicholas <45984215+liusy58@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2025-12-14 22:30:08 +08:00
Li Jinliang
6f0c77d7f8 [diffusion] app: support webui (#14961) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-14 21:07:36 +08:00
Xiaoyu Zhang
e3f51e823e [diffusion] feat: add support for additional sampling parameters in video generation API (#15062) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-14 19:44:03 +08:00
Xiaoyu Zhang
fdfabb7afc [diffusion] fix: tiny fix _templated_ring_attention bug (#15053) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-14 19:41:53 +08:00
blahblah
19c16748ce [diffusion] feat: support resolution check for video model (#14881) 
Co-authored-by: Brain97 <Brain97@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-14 17:50:13 +08:00
Liangsheng Yin
5c75907e62 Introduce native kv cache move (#15108) 2025-12-14 17:23:28 +08:00
liupeng374
4ea3642250 [NPU] bug fix for multi stream (#15048) 2025-12-14 16:38:12 +08:00
liupeng374
f50af32d8b [scheduler] remove scheduler allgather for best throughout (#14294) 2025-12-14 16:37:49 +08:00
Huang Lin
6abdf73f4d [Fix] Environment variable SGL_* is deprecated (#14943) 2025-12-13 19:55:43 -08:00
fzyzcjy
168a31eb00 Support prefill max requests limitation (#14993) 2025-12-14 10:52:10 +08:00
narutolhy
69cfb17b9a [Fix] avoid stream sync in _compute_mrope_positions (#14956) 2025-12-13 18:50:57 -08:00
fzyzcjy
fdc93b019c Add sglang:decode_sum_seq_lens metric (#15066) 2025-12-14 08:42:45 +08:00
Xinyuan Tong
96705514bd fix: dpskv32 chat history processing, default drop_thinking to true (#15064) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-13 12:16:04 -08:00
Baizhou Zhang
ab3ffd1c8e Add nightly accuracy test for DeepSeek V3.2 (#14935) 2025-12-13 12:11:16 -08:00
Yuhao Yang
a81cc1b8b3 add transformers version validation for glm-4.6v moe models (#14998) 2025-12-13 10:54:08 -08:00
liupeng374
3134d2b2a7 [scheduler] enhance scheduler in dp_attention mixed case with spec (#14201) 2025-12-14 02:52:26 +08:00
Yuhao Yang
06b58c5dc5 fix flaky image access in ci by switching to raw content url (#14940) 2025-12-13 10:52:06 -08:00
Simo Lin
5d9c6bac07 [bug] fix grpc secheduler launcher breaking change (#15080) 2025-12-13 10:36:49 -08:00