kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-11 01:36:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,605 Commits 157 Branches 0 Tags
c67fce160e29e3d4741db84c67445a4a88d6f676
Commit Graph

4403 Commits

Author SHA1 Message Date
Chen1022
c67fce160e Fix missing dp_max_padding argument in set_dp_buffer_len (#12812) 2025-11-07 21:27:43 +08:00
hlu1
bef37d6de8 [Deepseek V3.2] Only skip Indexer logits computation when is_extend_without_speculative (#12816) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-11-07 01:34:04 -08:00
Jonah Bernard
bc25ea6762 [MoE] Add Comprehensive MoE Integration Tests (#12090) 2025-11-07 00:34:46 -08:00
elvischenv
1fa788ec14 [Bugfix] Fix illegal memory access (#12758) 2025-11-06 21:59:57 -08:00
Johnsonms
125f76ea44 [Test] Add DeepSeekV3.2 NSA Indexer Test Suite (#12520) 2025-11-06 21:28:43 -08:00
Yuhao Yang
d8736c756a fix multimodal gen issues (#12765) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-07 13:19:49 +08:00
Tianyu Guo
0ee5ab5a74 Add PP support for dots_vlm (#12763) 2025-11-06 19:14:27 -08:00
hzh0425
ed5e905c89 [HotFix]: Add missing SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL env var (#12776) 2025-11-06 18:58:44 -08:00
hzh0425
d9c812d881 [hotfix]: Resolve ModuleNotFoundError in PD deployment for is_in_ci() (#12772) 2025-11-06 18:48:03 -08:00
YAMY
7257525cce [DeepSeek-V3.2][NSA] Enable MHA Pathway for Short Sequence Prefill on B200 (SM100) (#12788) 2025-11-06 17:26:08 -08:00
Gaurav Verma
34ba10efd6 [AMD] Update wave-lang to 3.8.2 (#12576) 
Signed-off-by: xintin <gaurav.verma@amd.com>
 2025-11-06 15:51:44 -08:00
Rain Jiang
a119363f08 ignore the deepgemm check when the model weight with nvfp4 and moe ba… (#12782) 2025-11-06 15:27:28 -08:00
Nicolas Castet
3a64844a1c Fix hang in deepgemm compilation with symmetric memory enabled (#12715) 2025-11-06 14:06:22 -08:00
Lee Nau
b0d1c21d03 [fix] Only enable flashinfer all reduce fusion by default for single-node servers (#12724) 2025-11-06 11:53:58 -08:00
R0CKSTAR
b07c5e4080 Pin uvloop to 0.21.0 (#12279) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2025-11-07 03:33:31 +08:00
jianan-gu
6e6009fb6b [CPU] Fix TP padding case with weight block size (#8243) 2025-11-07 03:24:48 +08:00
Jimmy
88a2a34041 [Fix]HTTP Stream raise exception (#11904) 2025-11-07 03:23:12 +08:00
Teng Ma
78c58621d2 [mem pool] bugfix: wrong position for self.device in Mamba (#12684) 2025-11-07 03:05:42 +08:00
gaopengff
4e234b4cf9 [Intel XPU]Update pytorch xpu to 2.9 (#12363) 2025-11-06 10:34:23 -08:00
gaopengff
4cc725ac1c [Intel]Add 'intel_xpu' attention backend for llama4 (#11051) 2025-11-06 10:33:41 -08:00
sglang-bot
0c006b8809 chore: bump SGLang version to 0.5.5 (#12739) 2025-11-07 00:46:19 +08:00
Ke Bao
cd135bfe30 Update dsv3 quantization auto setting for sm100 (#12778) 2025-11-06 22:39:13 +08:00
jiapingW
0da30dbc20 fix qwen3-omni audio length < 30s (#12674) 
Co-authored-by: canghua <canghua.wjp@alibaba-inc.com>
 2025-11-06 00:05:28 -08:00
yinghui
58095cb00a Add timing metrics for requests (#12646) 
Co-authored-by: Scott Lee <scottjlee@users.noreply.github.com>
 2025-11-05 23:07:16 -08:00
Yuan Luo
fd3034da75 [VLM] Optimize qwen_vl preprocess_video (#12240) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
 2025-11-06 14:55:01 +08:00
Binyao Jiang
fbbe16faab [GDN] Fuse b.sigmoid(), fused_gdn_gating and unsqueeze into one kernel: up to 0.85% e2e speedup (#12508) 2025-11-05 22:52:04 -08:00
Yi Zhang
6a1a64fa41 [BUGFIX] fix output_ids in abort (#12737) 2025-11-05 22:51:47 -08:00
Yi Zhang
e2715cf871 fix mamba prefix cache leak caused by abort (#12693) 2025-11-05 22:51:03 -08:00
Mick
73e9a2ef5c fix: tiny fix cli (#12744) 2025-11-06 14:45:50 +08:00
Atream
2104d20eba Temporarily fix missing routed_scaling_factor for CompressedTensorsWNA16MoEMethod (#12738) 2025-11-06 12:01:03 +08:00
YAMY
f235498eca DeepSeek-V3.2: Add Adaptive MHA Attention Pathway for Short-Sequence Prefill (#11892) 2025-11-05 19:33:26 -08:00
gongwei-130
97be66c358 fix sgl-kernel version (#12723) 2025-11-05 19:01:03 -08:00
Lianmin Zheng
c7d57d5bb3 Fix CI and style (#12658) 2025-11-05 15:08:15 -08:00
Kaixi Hou
141278048e [NVIDIA] Fix unit test of MoE and add it to nightly ci (#12709) 2025-11-05 14:33:18 -08:00
Shu Wang
82f39dc11d Add mm_fp4 trtllm backend (#12406) 2025-11-05 14:31:46 -08:00
Atream
627bac649c Support Expert Deferral Mechanism in KTransformers (#12586) 
Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com>
Co-authored-by: chenht2022 <cht22@mails.tsinghua.edu.cn>
 2025-11-05 13:41:52 -08:00
Morpheus Guo
c8547ecddd Enable Aiter Attention for VL model (#12699) 
Co-authored-by: yuechguo <yuechguo@amd.com>
 2025-11-05 13:01:23 -08:00
Mick
7bc1dae095 WIP: initial multimodal-gen support (#12484) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: JiLi <leege233@gmail.com>
Co-authored-by: CHEN Xi <78632976+RubiaCx@users.noreply.github.com>
Co-authored-by: laixin <xielx@shanghaitech.edu.cn>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: jzhang38 <a1286225768@gmail.com>
Co-authored-by: BrianChen1129 <yongqichcd@gmail.com>
Co-authored-by: Kevin Lin <42618777+kevin314@users.noreply.github.com>
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
Co-authored-by: rlsu9 <r3su@ucsd.edu>
Co-authored-by: Jinzhe Pan <48981407+eigensystem@users.noreply.github.com>
Co-authored-by: foreverpiano <pianoqwz@qq.com>
Co-authored-by: RandNMR73 <notomatthew31@gmail.com>
Co-authored-by: PorridgeSwim <yz3883@columbia.edu>
Co-authored-by: Jiali Chen <90408393+gary-chenjl@users.noreply.github.com>
 2025-11-05 12:28:52 -08:00
Lianmin Zheng
fb2e816e83 Fix server args for gpt oss so users can override the moe runner backend (#12696) 2025-11-05 11:36:59 -08:00
bigmoyan
508d2f7aa2 add Kimi k2 reasoning parser (#12702) 
Signed-off-by: wangzhengtao <wangzhengtao@msh.team>
 2025-11-06 00:37:54 +08:00
Yuxuan Zhang
a889c85459 [Grammar Fix] GLM-4-MOE self.first_k_dense_replace is undefined. (#12455) 2025-11-06 00:03:45 +08:00
Yuhong Guo
4d84f886e7 Refactor --debug-tensor-dump-layers to list (#12691) 2025-11-05 03:30:01 -08:00
yinghui
dc4f541823 fix trtllm_mla attention backend when disabling cuda graph. (#12687) 2025-11-05 01:35:02 -08:00
zejunchen-zejun
0648eb482d [Profiler] Add SGLANG_PROFILE_RECORD_SHAPES for recording shapes when profiling (#11641) 
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-11-04 23:41:46 -08:00
yinghui
b88fab3111 fix: add seed bench_serving to cache key, remove redundant function definition. (#12680) 2025-11-04 23:39:11 -08:00
Glen Liu
cbf23dbbfa [Feature] add --lora-request-distribution arg to bench_serving.py and support skewed and distinct workloads (#12175) 2025-11-04 21:41:40 -08:00
ai-easy-cpu
48641435d6 fix typo of args description in sglang.profiler (#12486) 
Co-authored-by: AI-bot-easy <litchys0123@outlook.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-04 20:15:13 -08:00
Liangsheng Yin
44b1b394a4 [PD-Disagg] Check finish after pop tranferred (#12638) 2025-11-05 11:18:09 +08:00
Kaixi Hou
0711d1509b [NVIDIA] Fix cutedsl backend of MoE (#12353) 2025-11-04 18:54:55 -08:00
sglang-bot
09938e1f82 chore: bump SGLang version to 0.5.4.post3 (#12639) 2025-11-04 18:32:11 -08:00