kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-11 01:36:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,688 Commits 157 Branches 0 Tags
661c1c97ad445a14e18f492155ff50e2aeb72113
Commit Graph

4450 Commits

Author SHA1 Message Date
sogalin
661c1c97ad Add pre-suffle weight for new aiter MoE support. (#12908) 
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Hubert Lu <55214931+hubertlu-tw@users.noreply.github.com>
 2025-11-10 11:42:55 -08:00
Liangsheng Yin
1086473111 Enhance retract test (page cases, long output cases) (#12781) 2025-11-11 03:03:26 +08:00
Liangsheng Yin
665416f6dd Unify memory management across (overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) (#12224) 2025-11-11 02:56:22 +08:00
Liangsheng Yin
f1f4c451ab Add process_prefill_chunk back to fix PP event loop (#13009) 2025-11-11 00:51:36 +08:00
fzyzcjy
b0ee99dd03 Super tiny fix typo (#13001) 2025-11-11 00:47:45 +08:00
Ke Bao
58b12ccb46 Support piecewise cuda graph for deepseek v3 (#12996) 2025-11-10 23:18:03 +08:00
sglang-bot
37c40a87a8 chore: bump sgl-kernel version to 0.3.17 (#12966) 2025-11-10 21:50:58 +08:00
Yuhao Yang
1240ac13b8 vlm: fix tiny multimodal cache bug (#12984) 2025-11-10 21:30:19 +08:00
Mick
5639145fac diffusion: reduce effort of supporting new model (#12982) 2025-11-10 21:20:33 +08:00
ShawnKung
afee2843d5 feat(metrics): add scheduler and hiradix cache metrics (#10218) (#10225) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-11-10 18:14:47 +08:00
Zhihao Zhang
6f08488042 fix missing output_token_logprobs when using ngram speculative decoding (#10702) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-10 02:04:42 -08:00
Lianmin Zheng
9ea2c686c7 [Auto Sync] Update batch_invariant_ops.py (20251109) (#12916) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-10 01:51:39 -08:00
PiteXChen
e2a784ecda [RadixTree] Reduce Syscalls, Optimize Collection Filtering and Align with cpp (#12239) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
 2025-11-10 17:47:59 +08:00
Neelabh Sinha
7bffc5dc25 [Fix] Add validation for served model name to reserve : for LoRA adapter syntax (#12912) 
Co-authored-by: neelabhsinha <neelabhsinha97@gmail.com>
 2025-11-09 23:42:02 -08:00
Xuchun Shang
ac19ce7efb [PP] put pp assert in model runner (#12934) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-11-10 14:58:19 +08:00
Mick
95876d75cb chore: include a minimum image for vlms when warming-up (#9528) 2025-11-10 14:56:59 +08:00
Liangsheng Yin
dc8a5a1ce7 [Refactor / Style] Unify all event loops (except for PP) (#12959) 2025-11-10 14:49:12 +08:00
Mick
e123648b36 diffusion: fix wan-2.2-TI2V and support sp (#12926) 2025-11-10 14:37:57 +08:00
kk
90401cf7d2 Fix the run-time error when calling fused_rms_mxfp4_quant that change return output number (#12803) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-11-09 21:48:36 -08:00
Atream
9cfe78dd30 clean redundant code in previous PR (#12957) 2025-11-09 21:40:32 -08:00
Atream
ddd1440d0f Refactor KTransformers heterogeneous compute with unified GPU-quantization backend (#12834) 
Co-authored-by: Chen Hongtao <56470055+chenht2022@users.noreply.github.com>
Co-authored-by: chenht2022 <cht22@mails.tsinghua.edu.cn>
Co-authored-by: skqliao <skqliao@gmail.com>
Co-authored-by: ovowei <1913953267@qq.com>
 2025-11-09 21:06:32 -08:00
Edwin Gao
d1be60c3c5 [Refactor] rename set_index_k_and_scale_buffer to set_index_k_scale_b… (#12956) 2025-11-09 20:35:19 -08:00
MMuzzammil1
1f2a6c691b Bugfix: LMCache Connector with Sglang (#12946) 
Signed-off-by: Mohd Muzzammil <me.muzzammil@samsung.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-10 12:13:37 +08:00
Liangsheng Yin
61c7fe7aed Minor code cleanup / improvement for PREBUILT_EXTEND mode (#12948) 2025-11-10 11:59:54 +08:00
Ke Bao
db24d34603 Support piecewise cuda graph for MLA (#11812) 2025-11-10 09:13:48 +08:00
Liangsheng Yin
4e916f9840 [lint] tiny fix unimported packages. (#12927) 2025-11-10 00:59:10 +08:00
Liangsheng Yin
4f65a64666 Refactor / Unify event loop across PD-Disagg, Overlap, DP-Attn cases (#12839) 
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
 2025-11-10 00:42:50 +08:00
Mick
f5b3ccd9a5 feat: basic support for server-level multimodal cache (#10775) 2025-11-10 00:27:50 +08:00
Teng Ma
b8ac4fcb51 [PD] feat: refactor custom mem pool and add barex pd support (#12332) 2025-11-09 18:51:34 +08:00
Zijian Zhang
3633f8b0cf Add Jet-Nemotron (#12448) 2025-11-09 01:32:47 -08:00
Minglei Zhu
8a821af793 fallback to triton mm_persistent kernel when deepGemm fail (#12911) 2025-11-08 23:42:26 -08:00
Ke Bao
b142831a26 Fix empty server args in marlin moe test (#12904) 2025-11-09 13:30:47 +08:00
Kangyan-Zhou
d134096319 Add Deepseek models into nightly tests (#12865) 2025-11-09 13:17:49 +08:00
Qiaolin Yu
f290e8016e Revert "Fix spec decoding acc length for dpsk-r1-fp4 tp8" (#12900) 2025-11-08 20:42:42 -08:00
Qiaolin Yu
9299a62fcb Fix spec decoding acc length for dpsk-r1-fp4 tp8 (#12896) 2025-11-08 19:55:45 -08:00
Liangsheng Yin
49543be9da Tiny simplify can_run_dp_cuda_graph gather logic (#12891) 2025-11-09 11:15:36 +08:00
alisonshao
d3a03aeef8 Refs/heads/add nightly test multi gpu configs (#12870) 2025-11-08 15:14:50 -08:00
Baizhou Zhang
5f02b918ec [Fix] Fix trtllm-mla backend when chunked prefix cache is disabled (#12361) 2025-11-08 15:10:25 -08:00
b8zhong
49653c8896 use fast stream instead of torch.cuda.current_stream in llama 4 shared experts overlap (#12811) 2025-11-08 15:04:37 -08:00
fzyzcjy
2b6c4257a0 Fix sending all requests to the first rank in DP attention (#12832) 2025-11-09 00:18:53 +08:00
Liangsheng Yin
243ea585fc [DP-Attn] Clarify MLP sync / idle batch preparation logic (#12843) 2025-11-08 23:23:14 +08:00
Charles Chen
f1a9c72de3 Support capturing aux_hidden_states for minimax m2. (#12798) 2025-11-08 01:54:22 -08:00
Lianmin Zheng
0296f1cdad [Auto Sync] Update activation.py, logits_processor.py, rota... (20251107) (#12853) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-07 22:07:51 -08:00
Rain Jiang
0f76976c3c remove the fa4 page_size hardcode to 128 restriction on mla model arch (#12801) 2025-11-07 13:30:30 -08:00
Lianmin Zheng
ae62279071 Fix data parallel controller launch for num nodes > 2 (#12822) 2025-11-07 12:13:21 -08:00
Mick
e316bcacb1 update multimodal_gen readme (#12825) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-11-07 09:04:44 -08:00
Yang Kaiyong
61bfd9fa9b optm(checkpoint-engine): disable multi-thread loading when update weights (#12374) 
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
 2025-11-07 21:30:04 +08:00
Chen1022
c67fce160e Fix missing dp_max_padding argument in set_dp_buffer_len (#12812) 2025-11-07 21:27:43 +08:00
hlu1
bef37d6de8 [Deepseek V3.2] Only skip Indexer logits computation when is_extend_without_speculative (#12816) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-11-07 01:34:04 -08:00
Jonah Bernard
bc25ea6762 [MoE] Add Comprehensive MoE Integration Tests (#12090) 2025-11-07 00:34:46 -08:00