kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-12 02:16:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,294 Commits 157 Branches 0 Tags
a6ea3add768ff4e26ad1c40b06b49ca75f41a574
Commit Graph

4198 Commits

Author SHA1 Message Date
Yineng Zhang
a6ea3add76 [Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) (#12235) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-10-27 17:21:08 -07:00
fzyzcjy
326c84c493 Compiling rope while preserving true on policy (#12161) 2025-10-28 08:02:17 +08:00
gongwei-130
8da608cce0 fix: AttributeError: 'NixlKVManager' object has no attribute 'prefill_tp_size_table' (#12234) 2025-10-27 15:57:33 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Chunyuan WU
c11b34d599 rope xpu: fix missing argument 'fused_set_kv_buffer_arg' and replace native with sgl_kernel_xpu impl (#12006) 2025-10-28 01:02:18 +08:00
ykcombat
05ad28f25e [Feature] PD-Multiplexing Context and Scheduler. (#11592) 2025-10-28 00:54:43 +08:00
pansicheng
0cae873fcd check_offload_progress more frequently (#11656) 2025-10-28 00:37:38 +08:00
Haichao Zhu
a8b91f6b2d improve mimax-m2 rmsnorm precision (#12186) 2025-10-28 00:01:42 +08:00
Jimmy
959d1ab84b fix(metrics): double times add_latency for DECODE_BOOTSTRAP (#12209) 2025-10-27 23:59:48 +08:00
Muqi Li
6c1c193308 [Detokenizer Manager] Cleanup state when reqs are finished (#12205) 2025-10-27 23:59:09 +08:00
cctry
3029d30189 Fix crash after flush cache (#12107) 
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
 2025-10-27 23:52:27 +08:00
Yuan Luo
f389f01714 Optimize triton_mrope with torch compile (#12112) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-27 23:49:22 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
Yuxuan Zhang
a88b006ecf GLM-4-0414 and GLM-4.1V Code Refactor (#12117) 2025-10-27 16:57:07 +08:00
sglang-bot
55d75e11bd chore: bump SGLang version to 0.5.4.post1 (#12169) 2025-10-27 09:35:20 +08:00
Chang Su
94aad0de99 [misc][grpc] Remove duplicate log (#12168) 2025-10-26 14:06:59 -07:00
赵晨阳
7ebc28f5d6 [WIP] support MiniMax M2 model (#12129) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Roger Young <42564206+rogeryoungh@users.noreply.github.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-10-26 13:58:54 -07:00
ash-sigh
0b3b3e9a69 transfer mrope_position_delta to device when first running (#11047) 2025-10-26 13:06:09 -07:00
Elfie Guo
a1d5bc4cce Avoid using flashinfer_allreduce_fusion when dp attention is enabled. (#11632) 2025-10-26 12:31:14 -07:00
Zijian Zhang
a8023891f6 model: support NVILA and NVILA Lite (#10399) 2025-10-26 09:58:09 -07:00
fzyzcjy
0103f374ba Support DeepGEMM for deterministic inference (#12142) 2025-10-26 22:36:17 +08:00
zyksir
96a5a949f6 [Fix] fix allreduce bug in Piecewise Graph (#12106) 2025-10-26 21:15:48 +08:00
Liangsheng Yin
ea385ae85a Fix ITL metrics when using openai endpoint with spec (#12156) 2025-10-26 18:06:25 +08:00
Kai-Hsun Chen
6371f7af27 [quantization] AWQ Marlin doesn't work when dtype is bfloat16 (#11494) 
Signed-off-by: Kai-Hsun Chen <khchen@x.ai>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-10-26 15:49:45 +08:00
Liangsheng Yin
8491c794ad [misc] depdencies & enviroment flag (#12113) 2025-10-26 14:52:35 +08:00
Liangsheng Yin
bda3758fac [log] Make forward iter count optional (#12116) 2025-10-26 14:51:07 +08:00
Lianmin Zheng
7b36c47b3b Clean up attention backend selection code & Other minor rename (#12136) 2025-10-25 23:50:12 -07:00
Kaixi Hou
ff60406429 [NVIDIA] Change default quant method for model_opt (#11991) 2025-10-25 22:04:57 -07:00
fzyzcjy
c001deba37 Make bmm batch invariant injection optional (#12118) 2025-10-26 10:18:35 +08:00
Lianmin Zheng
8e70064c37 Clean up server launch code and multi tokenizer (#12132) 2025-10-25 16:40:27 -07:00
Baizhou Zhang
4b0ac1d52a Update sgl-kernel version to 0.3.16.post4 (#12125) 2025-10-25 14:33:33 -07:00
YAMY
c8492978a1 Fix Illegal Instruction/IMA errors when using DP attention -- num_tokens_for_logprob calculation (#12115) 2025-10-25 12:28:26 -07:00
Lianmin Zheng
4caca1ba04 Clean up server args & Add CI scripts (#12124) 2025-10-25 11:53:57 -07:00
Lianmin Zheng
ea13cb1452 [Auto Sync] Update test_deterministic.py, test_deterministi... (20251024) (#12083) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-10-24 22:28:01 -07:00
vipwangerxiao
8982418957 Fix 'KeyError' for per_token expert distribution recorder (#9501) 
Signed-off-by: Peng Wang <rocking@linux.alibaba.com>
Co-authored-by: Peng Wang <rocking@linux.alibaba.com>
 2025-10-25 03:28:50 +00:00
fzyzcjy
20bd2271e2 Support true on-policy (#12058) 2025-10-25 10:23:42 +08:00
Cheng Wan
649949807f [10/N] MoE Refactor: reorganize deepgemm runner in DeepEPMoE (#12054) 2025-10-24 19:16:17 -07:00
fzyzcjy
d7056c5236 Enhance tests in deterministic kernels (#12070) 2025-10-25 08:53:22 +08:00
Jinwu
13bf565d60 [2/N]Support DeepSeek-R1 w4a8 low latency deepep (#8464) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
Co-authored-by: Shangchuan Huang <2510421000@qq.com>
 2025-10-24 17:41:16 -07:00
yinghui
e51046beaa perf: trtllm_mla attention backend spec decoding speedup w/ cuda graph (#12093) 2025-10-24 16:05:44 -07:00
Minglei Zhu
f4b78d137c [1/2] deepseek deterministic: support deterministic inference for deepseek arch models on a single GPU (#12000) 2025-10-24 15:17:28 -07:00
Jonah Bernard
4b046a72d3 docs(server-arguments): add allowed options for each argument (#11560) 2025-10-24 11:49:20 -07:00
ishandhanani
14203432b4 fix(compile_utils, ep_moe): update environment variable and dtype check (#12034) 2025-10-24 11:00:12 -07:00
Yuanhang Sun
0bfa394aff [Fix]: HiCache hasher failed when EAGLE mode enabled (#12025) 2025-10-24 23:53:13 +08:00
fzyzcjy
e04340bf48 Fix multi processing serializer bug (#11958) 2025-10-24 22:53:45 +08:00
Xiaoyu Zhang
8470133852 [b200] fix piecewise cuda graph launch bug (#12067) 2025-10-24 22:36:39 +08:00
Muqi Li
93ef9a094d [Profiler] expand '~' for torch_profiler_output_dir (#11999) 2025-10-24 17:20:46 +08:00
Muqi Li
b04cd3d487 Add 'gguf' to project dependencies (#12046) 2025-10-24 17:16:19 +08:00
Yuan Luo
7ef5d8afd4 Revise POINTSV15Chat model (#12049) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-24 17:09:45 +08:00
Qiaolin Yu
71d41212e4 Fix dpsk-r1-fp4 launching crash (#12063) 2025-10-24 17:04:50 +08:00