kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-10 17:27:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,885 Commits 157 Branches 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

6437 Commits

Author SHA1 Message Date
Praneth Paruchuri
a53f2d6c12 Support orion (#10665) 2025-11-15 03:08:32 +08:00
hlu1
a7002e614b [Deepseek V3.2] Clean up MTP (#13236) 2025-11-14 11:01:37 -08:00
Kshiteej K
84e151ac7f LLama4 Attention: Update assertion msg (#12777) 2025-11-15 02:52:50 +08:00
Yingchun Lai
875a25ddef refactor: remove duplicate function _get_bootstrap_info_from_server (#13277) 2025-11-15 02:48:17 +08:00
Vitaly Tuzov
fc55b45e5f Optimized prefill cache allocation for NPU (#13288) 2025-11-15 02:40:17 +08:00
Ling Zhang
0050ff254f [BugFix] fix bench_serving error when multimodal image is testing (#13254) 2025-11-14 22:46:19 +08:00
fzyzcjy
af9f71f9c5 Add script to create a model with fewer layers for debugging (#13284) 2025-11-14 22:13:27 +08:00
Yineng Zhang
f8d3d80f63 chore: bump flashinfer v0.5.2 (#13242) 2025-11-14 02:47:09 -08:00
fzyzcjy
3701f34dab Tiny add utility to parse server logs (#12605) 2025-11-14 17:34:22 +08:00
fzyzcjy
821fb060c3 Enhance dumper comparator with tensor unifier and location finder (#12623) 2025-11-14 17:34:08 +08:00
fzyzcjy
ace27c0c01 Tiny enhance dumper with ctx and enable flags (#12622) 2025-11-14 17:33:49 +08:00
fzyzcjy
ed1d18d472 Tiny fix update version logic location (#12620) 2025-11-14 17:33:36 +08:00
Morpheus Guo
e7b57b0d04 [BugFix] weight load bug when checkpoint expert.gate and exepert.up_proj are not fused (#13113) 
Co-authored-by: yuechguo <yuechguo@amd.com>
 2025-11-14 00:35:44 -08:00
Muqi Li
fc5da1e80b [Tool Call] Steamline function arguments when tool_choice="auto" for deepseekv31_detector (#11589) 2025-11-13 23:06:02 -08:00
Lianmin Zheng
7a2254b260 [Auto Sync] Update backend.py, forward_batch_info.py, piece... (20251113) (#13221) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-13 18:54:11 -08:00
Mick
922525ee6d fix: fix serve command without diffusion dependency (#13246) 2025-11-14 10:24:18 +08:00
billishyahao
ee3e337c62 [feat] make warmup timeout configurable through SGLANG_WARMUP_TIMEOUT (#13243) 2025-11-14 10:14:37 +08:00
b8zhong
e523e2167a remove deprecated tile_tokens_dim (#13186) 2025-11-13 17:09:27 -08:00
Lianmin Zheng
19f6a33ce9 [Auto Sync] Update pynccl_wrapper.py, environ.py, registry.... (20251111) (#13097) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-13 15:58:47 -08:00
Binyao Jiang
9b41f31a66 Use 32x32 black image for VLM server warmup and bring glm4.1v back to UT (#13222) 2025-11-13 14:21:43 -08:00
dtc
8779100768 Fix wrong running_bs in priority scheduling (#13142) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
 2025-11-14 03:56:38 +08:00
ykcombat
dd192a55f4 [Feature] Enable CUDA graph for PD-Multiplexing. (#11595) 2025-11-14 03:39:40 +08:00
huangtingwei
4ac65e3c7d [PD / HiCache]fix deocde kvcache offload manager memory leak (#12774) 2025-11-14 03:31:53 +08:00
neo
c8b7516fe8 Fix accept rate in speculative decoding metrics (#13212) 2025-11-14 00:54:50 +08:00
Bowen Bao
67e9d287ee [Quantization] Support Quark Dense + MoE FP8 & FP8 PTPC (#10485) 
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2025-11-13 08:16:00 -08:00
Sam
e7e89349c9 Enable Flashinfer TRTLLM-GEN-MoE FP8 blockwise kernel for Qwen3-Next on Blackwell (#12543) 2025-11-13 19:44:44 +08:00
Ling Zhang
aead0ef5e5 [FEAT][ROCM] enable fused shared expert for Rocm (#12201) 
Co-authored-by: ZLkanyo009 <4071250045@qq.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-11-13 00:41:40 -08:00
Shu Wang
6664083522 Replace [silu_and_mul_]scaled_fp4_group_quant by Flashinfer equivalent (#12376) 2025-11-13 00:26:00 -08:00
fzyzcjy
86255f27b4 Revert "fallback to triton mm_persistent kernel when deepGemm fail" (#13178) 2025-11-12 22:03:35 -08:00
Hubert Lu
e4b2937017 [AMD] Add AITER Custom All-Reduce (#13102) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: HaiShaw <hixiao@gmail.com>
 2025-11-12 21:53:44 -08:00
Mayyyy
7a8524b444 [Ascend] add npu synchronize (#13154) 
Co-authored-by: may_feimei <meifei5@huawei.com>
 2025-11-13 11:03:32 +08:00
Tejesh Anand
e42df37df0 Remove EBNF Composer (#13163) 2025-11-12 17:55:30 -08:00
Xiaoyu Zhang
a1cb717d0b Opt kimi_k2_thinking biased topk module (#13150) 2025-11-12 16:52:06 -08:00
Scott Lee
c94564912e Add RequestMetricsExporter utility to export request-level metrics (#10973) 2025-11-12 16:33:37 -08:00
Vladimir Serov
c4b74c1db2 [Ascend] LoRA: adding Ascend LoRA backend with using kernels from sgl_kernel_npu (#12288) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
 2025-11-12 16:20:34 -08:00
Shu Wang
7aa443903d Fix nan in global scaling factor for large scale nvfp4 EP (#13162) 2025-11-12 15:32:23 -08:00
bingps
4eda9969e8 [DeepseekV32]: use _concat_mla_absorb_q_general to replace torch.cat (#12215) 
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
 2025-11-12 15:20:01 -08:00
Lianmin Zheng
d646cf6347 [Auto Sync] Update test_deterministic.py (20251112) (#13128) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-12 11:12:23 -08:00
Ke Bao
4edb240112 Fuse routed_scaling_factor to fused_marlin_moe (#12998) 2025-11-13 00:18:58 +08:00
Yuan Luo
706502ff6c [VLM] Support PP for Qwen2.5-VL (#13075) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2025-11-12 23:18:44 +08:00
Makcum888e
c2e56dadb2 [Ascend] torch_npu.npu_mrope for MRotaryEmbedding (#10907) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: ssshinigami <44640852+ssshinigami@users.noreply.github.com>
 2025-11-12 21:45:54 +08:00
Zhihao Zhang
9c1c5c6d7d [ngram] use SGLANG_NGRAM_FORCE_GREEDY_VERIFY to control verify method (#13153) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-11-12 21:07:33 +08:00
khalilzhk
2d531946db [Ascend][feature] support L1+ L2 radixcache on ascend (#12214) 
Co-authored-by: khalilzhk <zhangkaikai12@huawei.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-11-12 20:45:24 +08:00
sglang-bot
ebaf86d441 chore: bump SGLang version to 0.5.5.post2 (#13129) 
Include the critical fix https://github.com/sgl-project/sglang/pull/12915.
 2025-11-12 20:35:20 +08:00
zhanghaotong
8359f185a1 [Feature] Propagate Trace Headers into Root Span for OpenTelemetry Cross-Service Context (#10808) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-11-12 20:28:59 +08:00
ronnie_zheng
5324f37ab3 [Ascend]adapt enable-profile-cuda-graph for NPU (#12617) 
Co-authored-by: Leo920320 <1530593013@qq.com>
Co-authored-by: liupengcheng <6531487+leo920320@user.noreply.gitee.com>
 2025-11-12 19:57:24 +08:00
Charlie Ruan
2864c49fbc [RPC] Fix handle_rpc_request with **recv_req.parameters (#7906) 2025-11-12 19:29:12 +08:00
Yikai Zhang
9c546bfd6d Fix the Wrong Return Type of Scheduler.recv_requests (#7886) 2025-11-12 18:54:56 +08:00
Rohan Potdar
c9b581644d Dump total_throughput to output-file in bench_serving.py (#9790) 2025-11-12 18:48:28 +08:00
SijiaYang
ffeb28ba6f fix: duplicate resize images logic of qwen-vl series models (#12458) 
Signed-off-by: yangsijia.614 <yangsijia.614@bytedance.com>
 2025-11-12 18:08:40 +08:00