kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-11 01:36:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,828 Commits 157 Branches 0 Tags
6abdf73f4d4e0e8a652aba8257a6cf329d9226d4
Commit Graph

5120 Commits

Author SHA1 Message Date
Huang Lin
6abdf73f4d [Fix] Environment variable SGL_* is deprecated (#14943) 2025-12-13 19:55:43 -08:00
fzyzcjy
168a31eb00 Support prefill max requests limitation (#14993) 2025-12-14 10:52:10 +08:00
narutolhy
69cfb17b9a [Fix] avoid stream sync in _compute_mrope_positions (#14956) 2025-12-13 18:50:57 -08:00
fzyzcjy
fdc93b019c Add sglang:decode_sum_seq_lens metric (#15066) 2025-12-14 08:42:45 +08:00
Xinyuan Tong
96705514bd fix: dpskv32 chat history processing, default drop_thinking to true (#15064) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-13 12:16:04 -08:00
Baizhou Zhang
ab3ffd1c8e Add nightly accuracy test for DeepSeek V3.2 (#14935) 2025-12-13 12:11:16 -08:00
Yuhao Yang
a81cc1b8b3 add transformers version validation for glm-4.6v moe models (#14998) 2025-12-13 10:54:08 -08:00
liupeng374
3134d2b2a7 [scheduler] enhance scheduler in dp_attention mixed case with spec (#14201) 2025-12-14 02:52:26 +08:00
Yuhao Yang
06b58c5dc5 fix flaky image access in ci by switching to raw content url (#14940) 2025-12-13 10:52:06 -08:00
Simo Lin
5d9c6bac07 [bug] fix grpc secheduler launcher breaking change (#15080) 2025-12-13 10:36:49 -08:00
Liangsheng Yin
ed52d01b0b Fix spec info's filter when reqs are finished right after prefill (#14742) 2025-12-14 00:32:54 +08:00
Liangsheng Yin
90e7d4f78f Tiny adjust CI run suite (#15074) 2025-12-14 00:05:19 +08:00
Mick
c20d43d2e6 [diffusion] doc: update profiling.md with output location details (#15072) 2025-12-13 23:15:23 +08:00
Liangsheng Yin
d977dd2e07 Fix IMA with flashinfer + spec + topk & Add radix attention test cases for eagle (#13740) 2025-12-13 23:04:41 +08:00
Xiaoyu Zhang
0c23331e2e [diffusion] doc: add multimodal-gen profiling doc (#15069) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-13 22:26:20 +08:00
Yuan Luo
3b8a824b8b [VLM] Support VLM ViT Piecewise CUDA Graph (#14422) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: kousakawang <wanghanpei@bytedance.com>
 2025-12-13 20:49:40 +08:00
Stefan He
f1bbd26ff7 Clean up GDN Init (#14855) 2025-12-13 02:56:54 -08:00
liupeng374
d36299ad77 [NPU] perf update with kvcache nz & w4a8 quant (#14423) 2025-12-13 17:39:55 +08:00
YAMY
0e7d7969d5 [PP Prefill][NIXL] Fix PP mode transfer completion tracking to wait for all ranks (#15027) 2025-12-13 00:55:28 -08:00
cynial
80554598d3 Fix GLM-4.6 tool calls don't support streaming output for arguments i… (#13989) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-13 00:37:18 -08:00
Fenglin Yu
dcc5f5c0da [diffusion] feat: Improve LoRA compatibility by adding unified format detection and diffusers-based normalization (#14659) 2025-12-13 16:04:36 +08:00
Sam
4eda4194f2 [Fix] Disable trtllm moe backend for draft model for a qucik fix (#15002) 2025-12-12 23:58:47 -08:00
Mick
875f84db7b [diffusion] fix: use NDRotaryEmbedding in flux_2 (#15034) 2025-12-13 13:42:38 +08:00
Daniel Cámpora
f6031adf08 Mistral Large 3 NVFP4 support (#14485) 
Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-12-13 13:34:42 +08:00
Chunyuan WU
2a39cfe0ff call check_quantized_moe_compatibility after initialize (#13876) 2025-12-13 13:32:19 +08:00
Liangsheng Yin
01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
Shu Wang
665cb02003 Fix regression caused by fa3 block_table (#15009) 2025-12-12 19:11:09 -08:00
Liangsheng Yin
7160283800 Tiny remove the duplicate function in spec v2 (#14957) 2025-12-13 11:08:27 +08:00
Qiaolin Yu
77873343c4 tiny update: use rope kernel in sgl-kernel for amd (#14955) 2025-12-12 18:48:12 -08:00
Lianmin Zheng
267170bf1d Clean up server args and engine startup processes (#15015) 2025-12-12 18:46:07 -08:00
fzyzcjy
313f59ad80 Add soft watchdogs to debug soft hangs (#15023) 2025-12-13 10:41:35 +08:00
fzyzcjy
487cf81a68 Tiny extract SchedulerWatchdog (#15021) 2025-12-13 10:40:32 +08:00
fzyzcjy
df111bc0fe Super tiny add gsp-fast-prepare (#14992) 2025-12-13 09:45:21 +08:00
fzyzcjy
8cc77261ec Super tiny remove unused argument (#14966) 2025-12-13 09:41:01 +08:00
Yinghai Lu
d143b02097 [registry] Add a strict mode to model registration (#14933) 2025-12-13 09:25:48 +08:00
Chenxi Li
9b9d21312a Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly (#14795) 2025-12-12 17:13:05 -08:00
Qiaolin Yu
44fd701732 Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 (#15020) 2025-12-12 17:04:56 -08:00
Lianmin Zheng
1b5e903480 Refactor of http and engine entrypoints to allow custom override (#14869) 2025-12-12 12:01:08 -08:00
Ho-Ren (Jack) Chuang
171b442ad3 Add KV4-capable backend flashmla and update server args (#14989) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-12 11:50:27 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Kangyan-Zhou
b243154614 Fix CI by reverting incorrect metric check logic (#15004) 2025-12-12 10:07:38 -08:00
Xiaoyu Zhang
3e1e71575c [diffusion] docker: Tiny fix Docker Hub link in installation documentation (#14987) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-12 20:25:36 +08:00
Kevin Li
8fa8d9d7e8 [PD] Add decode PP event loop for PD disaggregation (#14945) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-12 20:15:00 +08:00
DefTruth
d71baa72dc [diffusion] dependency: upgrade cache-dit for better compatibility (#14534) 2025-12-12 18:52:58 +08:00
Mick
82e33170e1 [diffusion] chore: minor code cleanups and improve logging (#14916) 2025-12-12 18:48:07 +08:00
Zaili Wang
4dabfbc827 [Fix] suppress remote weight loading engine w/o mooncake installed (#14937) 2025-12-11 23:12:26 -08:00
Sam
d7ed8a8c24 [NVIDIA] Enable TRTLLM BF16 MoE on Blackwell GPUs (#13798) 2025-12-11 22:56:13 -08:00
ZhengdQin
c05d3afb5d [NPU] optimization for dsv3.2 (#14572) 2025-12-12 14:52:16 +08:00
b8zhong
fe6d38d2fa fix: trtllm mha attention auto-selection on sm120 (#14842) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 22:35:25 -08:00
b8zhong
dab31e4cc7 direct register custom op for mm_fp4 (#13699) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 22:33:17 -08:00