kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-09 08:47:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,798 Commits 157 Branches 0 Tags
dcc5f5c0daa5016991958d4e8cc4e071af1fbf2f
Commit Graph

5100 Commits

Author SHA1 Message Date
Fenglin Yu
dcc5f5c0da [diffusion] feat: Improve LoRA compatibility by adding unified format detection and diffusers-based normalization (#14659) 2025-12-13 16:04:36 +08:00
Sam
4eda4194f2 [Fix] Disable trtllm moe backend for draft model for a qucik fix (#15002) 2025-12-12 23:58:47 -08:00
Mick
875f84db7b [diffusion] fix: use NDRotaryEmbedding in flux_2 (#15034) 2025-12-13 13:42:38 +08:00
Daniel Cámpora
f6031adf08 Mistral Large 3 NVFP4 support (#14485) 
Co-authored-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-12-13 13:34:42 +08:00
Chunyuan WU
2a39cfe0ff call check_quantized_moe_compatibility after initialize (#13876) 2025-12-13 13:32:19 +08:00
Liangsheng Yin
01e3b3f3a3 Fix decode OOM caused by retraction (#14939) 2025-12-13 12:59:17 +08:00
Shu Wang
665cb02003 Fix regression caused by fa3 block_table (#15009) 2025-12-12 19:11:09 -08:00
Liangsheng Yin
7160283800 Tiny remove the duplicate function in spec v2 (#14957) 2025-12-13 11:08:27 +08:00
Qiaolin Yu
77873343c4 tiny update: use rope kernel in sgl-kernel for amd (#14955) 2025-12-12 18:48:12 -08:00
Lianmin Zheng
267170bf1d Clean up server args and engine startup processes (#15015) 2025-12-12 18:46:07 -08:00
fzyzcjy
313f59ad80 Add soft watchdogs to debug soft hangs (#15023) 2025-12-13 10:41:35 +08:00
fzyzcjy
487cf81a68 Tiny extract SchedulerWatchdog (#15021) 2025-12-13 10:40:32 +08:00
fzyzcjy
df111bc0fe Super tiny add gsp-fast-prepare (#14992) 2025-12-13 09:45:21 +08:00
fzyzcjy
8cc77261ec Super tiny remove unused argument (#14966) 2025-12-13 09:41:01 +08:00
Yinghai Lu
d143b02097 [registry] Add a strict mode to model registration (#14933) 2025-12-13 09:25:48 +08:00
Chenxi Li
9b9d21312a Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly (#14795) 2025-12-12 17:13:05 -08:00
Qiaolin Yu
44fd701732 Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 (#15020) 2025-12-12 17:04:56 -08:00
Lianmin Zheng
1b5e903480 Refactor of http and engine entrypoints to allow custom override (#14869) 2025-12-12 12:01:08 -08:00
Ho-Ren (Jack) Chuang
171b442ad3 Add KV4-capable backend flashmla and update server args (#14989) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-12 11:50:27 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Kangyan-Zhou
b243154614 Fix CI by reverting incorrect metric check logic (#15004) 2025-12-12 10:07:38 -08:00
Xiaoyu Zhang
3e1e71575c [diffusion] docker: Tiny fix Docker Hub link in installation documentation (#14987) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-12 20:25:36 +08:00
Kevin Li
8fa8d9d7e8 [PD] Add decode PP event loop for PD disaggregation (#14945) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-12 20:15:00 +08:00
DefTruth
d71baa72dc [diffusion] dependency: upgrade cache-dit for better compatibility (#14534) 2025-12-12 18:52:58 +08:00
Mick
82e33170e1 [diffusion] chore: minor code cleanups and improve logging (#14916) 2025-12-12 18:48:07 +08:00
Zaili Wang
4dabfbc827 [Fix] suppress remote weight loading engine w/o mooncake installed (#14937) 2025-12-11 23:12:26 -08:00
Sam
d7ed8a8c24 [NVIDIA] Enable TRTLLM BF16 MoE on Blackwell GPUs (#13798) 2025-12-11 22:56:13 -08:00
ZhengdQin
c05d3afb5d [NPU] optimization for dsv3.2 (#14572) 2025-12-12 14:52:16 +08:00
b8zhong
fe6d38d2fa fix: trtllm mha attention auto-selection on sm120 (#14842) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 22:35:25 -08:00
b8zhong
dab31e4cc7 direct register custom op for mm_fp4 (#13699) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 22:33:17 -08:00
Prozac614
198c8ecf98 [diffusion] fix: support applying different LoRA adapters to different transformers in multi-transformer pipelines (#14839) 2025-12-12 13:08:57 +08:00
Xuchun Shang
c01b2ee094 [PP] Refactor PP to async mode (#11852) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: bluecoffee8 <jasperli2002@gmail.com>
Co-authored-by: zhangxiaolei123456 <zhangxiaolei.666@bytedance.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-12 12:54:16 +08:00
Peng Zhang
8f5adac8c6 [RL] refactor flash rl weight reload in sglang (#14870) 
Co-authored-by: eternally-z <zzywzj@gmail.com>
 2025-12-12 12:50:47 +08:00
Xinyuan Tong
8bf10e71cd remove dpsk3.2 sys prompt (#14923) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 20:26:13 -08:00
Liangsheng Yin
c660d8dfd0 Re-org eagle unit tests (#14909) 2025-12-12 12:25:39 +09:00
Ho-Ren (Jack) Chuang
10146af099 Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc (#14467) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-11 19:00:53 -08:00
Yuhao Yao
e9e7f15eb5 [bugfix] fix TBO crashes when attn_tp_size > 1 (#13730) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-12-11 18:18:40 -08:00
Alison Shao
0aa3dec5c7 Fix black formatting in ci_utils.py (#14932) 2025-12-11 17:28:52 -08:00
Xinyuan Tong
4885f8b9c1 fix: handle Jinja2 template errors as client errors in OpenAIServingChat (#14748) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 17:06:26 -08:00
Alison Shao
e59435c34b Add retry logic for scheduled CI tests (#14771) 2025-12-11 16:59:58 -08:00
Zaili Wang
d6bd2d1126 [CPU] layernorm & fused add-layernorm kernels (#14074) 2025-12-11 16:58:23 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Xinyuan Tong
fd1ebbb0d6 update mistral detector (#14921) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:43:36 -08:00
b8zhong
6107268fe7 extend timeout for b200 test (#14925) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 15:43:53 -08:00
sglang-bot
5c8bd8b51b chore: bump SGLang version to 0.5.6.post2 (#14858) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-12-11 12:29:52 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
Jimmy
5c961756a3 [FIX][DS32]openai protocol: support openai message role: developer (#14304) 2025-12-11 11:34:41 -08:00
Mick
ca1144212a [diffusion] UX: suppress excessive loggers (#14900) 2025-12-11 22:10:22 +08:00
Liangsheng Yin
543d62d11a Introduce server_fixtures in sglang.test (#14899) 2025-12-11 22:30:33 +09:00