kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 16:27:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,885 Commits 157 Branches 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

6437 Commits

Author SHA1 Message Date
fzyzcjy
8cc77261ec Super tiny remove unused argument (#14966) 2025-12-13 09:41:01 +08:00
Yinghai Lu
d143b02097 [registry] Add a strict mode to model registration (#14933) 2025-12-13 09:25:48 +08:00
Chenxi Li
9b9d21312a Feature/Fix multi lora scheduler blocking issue and evict LoRA None lastly (#14795) 2025-12-12 17:13:05 -08:00
Qiaolin Yu
44fd701732 Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 (#15020) 2025-12-12 17:04:56 -08:00
Lianmin Zheng
1b5e903480 Refactor of http and engine entrypoints to allow custom override (#14869) 2025-12-12 12:01:08 -08:00
Ho-Ren (Jack) Chuang
171b442ad3 Add KV4-capable backend flashmla and update server args (#14989) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-12 11:50:27 -08:00
Yineng Zhang
4b7b5af36a Revert several PRs (#14958) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-12-12 11:25:12 -08:00
Kangyan-Zhou
b243154614 Fix CI by reverting incorrect metric check logic (#15004) 2025-12-12 10:07:38 -08:00
Xiaoyu Zhang
3e1e71575c [diffusion] docker: Tiny fix Docker Hub link in installation documentation (#14987) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-12 20:25:36 +08:00
Kevin Li
8fa8d9d7e8 [PD] Add decode PP event loop for PD disaggregation (#14945) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-12 20:15:00 +08:00
DefTruth
d71baa72dc [diffusion] dependency: upgrade cache-dit for better compatibility (#14534) 2025-12-12 18:52:58 +08:00
Mick
82e33170e1 [diffusion] chore: minor code cleanups and improve logging (#14916) 2025-12-12 18:48:07 +08:00
Zaili Wang
4dabfbc827 [Fix] suppress remote weight loading engine w/o mooncake installed (#14937) 2025-12-11 23:12:26 -08:00
Sam
d7ed8a8c24 [NVIDIA] Enable TRTLLM BF16 MoE on Blackwell GPUs (#13798) 2025-12-11 22:56:13 -08:00
ZhengdQin
c05d3afb5d [NPU] optimization for dsv3.2 (#14572) 2025-12-12 14:52:16 +08:00
b8zhong
fe6d38d2fa fix: trtllm mha attention auto-selection on sm120 (#14842) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 22:35:25 -08:00
b8zhong
dab31e4cc7 direct register custom op for mm_fp4 (#13699) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 22:33:17 -08:00
Prozac614
198c8ecf98 [diffusion] fix: support applying different LoRA adapters to different transformers in multi-transformer pipelines (#14839) 2025-12-12 13:08:57 +08:00
Xuchun Shang
c01b2ee094 [PP] Refactor PP to async mode (#11852) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: bluecoffee8 <jasperli2002@gmail.com>
Co-authored-by: zhangxiaolei123456 <zhangxiaolei.666@bytedance.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-12 12:54:16 +08:00
Peng Zhang
8f5adac8c6 [RL] refactor flash rl weight reload in sglang (#14870) 
Co-authored-by: eternally-z <zzywzj@gmail.com>
 2025-12-12 12:50:47 +08:00
Xinyuan Tong
8bf10e71cd remove dpsk3.2 sys prompt (#14923) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 20:26:13 -08:00
Liangsheng Yin
c660d8dfd0 Re-org eagle unit tests (#14909) 2025-12-12 12:25:39 +09:00
Ho-Ren (Jack) Chuang
10146af099 Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc (#14467) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-11 19:00:53 -08:00
Yuhao Yao
e9e7f15eb5 [bugfix] fix TBO crashes when attn_tp_size > 1 (#13730) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-12-11 18:18:40 -08:00
Alison Shao
0aa3dec5c7 Fix black formatting in ci_utils.py (#14932) 2025-12-11 17:28:52 -08:00
Xinyuan Tong
4885f8b9c1 fix: handle Jinja2 template errors as client errors in OpenAIServingChat (#14748) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 17:06:26 -08:00
Alison Shao
e59435c34b Add retry logic for scheduled CI tests (#14771) 2025-12-11 16:59:58 -08:00
Zaili Wang
d6bd2d1126 [CPU] layernorm & fused add-layernorm kernels (#14074) 2025-12-11 16:58:23 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Xinyuan Tong
fd1ebbb0d6 update mistral detector (#14921) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:43:36 -08:00
b8zhong
6107268fe7 extend timeout for b200 test (#14925) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 15:43:53 -08:00
sglang-bot
5c8bd8b51b chore: bump SGLang version to 0.5.6.post2 (#14858) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-12-11 12:29:52 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
Jimmy
5c961756a3 [FIX][DS32]openai protocol: support openai message role: developer (#14304) 2025-12-11 11:34:41 -08:00
Mick
ca1144212a [diffusion] UX: suppress excessive loggers (#14900) 2025-12-11 22:10:22 +08:00
Liangsheng Yin
543d62d11a Introduce server_fixtures in sglang.test (#14899) 2025-12-11 22:30:33 +09:00
Vladimir221
27032cecd9 [Ascend]Support of piecewise graph compilation for prefill on NPU (#12287) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-12-11 21:10:07 +08:00
Mick
5d804a3767 [diffusion] feat: support sageattn & sageattn3 backend (#14878) 2025-12-11 20:59:44 +08:00
liupeng374
388018a5bd [NPU] adapt dsv3.2 nsa prefill context parallel (#14541) 2025-12-11 20:12:59 +08:00
AichenF
fca8e88f28 [diffusion] feat: enable torch compile to eliminate GPU bubble (#13641) 
Co-authored-by: jianyingzhu <53300651@qq.com>
Co-authored-by: Jianying <53503712+jianyingzhu@users.noreply.github.com>
Co-authored-by: root <root@2u2g-spr-0417.ipp4a1.colossus.nvidia.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-12-11 19:06:32 +08:00
fzyzcjy
45eeeb9a3c Tiny support range ratio in GSP in bench serving (#14828) 2025-12-11 17:26:07 +08:00
fzyzcjy
a368df2818 Tiny add more error info for bench_serving (#14827) 2025-12-11 17:25:49 +08:00
Alison Shao
f85460fb19 Avoid deleting entire cache for missing shards (#14754 follow-up) (#14853) 2025-12-11 01:17:04 -08:00
Yuhao Yang
b62fe8504c fix nightly vlm ci : restore original eval for requests without regex (#14875) 2025-12-10 23:13:25 -08:00
yuchengz816-bot
e54307f26a [6/n] Fix num_token_non_padded computation in prefill (#14313) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Runkai Tao <rt572@physics.rutger.edu>
 2025-12-10 19:15:19 -08:00
Trang Do
8642dbe416 Refactor Marlin MoeRunner (#14554) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-12-10 19:15:00 -08:00
roikoren755
7c985331dd Enable TP for Mamba-based models (#14811) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-10 18:03:50 -08:00
Liangsheng Yin
bd7824b24d Minor code style fix for dllm (#14836) 2025-12-11 10:35:42 +09:00
Binyao Jiang
312df1d6c0 Fix TestGLM41VPPAccuracy test flakiness (#14848) 2025-12-10 16:59:58 -08:00