kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-09 00:37:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,735 Commits 157 Branches 0 Tags
c660d8dfd08029a5f0e4d58dc5cf64336b294992
Commit Graph

5066 Commits

Author SHA1 Message Date
Liangsheng Yin
c660d8dfd0 Re-org eagle unit tests (#14909) 2025-12-12 12:25:39 +09:00
Ho-Ren (Jack) Chuang
10146af099 Check KV4 compatibility with attention backends and add KV4 support to the attention_backend doc (#14467) 
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
 2025-12-11 19:00:53 -08:00
Yuhao Yao
e9e7f15eb5 [bugfix] fix TBO crashes when attn_tp_size > 1 (#13730) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-12-11 18:18:40 -08:00
Alison Shao
0aa3dec5c7 Fix black formatting in ci_utils.py (#14932) 2025-12-11 17:28:52 -08:00
Xinyuan Tong
4885f8b9c1 fix: handle Jinja2 template errors as client errors in OpenAIServingChat (#14748) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 17:06:26 -08:00
Alison Shao
e59435c34b Add retry logic for scheduled CI tests (#14771) 2025-12-11 16:59:58 -08:00
Zaili Wang
d6bd2d1126 [CPU] layernorm & fused add-layernorm kernels (#14074) 2025-12-11 16:58:23 -08:00
Xinyuan Tong
9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
amysaq2023
70758d457e support non-disturbing remote-instance-weight-loader (#13125) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-12-11 16:45:32 -08:00
Xinyuan Tong
fd1ebbb0d6 update mistral detector (#14921) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:43:36 -08:00
b8zhong
6107268fe7 extend timeout for b200 test (#14925) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-11 15:43:53 -08:00
sglang-bot
5c8bd8b51b chore: bump SGLang version to 0.5.6.post2 (#14858) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2025-12-11 12:29:52 -08:00
Yinghai Lu
b05b346a13 [loader] enable private loader (#14620) 2025-12-11 11:49:46 -08:00
Jimmy
5c961756a3 [FIX][DS32]openai protocol: support openai message role: developer (#14304) 2025-12-11 11:34:41 -08:00
Mick
ca1144212a [diffusion] UX: suppress excessive loggers (#14900) 2025-12-11 22:10:22 +08:00
Liangsheng Yin
543d62d11a Introduce server_fixtures in sglang.test (#14899) 2025-12-11 22:30:33 +09:00
Vladimir221
27032cecd9 [Ascend]Support of piecewise graph compilation for prefill on NPU (#12287) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2025-12-11 21:10:07 +08:00
Mick
5d804a3767 [diffusion] feat: support sageattn & sageattn3 backend (#14878) 2025-12-11 20:59:44 +08:00
liupeng374
388018a5bd [NPU] adapt dsv3.2 nsa prefill context parallel (#14541) 2025-12-11 20:12:59 +08:00
AichenF
fca8e88f28 [diffusion] feat: enable torch compile to eliminate GPU bubble (#13641) 
Co-authored-by: jianyingzhu <53300651@qq.com>
Co-authored-by: Jianying <53503712+jianyingzhu@users.noreply.github.com>
Co-authored-by: root <root@2u2g-spr-0417.ipp4a1.colossus.nvidia.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-12-11 19:06:32 +08:00
fzyzcjy
45eeeb9a3c Tiny support range ratio in GSP in bench serving (#14828) 2025-12-11 17:26:07 +08:00
fzyzcjy
a368df2818 Tiny add more error info for bench_serving (#14827) 2025-12-11 17:25:49 +08:00
Alison Shao
f85460fb19 Avoid deleting entire cache for missing shards (#14754 follow-up) (#14853) 2025-12-11 01:17:04 -08:00
Yuhao Yang
b62fe8504c fix nightly vlm ci : restore original eval for requests without regex (#14875) 2025-12-10 23:13:25 -08:00
yuchengz816-bot
e54307f26a [6/n] Fix num_token_non_padded computation in prefill (#14313) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Runkai Tao <rt572@physics.rutger.edu>
 2025-12-10 19:15:19 -08:00
Trang Do
8642dbe416 Refactor Marlin MoeRunner (#14554) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-12-10 19:15:00 -08:00
roikoren755
7c985331dd Enable TP for Mamba-based models (#14811) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2025-12-10 18:03:50 -08:00
Liangsheng Yin
bd7824b24d Minor code style fix for dllm (#14836) 2025-12-11 10:35:42 +09:00
Binyao Jiang
312df1d6c0 Fix TestGLM41VPPAccuracy test flakiness (#14848) 2025-12-10 16:59:58 -08:00
Alison Shao
b6523a4f72 fix: restrict cache validation behaviors to CI only (#14849) 2025-12-10 16:03:53 -08:00
b8zhong
c51efb8b84 fix fp8 gemm nightly CI (#14844) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-10 15:57:51 -08:00
Binyao Jiang
a4992873d4 Treat unittest SkipTest exception as pass instead of as failure (#14847) 2025-12-10 15:28:21 -08:00
b8zhong
da9b801eb7 fix lora target all + csgmv backend (#14796) 2025-12-10 11:42:51 -08:00
Siyuan Chen
0e54a69548 [bugfix] qwen25-VL support lora (#14638) 2025-12-10 11:38:51 -08:00
Yuhao Yang
c1bd5ee8c5 Revert transformers to 4.57.1 (#14801) 2025-12-10 11:04:36 -08:00
Mick
d659873762 [diffusion] CI: use unified sampling_params for CI (#14045) 2025-12-11 01:18:56 +08:00
Mick
6c5ebc0ef7 [diffusion] parallel: pad tokens for video models under sp (#14833) 2025-12-11 01:15:37 +08:00
Ke Bao
5b5571a8da Apply back moe_sum_reduce for fused_marlin_moe (#14829) 2025-12-11 00:39:41 +08:00
Li Jinliang
83e35a7c29 [diffusion] doc: fix tiny typo in multimodal_gen/README.md (#14830) 2025-12-11 00:27:25 +08:00
liupeng374
b8cfa02c01 [NPU] bug fix for mtp and w4a8 (#14806) 2025-12-10 22:55:13 +08:00
fzyzcjy
5eccaf7737 Support HTTP response status code prometheus metrics (#14710) 2025-12-10 22:28:57 +08:00
Xiaoyu Zhang
12b7a4fab0 [diffusion] performance: refactor diffusion fuse qkv and apply to qwen-image (#14793) 2025-12-10 18:55:41 +08:00
Yuhao Yang
02f1e81e2d Revert "fix: checking if tokenizer is in cache before downloading from HF" (#14808) 2025-12-10 01:14:35 -08:00
Prozac614
908c7186af [diffusion] CI: Add LoRA support to diffusion server configuration and test cases (#14697) 2025-12-10 16:51:47 +08:00
Yuan Luo
03836d85d2 [GLM-4.6V] Support Pipeline Parallelism for GLM-4.6V & GLM-4.1V (#14720) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-10 16:40:12 +08:00
Mick
87dbdddc93 [diffusion] profile: early exit when enough steps are captured to reduce the size of the trace file (#14803) 2025-12-10 16:11:22 +08:00
yrk111222
6c9c8da64d fix: add missing logic for SGLANG_USE_MODELSCOPE variable (#14794) 2025-12-10 15:49:34 +08:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
Yuhao Yang
793c98afaf handling incomplete rope_scaling config ci after transformers upgrade (#14784) 2025-12-09 22:56:16 -08:00
TomerBN-Nvidia
b1cbfce612 fix server args bug (#14725) 2025-12-09 21:05:05 -08:00