kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-10 01:07:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,652 Commits 157 Branches 0 Tags
87dbdddc93b15d9bf87ffe07dca03b59dc94addf
Commit Graph

5021 Commits

Author SHA1 Message Date
Mick
87dbdddc93 [diffusion] profile: early exit when enough steps are captured to reduce the size of the trace file (#14803) 2025-12-10 16:11:22 +08:00
yrk111222
6c9c8da64d fix: add missing logic for SGLANG_USE_MODELSCOPE variable (#14794) 2025-12-10 15:49:34 +08:00
Peng Zhang
21028b5507 [RL] support weight reload for low-bit rollout (#9650) 
Co-authored-by: Hecate0821 <hec4te0821@gmail.com>
Co-authored-by: eternally-z <zzywzj@gmail.com>
Co-authored-by: Wilboludriver <wilbolu@outlook.com>
Co-authored-by: Wilbolu <81792854+Wilboludriver@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2025-12-10 15:44:01 +08:00
Yuhao Yang
793c98afaf handling incomplete rope_scaling config ci after transformers upgrade (#14784) 2025-12-09 22:56:16 -08:00
TomerBN-Nvidia
b1cbfce612 fix server args bug (#14725) 2025-12-09 21:05:05 -08:00
Alison Shao
01835998e1 fix: race condition between validation and download locks (#14761) 2025-12-09 20:36:54 -08:00
Lianmin Zheng
4285e99da7 [Auto Sync] Update data_parallel_controller.py, detokenizer... (20251209) (#14759) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-09 18:38:38 -08:00
ShawnY112358
f077436831 [fix] Fix issues for in-flight weight updates (#14064) 
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
 2025-12-09 18:12:19 -08:00
Liangsheng Yin
cbc7dcdaa7 Re-add the API serving timing metrics. (#14744) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
Co-authored-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-10 10:17:48 +09:00
Ethan (Yusheng) Su
0c63fb9420 [Feature] Add LoRA support for embedding layers (#14177) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Beichen-Ma <bm685@cornell.edu>
 2025-12-09 15:53:33 -08:00
Lianmin Zheng
18bd8e8d6d Improve CI by trying a warmup before unit tests (#14669) 2025-12-09 15:17:59 -08:00
Lianmin Zheng
036e64dafa move multi-item scoring functions in tokenizer manager into a separate file (#14740) 2025-12-09 14:47:06 -08:00
Douglas Yang
8b0b6a45c8 fix: checking if tokenizer is in cache before downloading from HF (#14698) 2025-12-09 13:08:07 -08:00
b8zhong
55504df2f7 Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379) 2025-12-09 12:05:56 -08:00
Yuhao Yang
15bc8cbd74 fix rope parameter initialization error caused by transformers v5.0 update (#14745) 2025-12-09 10:51:26 -08:00
yudian0504
9496f12d00 [Model] Add PaddleOCR-VL Model Support (#12953) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-09 10:16:02 -08:00
Lianmin Zheng
ab0048793c Revert "[Feat] Add received_time in serving_base" (#14743) 2025-12-09 06:52:27 -08:00
blahblah
6ec77680df [diffusion] feat: support comparing batch perf (#14738) 
Co-authored-by: shuxiguo <shuxiguo@meituan.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-09 22:48:52 +08:00
yrk111222
98c430e114 fix: prevent HugginqFace access when SGLANG_USE_MODELSCOPE is enabled (#12039) 2025-12-09 21:02:05 +08:00
zhanghaotong
fe7f91ef82 [Feat] Add received_time in serving_base (#13432) 
Signed-off-by: zhanghaotong <zhanghaotong.zht@antgroup.com>
 2025-12-09 22:00:24 +09:00
kun-llfl
cef5ba65b1 [Bugfix] Fix environ error in scheduler_runtime_checker_mixin.py (#14461) 
Signed-off-by: Kun(llfl) <i@imux.top>
 2025-12-09 20:33:41 +08:00
Xiaoyu Zhang
53d170883a Add fuse_marlin_moe test to ci and add new ep test (#14686) 2025-12-09 20:17:38 +08:00
Rain Jiang
f0e948a0f1 fix the deepep 8 gpu unit test (#14601) 2025-12-09 01:40:09 -08:00
Yineng Zhang
b626334475 fix: make override DeepseekV2Model work (#14707) 2025-12-08 23:11:18 -08:00
fzyzcjy
da3dc497b0 Fix dp-aware incompatible with completions and chat completions APIs (#14647) 2025-12-09 13:24:53 +08:00
Lianmin Zheng
ce4e836be5 Add per-request decode tp size (#14678) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-12-08 20:24:31 -08:00
Yineng Zhang
0e0b0c0566 Revert "[Bug] fix not desired disable fused share experts caused by r… (#14676) 2025-12-08 20:06:52 -08:00
Alison Shao
08da4c2618 [Bugfix] Fix KeyError for Mistral-Large-3 rope_scaling config (#14627) 2025-12-08 19:16:00 -08:00
kk
c106b54b57 Aiter fp8 kv cache (#13147) 2025-12-08 16:39:53 -08:00
fzyzcjy
119fd956fb Tiny support printing requests in bench_serving for observability (#14652) 2025-12-09 08:27:58 +08:00
PiteXChen
eac5b66485 [RadixTree] Optimize the Time Complexity of Node Retrieval Operation from O(n*m) to O(n) (#13334) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-12-08 15:55:09 -08:00
Francis
07404d7689 [HiCache] fix condition check when use decode offload (#14489) 2025-12-08 15:52:40 -08:00
Binyao Jiang
6abb8051e8 Bump up diffusers to latest official release version (#14670) 2025-12-08 13:41:01 -08:00
b8zhong
32f8b6064e improve default glm mtp setting (#14457) 
Signed-off-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-08 13:27:13 -08:00
Yuhao Yang
b9bef31a15 fix: use .get() when accessing strict mem-check env variable (#14657) 2025-12-08 13:25:42 -08:00
Mick
8810152e88 vlm: Use fa3 as the default backend for qwen3 vl (#14634) 2025-12-08 12:56:20 -08:00
yctseng0211
763888b5a8 [AMD] change fused rms quant interface for aiter upgrade (#14497) 2025-12-08 09:09:23 -08:00
sglang-bot
9a327bdfcf chore: bump SGLang version to 0.5.6.post1 (#14651) 2025-12-09 00:35:28 +08:00
sglang-bot
2de98010b5 chore: bump sgl-kernel version to 0.3.19 (#14649) 2025-12-08 22:53:08 +08:00
Yuhao Yang
8200fb56cb update transformers package version to 5.0.0rc0 (#14356) 2025-12-08 22:46:01 +08:00
Mick
80cfca50bc [diffusion] chore: further refine output resolution adjustment logic (#14558) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-08 19:08:38 +08:00
Yibo Cai
7871593cc8 [cpu] Implement all gather/reduce for arm64 cpu (#12527) 2025-12-08 19:03:04 +08:00
Prozac614
12a08efc20 [diffusion] feat: add support for LoRA layers in transformer_2 within LoRAPipeline (#14606) 2025-12-08 17:57:13 +08:00
Muqi Li
06836ad02a [Reasoning + Structured Output] make reasoning compatible with structured output (#12551) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-08 01:28:39 -08:00
Binyao Jiang
cf0478d602 [Glm46v] Bug fix for accuracy drop and unable to launch server (#14585) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
 2025-12-07 23:45:02 -08:00
Tiwei Bie
36361adcbf [DLLM] Add initial cuda graph support (#14203) 2025-12-08 14:12:35 +08:00
Qiaolin Yu
661e9775d0 [2/2] Add rope kernel in sgl-kernel (#14452) 2025-12-07 21:37:29 -08:00
Nicholas
f57d4fe78e [feat] use cachebuffer to store mm feature to speedup hash (#14386) 2025-12-08 10:35:20 +08:00
wentx
b7b7524e95 [Tool Call] Fix DeepSeekV32Detector skipping functions with no params in streaming mode (#14573) 2025-12-07 18:32:10 -08:00
Xiaoyu Zhang
03b835e7d1 Refactor tuning block wise kernel and opt Qwen/Qwen3-VL-32B-Instruct-FP8 (#14141) 2025-12-08 09:24:58 +08:00