kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 16:27:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,885 Commits 157 Branches 0 Tags
cc63c99f112f781605017db018fced686cdc94de
Commit Graph

6437 Commits

Author SHA1 Message Date
Qiaolin Yu
173940927f Add customized sampler registration (#15423) 2025-12-18 23:10:23 -08:00
Lianmin Zheng
f228b662a7 Update readme (#15425) 2025-12-18 23:06:00 -08:00
b8zhong
a36142aa71 Fix Mindspore model import warning (#15287) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-12-18 23:15:33 -05:00
Yingchun Lai
160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) 
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
 2025-12-19 11:40:07 +08:00
Mick
a0985dd5e5 [diffusion] profiling: add bench_serving.py and VBench (#15410) 2025-12-19 10:57:39 +08:00
Zehuan Li
f6c9db4bc4 [DLLM] Fix dLLM regression (#15371) 2025-12-19 10:55:46 +08:00
b8zhong
e88e75a9df [Deepseek V3.2] Fix Deepseek MTP in V1 mode (#15429) 2025-12-18 17:56:07 -08:00
Zehuan Li
b2803ff207 [DLLM] Add CI for diffusion LLMs (#14723) 2025-12-19 08:54:03 +08:00
Charles Chen
9e0ef04e5b Support using different attention backend for draft decoding. (#14843) 2025-12-19 08:49:11 +08:00
Jimmy
216067c0cb feat(dsv32): better error handling for DeepSeek-v3.2 encoder (#14353) 2025-12-18 16:34:27 -08:00
b8zhong
e72b02db28 tiny fix lint on main (#15424) 2025-12-18 15:55:02 -08:00
Feng Su
29e8f7f9e5 multimodal: precompute hash for MultimodalDataItem (#14354) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
Signed-off-by: Junjie Mao <junjie.mao@linux.alibaba.com>
 2025-12-18 15:27:59 -08:00
Johnsonms
e0026f7c92 [Performance] optimize NSA backend metadata computation for multi-step speculative decoding (#14781) 2025-12-18 13:48:27 -08:00
Frank
9749d3e346 Update benchmarks to use HF token from environment. (#15421) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-18 13:47:27 -08:00
Baizhou Zhang
2b0ddf89f5 [Tiny]Add warning for deepgemm on Blackwell (#15352) 2025-12-18 13:45:26 -08:00
fzyzcjy
88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) 2025-12-18 22:24:23 +08:00
fzyzcjy
602fe3b296 Super tiny add moe_ep_rank to prometheus labels (#15407) 2025-12-18 22:23:20 +08:00
fzyzcjy
ad9616f13a Tiny extract ModelRunnerOutput (#15400) 2025-12-18 22:18:45 +08:00
fzyzcjy
c5f4e20f2f Support GPU execution time breakdown by forward mode metrics (#15396) 2025-12-18 22:18:17 +08:00
fzyzcjy
2c196f95a7 Tiny fix unknown route in prometheus metrics (#15404) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-18 22:17:46 +08:00
Xiaoyu Zhang
9a7641d7bf [diffusion] profiling: include per-denoising-step timings in perf-dump-path (#15397) 2025-12-18 18:43:33 +08:00
Bruce-x-1997
793c96c3d2 [perf]optimize w4afp8 kernel on deepseek-v3-0324 (#12921) 
Signed-off-by: bruce.xu <bruce.x@gmicloud.ai>
 2025-12-18 18:13:22 +08:00
Lianmin Zheng
d1f0063262 Clean up __init__ function of the scheduler and event loop for PD (#15298) 2025-12-18 01:35:14 -08:00
Xuchun Shang
fea2d5211d [bug fix][pp] fix inconsistent latency between tp (#15379) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 16:59:56 +08:00
Liangsheng Yin
374ad4cce0 Fix gpt-oss yarn with truncate argument (#14270) 2025-12-18 16:31:15 +08:00
Liangsheng Yin
8b0a68f1bf Monkey patch deepseek-ocr's v_head_dim (#15384) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-18 16:28:07 +08:00
Shangming Cai
ee1ca51de8 [PP] Fix dynamic chunking strategy for PP (#15372) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 14:24:55 +08:00
Yuhao Yang
3d42b7e7b0 unified management of environment variables for vlm cuda ipc transport (#14501) 2025-12-18 12:28:06 +08:00
elvischenv
9970ee34e8 Mistral Large 3 NVFP4 TRTLLM MoE support (#15049) 2025-12-18 11:11:42 +08:00
Xinyuan Tong
41683536d3 feat: DeepSeek-V3.2 Streaming tool call output (#15278) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: momaek <momaek17@gmail.com>
Co-authored-by: Muqi Li <muqi1029@gmail.com>
 2025-12-17 17:43:58 -08:00
Dongjie Zou
891ee8221f Feature/trtllm mha workspace size configurable #15089 (#15131) 2025-12-17 17:07:23 -08:00
Yuan Luo
8fa3dc36c5 [VLM] Support cos sin cache for Qwen3-VL & GLM-4.1V (#15205) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-18 08:39:50 +08:00
b8zhong
d20699a33c [Deepseek V3.2] Support Overlap Spec + NSA (#15307) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-17 13:35:39 -08:00
Scott Lee
169a75dfb8 Add request-level timestamp for when prefill finishes (#14860) 2025-12-17 13:34:46 -08:00
Alison Shao
4128d4f5cb [CI] Migrate LoRA tests to test/registered/lora/ (#15176) 2025-12-17 13:19:42 -08:00
Cheng Wan
5290cef97c Fix condition check for require_gathered_buffer (#15328) 2025-12-17 12:42:47 -08:00
许霖
726fe3e72e fix: prevent points regex from matching checkpoints/endpoints (#15120) 2025-12-17 12:38:18 -08:00
kk
5d087891c9 Fix dp run error with fp8-kv enable in high concurrency test (#15241) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-12-17 11:49:18 -08:00
Liangsheng Yin
0c00220795 tiny unify environ usage (#15335) 2025-12-17 23:31:43 +08:00
Shangming Cai
eeb2b9b259 [PP] Minor code cleanup for Pipeline Parallelism (#15329) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-17 23:31:25 +08:00
fzyzcjy
9d04b570bc Tiny dump native stacktraces in watchdog (#15222) 2025-12-17 21:20:41 +08:00
fzyzcjy
3e690cce53 Add realtime token counter metrics (#15198) 2025-12-17 21:20:24 +08:00
fzyzcjy
b12b40dea4 Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189) 2025-12-17 21:19:42 +08:00
Xiaoyu Zhang
6c4bf8a0be [diffusion] profiling: enhance trace export with gzip and integrity check (#15326) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-17 20:44:55 +08:00
Xiaoyu Zhang
533851fbcb [diffusion] ci: add flux2 tp2 test into ci to avoid breaking tensor parallel (#15237) 2025-12-17 20:44:21 +08:00
Xuchun Shang
0071fe9c40 [bug fix][pp] fix weight load for qwen2.5-vl (#15138) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-17 18:33:50 +08:00
b8zhong
ffa7e03506 [Piecewise CUDA Graph] Support INT8 (#14918) 2025-12-17 18:20:57 +08:00
LHXuuu
712f44ee2b fix qwenvl compressed tensors quantization weight loader (#11914) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
 2025-12-17 18:01:43 +08:00
kk
8c34e18140 Fix the accuracy issue when running mxfp4 dsv3 model and enable ep (#15304) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-12-17 01:53:22 -08:00
Xuchun Shang
e9abb52576 [bug fix][pp] fix qwen3 model load (#15223) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-17 17:26:22 +08:00