kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 16:27:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,998 Commits 157 Branches 0 Tags
9749d3e346c50135bfaffe60a5d3beba88f327aa
Commit Graph

5224 Commits

Author SHA1 Message Date
Frank
9749d3e346 Update benchmarks to use HF token from environment. (#15421) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-18 13:47:27 -08:00
Baizhou Zhang
2b0ddf89f5 [Tiny]Add warning for deepgemm on Blackwell (#15352) 2025-12-18 13:45:26 -08:00
fzyzcjy
88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) 2025-12-18 22:24:23 +08:00
fzyzcjy
602fe3b296 Super tiny add moe_ep_rank to prometheus labels (#15407) 2025-12-18 22:23:20 +08:00
fzyzcjy
ad9616f13a Tiny extract ModelRunnerOutput (#15400) 2025-12-18 22:18:45 +08:00
fzyzcjy
c5f4e20f2f Support GPU execution time breakdown by forward mode metrics (#15396) 2025-12-18 22:18:17 +08:00
fzyzcjy
2c196f95a7 Tiny fix unknown route in prometheus metrics (#15404) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-18 22:17:46 +08:00
Xiaoyu Zhang
9a7641d7bf [diffusion] profiling: include per-denoising-step timings in perf-dump-path (#15397) 2025-12-18 18:43:33 +08:00
Bruce-x-1997
793c96c3d2 [perf]optimize w4afp8 kernel on deepseek-v3-0324 (#12921) 
Signed-off-by: bruce.xu <bruce.x@gmicloud.ai>
 2025-12-18 18:13:22 +08:00
Lianmin Zheng
d1f0063262 Clean up __init__ function of the scheduler and event loop for PD (#15298) 2025-12-18 01:35:14 -08:00
Xuchun Shang
fea2d5211d [bug fix][pp] fix inconsistent latency between tp (#15379) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 16:59:56 +08:00
Liangsheng Yin
374ad4cce0 Fix gpt-oss yarn with truncate argument (#14270) 2025-12-18 16:31:15 +08:00
Liangsheng Yin
8b0a68f1bf Monkey patch deepseek-ocr's v_head_dim (#15384) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-18 16:28:07 +08:00
Shangming Cai
ee1ca51de8 [PP] Fix dynamic chunking strategy for PP (#15372) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 14:24:55 +08:00
Yuhao Yang
3d42b7e7b0 unified management of environment variables for vlm cuda ipc transport (#14501) 2025-12-18 12:28:06 +08:00
elvischenv
9970ee34e8 Mistral Large 3 NVFP4 TRTLLM MoE support (#15049) 2025-12-18 11:11:42 +08:00
Xinyuan Tong
41683536d3 feat: DeepSeek-V3.2 Streaming tool call output (#15278) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: momaek <momaek17@gmail.com>
Co-authored-by: Muqi Li <muqi1029@gmail.com>
 2025-12-17 17:43:58 -08:00
Dongjie Zou
891ee8221f Feature/trtllm mha workspace size configurable #15089 (#15131) 2025-12-17 17:07:23 -08:00
Yuan Luo
8fa3dc36c5 [VLM] Support cos sin cache for Qwen3-VL & GLM-4.1V (#15205) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-18 08:39:50 +08:00
b8zhong
d20699a33c [Deepseek V3.2] Support Overlap Spec + NSA (#15307) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2025-12-17 13:35:39 -08:00
Scott Lee
169a75dfb8 Add request-level timestamp for when prefill finishes (#14860) 2025-12-17 13:34:46 -08:00
Alison Shao
4128d4f5cb [CI] Migrate LoRA tests to test/registered/lora/ (#15176) 2025-12-17 13:19:42 -08:00
Cheng Wan
5290cef97c Fix condition check for require_gathered_buffer (#15328) 2025-12-17 12:42:47 -08:00
许霖
726fe3e72e fix: prevent points regex from matching checkpoints/endpoints (#15120) 2025-12-17 12:38:18 -08:00
kk
5d087891c9 Fix dp run error with fp8-kv enable in high concurrency test (#15241) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-12-17 11:49:18 -08:00
Liangsheng Yin
0c00220795 tiny unify environ usage (#15335) 2025-12-17 23:31:43 +08:00
Shangming Cai
eeb2b9b259 [PP] Minor code cleanup for Pipeline Parallelism (#15329) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-17 23:31:25 +08:00
fzyzcjy
9d04b570bc Tiny dump native stacktraces in watchdog (#15222) 2025-12-17 21:20:41 +08:00
fzyzcjy
3e690cce53 Add realtime token counter metrics (#15198) 2025-12-17 21:20:24 +08:00
fzyzcjy
b12b40dea4 Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189) 2025-12-17 21:19:42 +08:00
Xiaoyu Zhang
6c4bf8a0be [diffusion] profiling: enhance trace export with gzip and integrity check (#15326) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-17 20:44:55 +08:00
Xiaoyu Zhang
533851fbcb [diffusion] ci: add flux2 tp2 test into ci to avoid breaking tensor parallel (#15237) 2025-12-17 20:44:21 +08:00
Xuchun Shang
0071fe9c40 [bug fix][pp] fix weight load for qwen2.5-vl (#15138) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-17 18:33:50 +08:00
b8zhong
ffa7e03506 [Piecewise CUDA Graph] Support INT8 (#14918) 2025-12-17 18:20:57 +08:00
LHXuuu
712f44ee2b fix qwenvl compressed tensors quantization weight loader (#11914) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
 2025-12-17 18:01:43 +08:00
kk
8c34e18140 Fix the accuracy issue when running mxfp4 dsv3 model and enable ep (#15304) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-12-17 01:53:22 -08:00
Xuchun Shang
e9abb52576 [bug fix][pp] fix qwen3 model load (#15223) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-17 17:26:22 +08:00
kk
888594333e Fix gpu-fault when running mtp in eager mode (#15233) 2025-12-17 01:25:06 -08:00
elvischenv
feb8e30b9d [Hotfix] Fix required enable_mamba_track argument for Flashinfer autotune path (#15314) 2025-12-17 17:03:24 +08:00
Xuchun Shang
45a959d3e9 [PP] Add pp support for Qwen3-VL (#12333) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Kun(llfl) <llfl@linux.alibaba.com>
Co-authored-by: kun-llfl <i@imux.top>
Co-authored-by: Kun(llfl) <llfl@linux.alibaba.com>
 2025-12-17 16:03:58 +08:00
WenhaoZhang
cdce516331 [diffusion] api: add sampling parameters and model info endpoint to OpenAI API (#15071) 
Co-authored-by: niehen6174 <niehen.6174@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: niehen6174 <nihen6174@gmail.com>
 2025-12-17 15:33:18 +08:00
b8zhong
79ab57bd7a Revert "direct register custom op for mm_fp4 (#13699)" (#15284) 2025-12-16 23:09:44 -08:00
elvischenv
435d1c83c1 [Perf] Enable Flashinfer autotune by default (#14357) 2025-12-16 23:01:39 -08:00
Alison Shao
31d48d7f6f Add Ollama-compatible API endpoints + Smart Router (#14376) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-12-16 20:43:38 -08:00
luqitao
0129c911e0 fix(function_call): fallback to decode when batch decode options differ (#15155) 
Signed-off-by: luqitao <luqt@awcloud.com>
 2025-12-16 20:21:08 -08:00
ehuaa
03f9eb2564 [BugFix] fix gptq_marlin_gemm has no parameter called b_bias (#13571) 
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
Co-authored-by: Fan Yin <1106310035@qq.com>
 2025-12-17 10:03:55 +08:00
Lianmin Zheng
9d64a7b24f Minor style fixes to the scheduler.py (#15218) 2025-12-16 17:09:44 -08:00
Even Zhou
71cb90378b [NPU] fix for NPU memory settings logic (#15258) 2025-12-16 17:04:22 -08:00
Trevor Morris
c8c64876a7 [NVIDIA] Fixes for NVFP4 all-gather with spec decoding (#15280) 2025-12-16 16:58:39 -08:00
Yineng Zhang
0861dca81f Revert "[misc] Upgrade cutedsl to 4.3.1 (#14857)" (#15293) 2025-12-16 16:31:32 -08:00