kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-13 10:56:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,909 Commits 157 Branches 0 Tags
18359aadc8a456397fb102455edaf64b5ec1192e
Commit Graph

7901 Commits

Author SHA1 Message Date
Kangyan-Zhou
18359aadc8 [CI] Lower GSM8K baselines for B200 nightly after eval unification (#22136) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-22 22:30:54 -07:00
HuangJi
9716599383 [diffusion] fix: avoid illegal memory access in qwen image (#22953) 2026-04-23 12:41:26 +08:00
Mick
4d3c7e781a [diffusion] CI: do not retry consistency failures (#23517) 2026-04-23 12:39:37 +08:00
maocheng23
d3aa9128be Change SGLANG_SIMULATE_ACC_METHOD to 'match-expected' (#23527) 2026-04-22 21:26:08 -07:00
Jimmy Shong
68a8ed9b11 [Fix/Kernel] Add JIT rmsnorm_hf kernel to fix transformers backend MMLU accuracy regression (#22931) 
Co-authored-by: SGLang CI <ci@sglang.ai>
 2026-04-23 12:00:31 +08:00
Liangsheng Yin
0f21fe924a fix ngram greedy verify kwarg (#23521) 2026-04-22 20:49:54 -07:00
ori
887d380ace [MUSA] Resolve output garbage in Context Parallel on MusaFlashAttentionBackend (#23270) 
Co-authored-by: zhiguo.qin <zhiguo.qin@mthreads.com>
 2026-04-22 20:22:20 -07:00
Liangsheng Yin
f611dd24f1 fix retrive -> retrieve typo (#23503) 
Co-authored-by: SoluMilken <19161836+solumilken@users.noreply.github.com>
 2026-04-22 16:35:04 -07:00
Yanbin Jiang
917d2aa1dc [LoRA] Fix EP + per-expert MoE LoRA illegal memory access (#23178) 2026-04-22 14:22:32 -07:00
Sam Shleifer
b9e33d6a5b Dual MoE CUDA graph capture for lora/nolora batches (#22809) 2026-04-22 14:11:11 -07:00
jianan-gu
ad0fc88810 [CPU] [Quantization] Add GPTQ/AWQ 4bits quantization support for CPU (#22685) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-22 13:34:02 -07:00
Byron Hsu
0b77284587 [minor] Make DEFAULT_FORCE_STREAM_INTERVAL configurable via SGLANG_FORCE_STREAM_INTERVAL (#23215) 2026-04-22 13:05:40 -07:00
JasonHe-WQ
f85e3140bf Fix:fix(timeout): fix timeout not propagated (#21944) 2026-04-22 12:48:48 -07:00
Yuxuan Zhang
28cfd3d272 Support defer_loading field at function level for Chat Completions API (#22702) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-04-22 10:09:54 -07:00
Todobe
92f28e9ba8 [NPU]Fix GLM-4.7-Flash failed on NPU (#22509) 2026-04-23 01:06:58 +08:00
cctry
0addd185af Fix /generate endpoint crash when sampling params contain null values (#23401) 2026-04-22 09:56:10 -07:00
Aleksi Vesanto
ac351c1f04 [diffusion] [AMD] model: allow AITER backends in Flux 2 pipeline (#22802) 2026-04-22 08:15:44 -07:00
Shenxiu Liu
8b78e0888c Skip mamba_pool_idx revert for session requests in _get_new_batch_prefill_raw (#23327) 2026-04-22 22:28:06 +08:00
Mick
4323fce82a fix: dot-boundary match in is_layer_skipped for FP8 modules_to_not_convert (#23467) 2026-04-22 22:16:22 +08:00
Shangming Cai
1c06a3d072 [CI] Move disaggregation basic CI back to 2-gpu suite (#23447) 2026-04-22 17:50:33 +08:00
Ming Yang
7b10f01d1c [model_runner] Label forward steps in profile traces with mode and token counts (#23419) 2026-04-22 02:31:18 -07:00
inkcherry
1e34cd0ba5 PD streaming: batch notify + SSE fast path (#22658) 2026-04-22 02:21:02 -07:00
Fengyuan Yu
5c245d978f [Diffusion] Add mixed-resolution benchmark support (for #20762) (#20863) 
Signed-off-by: Fengyuan Yu <15fengyuan@gmail.com>
Co-authored-by: Fengyuan Yu <15fengyuan@gmail.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-04-22 09:22:19 +03:00
cctry
e39f0f4ff3 Use libdevice tanh and support 2D-strided tensors in fused softcap kernel (#23157) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-21 22:54:37 -07:00
Wenxuan Tan
c3ea2d7b92 Rename mixed_with_decode_tokens in mixed chunk prefill adder (#6506) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-04-21 22:48:34 -07:00
Tarushii Goel
7607e4d180 py-spy without --native for ARM devices (#23410) 2026-04-21 20:45:52 -07:00
shuwenn
4befc31408 fix: pass v_head_dim to MHA KV pools and validate MiMo HiCache geometry (#23173) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-21 19:48:45 -07:00
MARATRIX
bf5e71dcec [MUSA][19/N] Support HiCache with pin_memory allocator (#23361) 
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
 2026-04-21 19:45:53 -07:00
Piotr Mazurek
6cf0b004ca [MoE] Add LFM2 MoE tuning support + tuned configs for H100/B200/MI325X (#22791) 
Co-authored-by: Piotr Mazurek <piotr.mazurek@liquid.ai>
 2026-04-21 18:32:05 -07:00
Byron Hsu
c090f71bf2 feat: enable SGLANG_PATCH_TOKENIZER by default (#23409) 2026-04-21 17:53:43 -07:00
hlu1
415f64e763 Add MambaPool kvcache offloading during retraction (#22493) 2026-04-22 08:51:03 +08:00
Qiaolin Yu
c560326884 [perf] support return_routed_experts with overlap scheduling (#22911) 
Co-authored-by: Yuzhen Zhou <82826991+zyzshishui@users.noreply.github.com>
 2026-04-21 14:42:49 -07:00
Yanbin Jiang
4f764dfbb8 [Lora] Support LoRA and multi-batch in bench_one_batch_server (#23047) 2026-04-21 14:20:11 -07:00
Charles Chen
c396e4924b [bug] Fix cache salt and extra keys for prefix cache isolation (#23300) 2026-04-21 13:53:24 -07:00
shuwenn
e3782d04d2 fix: fallback to triton for attention-sink models (flashinfer unsupported) (#23139) 
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-04-21 13:48:50 -07:00
Liangsheng Yin
6c2714f5ae guard adaptive speculative against unsupported configs (#23289) 2026-04-21 13:47:34 -07:00
wxzhoucs
5273f11fd8 [PD] Resolve missing bootstrap_room problem about fake-decode in load-balance method (#18399) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-04-21 13:47:01 -07:00
Yuan Luo
48daa831ea [KDA] Fuse gate+cumsum and reuse chunk index for KDA (#23038) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-04-21 17:54:20 +08:00
Alan Kao
8589b92a89 [AMD] Fused qk rmsnorm bf16 for amd/Kimi-K2.5-MXFP4 (#23186) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-21 02:35:09 -07:00
ybyang
fa8993111d Fix: Add token heuristic increment in total_tokens load balancing (#22614) 2026-04-21 01:29:33 -07:00
Xiaoyu Zhang
0d69012ef8 Optimize LTX2 feed-forward tensor parallelism (#23221) 2026-04-21 16:29:23 +08:00
huangtingwei
efa71ce5ab [HiCache]Fix hybrid model move_indices (#22940) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: flyerming <flyerming@163.com>
 2026-04-21 00:15:39 -07:00
jianan-gu
2cf3ac515b [Diffusion][CPU] Init CPU platform support for SGLang Diffusion (#20816) 2026-04-21 14:25:54 +08:00
Liangsheng Yin
2b2cad70d6 [Refactor] Move radix-cache utils onto RadixKey as methods (#23209) 2026-04-20 23:11:58 -07:00
Liangsheng Yin
a490632416 Opt-in strip of thinking tokens from radix cache (#23315) 
Co-authored-by: ianliuy <ianl@alumni.usc.edu>
Co-authored-by: Wen-xuan-Xu <lilmeep727@gmail.com>
 2026-04-20 22:59:50 -07:00
jsheng_Linkedin
a8e3a534a4 [Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544) 
Co-authored-by: Chanh Nguyen <chanhnguyen@gmail.com>
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
 2026-04-20 22:50:40 -07:00
Shangming Cai
a58c7f381e [PD] Fix clip logic when state indices lens are mismatch (#23323) 2026-04-21 13:22:20 +08:00
Yuhao Yang
5595f6e988 Fix trtllm mla chunked-prefill zero-length bug (#22291) (#22688) 2026-04-20 22:10:13 -07:00
Liangsheng Yin
6cc2eee50d [misc] CI hygiene: enforce __main__ entry, drop silent-skipped tests, fix rerun-test protoc (#23305) 2026-04-20 21:16:24 -07:00
Lewis
0d0405273b [Fix] Solve the error lead by _commit_transfer_to_req() when using IntraNode NVLink in PD disaggregation (#23252) 
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
 2026-04-21 11:02:18 +08:00