kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-17 09:17:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,459 Commits 157 Branches 0 Tags
support_multi_protocol
Commit Graph

8284 Commits

Author SHA1 Message Date
Mick
4323fce82a fix: dot-boundary match in is_layer_skipped for FP8 modules_to_not_convert (#23467) 2026-04-22 22:16:22 +08:00
Shangming Cai
1c06a3d072 [CI] Move disaggregation basic CI back to 2-gpu suite (#23447) 2026-04-22 17:50:33 +08:00
Ming Yang
7b10f01d1c [model_runner] Label forward steps in profile traces with mode and token counts (#23419) 2026-04-22 02:31:18 -07:00
inkcherry
1e34cd0ba5 PD streaming: batch notify + SSE fast path (#22658) 2026-04-22 02:21:02 -07:00
Fengyuan Yu
5c245d978f [Diffusion] Add mixed-resolution benchmark support (for #20762) (#20863) 
Signed-off-by: Fengyuan Yu <15fengyuan@gmail.com>
Co-authored-by: Fengyuan Yu <15fengyuan@gmail.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-04-22 09:22:19 +03:00
cctry
e39f0f4ff3 Use libdevice tanh and support 2D-strided tensors in fused softcap kernel (#23157) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-21 22:54:37 -07:00
Wenxuan Tan
c3ea2d7b92 Rename mixed_with_decode_tokens in mixed chunk prefill adder (#6506) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-04-21 22:48:34 -07:00
Tarushii Goel
7607e4d180 py-spy without --native for ARM devices (#23410) 2026-04-21 20:45:52 -07:00
shuwenn
4befc31408 fix: pass v_head_dim to MHA KV pools and validate MiMo HiCache geometry (#23173) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-21 19:48:45 -07:00
MARATRIX
bf5e71dcec [MUSA][19/N] Support HiCache with pin_memory allocator (#23361) 
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
 2026-04-21 19:45:53 -07:00
Piotr Mazurek
6cf0b004ca [MoE] Add LFM2 MoE tuning support + tuned configs for H100/B200/MI325X (#22791) 
Co-authored-by: Piotr Mazurek <piotr.mazurek@liquid.ai>
 2026-04-21 18:32:05 -07:00
Byron Hsu
c090f71bf2 feat: enable SGLANG_PATCH_TOKENIZER by default (#23409) 2026-04-21 17:53:43 -07:00
hlu1
415f64e763 Add MambaPool kvcache offloading during retraction (#22493) 2026-04-22 08:51:03 +08:00
Qiaolin Yu
c560326884 [perf] support return_routed_experts with overlap scheduling (#22911) 
Co-authored-by: Yuzhen Zhou <82826991+zyzshishui@users.noreply.github.com>
 2026-04-21 14:42:49 -07:00
Yanbin Jiang
4f764dfbb8 [Lora] Support LoRA and multi-batch in bench_one_batch_server (#23047) 2026-04-21 14:20:11 -07:00
Charles Chen
c396e4924b [bug] Fix cache salt and extra keys for prefix cache isolation (#23300) 2026-04-21 13:53:24 -07:00
shuwenn
e3782d04d2 fix: fallback to triton for attention-sink models (flashinfer unsupported) (#23139) 
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-04-21 13:48:50 -07:00
Liangsheng Yin
6c2714f5ae guard adaptive speculative against unsupported configs (#23289) 2026-04-21 13:47:34 -07:00
wxzhoucs
5273f11fd8 [PD] Resolve missing bootstrap_room problem about fake-decode in load-balance method (#18399) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-04-21 13:47:01 -07:00
Yuan Luo
48daa831ea [KDA] Fuse gate+cumsum and reuse chunk index for KDA (#23038) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-04-21 17:54:20 +08:00
Alan Kao
8589b92a89 [AMD] Fused qk rmsnorm bf16 for amd/Kimi-K2.5-MXFP4 (#23186) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-21 02:35:09 -07:00
ybyang
fa8993111d Fix: Add token heuristic increment in total_tokens load balancing (#22614) 2026-04-21 01:29:33 -07:00
Xiaoyu Zhang
0d69012ef8 Optimize LTX2 feed-forward tensor parallelism (#23221) 2026-04-21 16:29:23 +08:00
huangtingwei
efa71ce5ab [HiCache]Fix hybrid model move_indices (#22940) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: flyerming <flyerming@163.com>
 2026-04-21 00:15:39 -07:00
jianan-gu
2cf3ac515b [Diffusion][CPU] Init CPU platform support for SGLang Diffusion (#20816) 2026-04-21 14:25:54 +08:00
Liangsheng Yin
2b2cad70d6 [Refactor] Move radix-cache utils onto RadixKey as methods (#23209) 2026-04-20 23:11:58 -07:00
Liangsheng Yin
a490632416 Opt-in strip of thinking tokens from radix cache (#23315) 
Co-authored-by: ianliuy <ianl@alumni.usc.edu>
Co-authored-by: Wen-xuan-Xu <lilmeep727@gmail.com>
 2026-04-20 22:59:50 -07:00
jsheng_Linkedin
a8e3a534a4 [Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544) 
Co-authored-by: Chanh Nguyen <chanhnguyen@gmail.com>
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
 2026-04-20 22:50:40 -07:00
Shangming Cai
a58c7f381e [PD] Fix clip logic when state indices lens are mismatch (#23323) 2026-04-21 13:22:20 +08:00
Yuhao Yang
5595f6e988 Fix trtllm mla chunked-prefill zero-length bug (#22291) (#22688) 2026-04-20 22:10:13 -07:00
Liangsheng Yin
6cc2eee50d [misc] CI hygiene: enforce __main__ entry, drop silent-skipped tests, fix rerun-test protoc (#23305) 2026-04-20 21:16:24 -07:00
Lewis
0d0405273b [Fix] Solve the error lead by _commit_transfer_to_req() when using IntraNode NVLink in PD disaggregation (#23252) 
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
 2026-04-21 11:02:18 +08:00
Ke Bao
50fc2c9e23 Fix hybrid swa chunked prefill oom (#23174) 2026-04-21 10:46:45 +08:00
Zhangheng
ab3ce02de9 [Hybrid-Cache]: Refactor hybrid_pool_assembler.py (#23243) 2026-04-21 10:45:23 +08:00
ishandhanani
3c007ee5d4 fix(hicache): emit KV events for L2 host cache insertions (#22894) 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
 2026-04-20 19:07:03 -07:00
ChangLiu0709
ac08ebed65 [AMD] Resolve Qwen3.5 MTP (speculative decoding) radix cache conflict. (#22908) 2026-04-20 18:17:11 -07:00
Liangsheng Yin
c7a4ebf3c8 [Refactor] Replace page_align_keys helper with RadixKey.page_aligned method (#23107) 2026-04-20 18:10:42 -07:00
Tarushii Goel
3e367f9bcd [sgl] fix incorrect behavior in cuda graph draft extend (#22832) 2026-04-20 16:29:16 -07:00
Tarushii Goel
100b0f86dd [sgl] add support for weight update function in spedec (#22088) 2026-04-20 16:26:20 -07:00
Tarushii Goel
28f3a2d8ed [sgl] multilayereagleworkerv2 fix (#22954) 2026-04-20 16:22:16 -07:00
Thomas Wang
57ecce9807 [AMD] Enable MTP for GLM-5-mxfp4 model (#23219) 2026-04-20 16:09:07 -07:00
jsheng_Linkedin
575fdc2c4c [CI][LoRA] Drop flaky all-None batch from multi-LoRA parity test (#23287) 2026-04-20 14:43:25 -07:00
shuwenn
b65799cf83 [SPEC][1/N] feat: add adaptive speculative_num_steps for EAGLE topk=1 (#21599) 
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2026-04-20 14:25:04 -07:00
shuwenn
dbcf7459b5 fix: reset empty prefill batch fullness (#23138) 2026-04-20 14:14:00 -07:00
Liangsheng Yin
8cb957ccff [Perf] Make EAGLE bigram key an O(1) view on RadixKey (#23106) 2026-04-20 12:01:11 -07:00
Shunkangz
3dc1491c95 Support moe_dp_size = 1 for various attention_cp_size (#22003) 
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
 2026-04-20 11:58:19 -07:00
Lee Nau
b4bb036b73 fix legacy deepep path for flashinfer_cutedsl (#22925) 2026-04-20 11:49:33 -07:00
ishandhanani
b5d9a86e4c fix: add back priorty as radix cache policy (#23275) 2026-04-20 10:04:35 -07:00
Makcum888e
39c720d1b9 [Diffusion][NPU][CI] update perf numbers (#23056) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-04-20 19:34:11 +03:00
Mick
9a0fd2ff0c [diffusion] optimize: default to in-memory loading for URL/base64 image inputs (#23118) 2026-04-20 23:29:02 +08:00