kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-17 09:17:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,459 Commits 157 Branches 0 Tags
support_multi_protocol
Commit Graph

8284 Commits

Author SHA1 Message Date
Qiaolin Yu
d8831355a3 Fix multi_layer_eagle_worker_v2 draft extend selection, add chain style multi layer mtp test (#22340) 
Co-authored-by: 0xNullPath <luyan@nvidia.com>
 2026-04-10 12:44:52 -07:00
Trevor Morris
7dbd0dd9f0 MiniMax-M2.5 - Support dp attention, dp reduce scatter, FP4 all gather, AR fusion in prepare_attn (#20067) 2026-04-10 12:41:27 -07:00
KrishnanPrash
a937ec31be fix: server crash when stop_token_ids contains null (#22175) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
 2026-04-10 11:42:23 -07:00
Jia Guo
5cb4ea1d4d perf: enable inductor combo_kernels for horizontal fusion (#21977) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-11 01:01:14 +08:00
Tarushii Goel
2ba94136ce [sgl] improve mamba_track_indices perf in specdec (#22380) 2026-04-11 00:39:53 +08:00
Bi Xue
f652135d52 [sgl] fix using symmetric memory issues for attention_tp (#22286) 2026-04-11 00:26:18 +08:00
Ratish P
8227187d47 [SKILL]: add component accuracy guidance to the diffusion add-model skill (#22460) 2026-04-10 23:08:31 +08:00
Ratish P
cf5ad12612 [diffusion][CI]: route multimodal component accuracy through run_suite (#21960) 2026-04-10 23:06:03 +08:00
kingkingleeljj
84194c25c1 [BugFix] fix the bug of minimax_m2.5 model that causes repeated outputs when using tp16 (#20967) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-10 22:21:19 +08:00
Xiaoyu Zhang
1ff51555f2 [Diffusion] modelopt diffusion fp8 support for flux1/flux2 and wan2.2 (#22365) 2026-04-10 20:56:57 +08:00
Yujun Dong
8ba9646044 Make GDN support non-continuous B/A Tensor input to fix the accuracy regression of Qwen3.5-27B (#22312) 
Signed-off-by: cs-cat <118669451+cs-cat@users.noreply.github.com>
 2026-04-10 18:58:13 +08:00
Jincong Chen
0668a7f51a [Perf] Remove two operations in gdn_backend extend verify path (#22444) 2026-04-10 17:53:57 +08:00
Shangming Cai
1c76f322df [HiCache] Add CP support for HiCache (#20977) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-04-10 17:52:51 +08:00
Cheng Wan
37107bee6f [Observability] Add pending token count to prefill log and get_load (#22480) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-10 02:05:21 -07:00
Lee Nau
c554dc5c64 Add dedicated FlashInferCuteDslMoE layer for standard-path FP4 MoE (#21339) 2026-04-10 01:35:56 -07:00
Mick
7c6b5c095c [diffusion] fix: fix flux2 i2i accuracy (#22423) 2026-04-10 16:16:51 +08:00
Liangsheng Yin
6cf7f210bf Add page_size to admission token budget check (#22495) 2026-04-10 01:16:04 -07:00
Jacob0226
dd41764487 [AMD][HIP] NSA: bf16 passthrough from RMSNorm to eliminate FP8 dequantization (#22258) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-10 01:08:32 -07:00
jianan-gu
2ab141547d [CPU] Add apply_routed_scaling_factor_on_output support for biased_grouped_topk fusion (#22413) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-10 15:16:05 +08:00
Polisetty V R K Jyothendra Varma
599cce4d82 [Intel GPU] import flash_attn functions from sgl_kernel only (#22438) 2026-04-10 15:10:00 +08:00
xieminghe1
18f41ac427 [Reland] DeepSeek-R1-0528-w4a8: DeepEP Low Latency Dispatch Adopts FP8 Communication (#22316) 
Co-authored-by: undefined <zhouchen.arrebol@jd.com>
Co-authored-by: xq25478 <xq25478@qq.com>
 2026-04-10 14:56:05 +08:00
Tarushii Goel
0334d4b7e8 [sgl] Fix mamba tracking calculation in spec dec (#22239) 2026-04-10 14:46:16 +08:00
Ethan (Yusheng) Su
6d79c60995 [Lora] Lora kimi support (#22381) 2026-04-09 22:31:53 -07:00
Liangsheng Yin
722e25a621 Fix SWA eviction boundary and page-align chunked prefill (#22470) 2026-04-09 22:09:43 -07:00
Ke Bao
e77bfba24d Fix NCCL AllGather hanging issue for Qwen3 Next MTP (#22458) 2026-04-10 11:40:54 +08:00
Kangyan-Zhou
89553ff82b [Observability] Add Prometheus metrics endpoint for gRPC mode (#20801) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 20:04:54 -07:00
LHXuuu
42ffb168b3 [EPD][VLM] Support Kimi K25 EPD (#22269) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-04-10 10:58:42 +08:00
jacky.cheng
d283808457 [AMD] Replace triton rotary_emb with aiter rotary_emb for Wan2.2 denoise (#22422) 2026-04-09 18:21:02 -07:00
Shu Wang
5638d40f3a [nvidia] Gemma4 nvfp4 fix (#22079) 2026-04-10 08:44:34 +08:00
Tarushii Goel
cebd9c2a1e [sgl] add ability to return logprobs in MultiLayerEagleWorkerV2 (#22241) 2026-04-09 16:20:55 -07:00
Mohammad Miadh Angkad
c3833ba929 Enable DFLASH support for additional model backends (#22358) 
Co-authored-by: David Wang <21328423+dcw02@users.noreply.github.com>
 2026-04-09 14:36:12 -07:00
Ethan (Yusheng) Su
28ef6de091 [Lora] Lora quat info re-factor and support deepseekv3 mla lora (#22323) 2026-04-09 14:19:58 -07:00
Baizhou Zhang
60acdc31f2 [Fix] Fix several bugs on DSA models (#22430) 2026-04-09 12:46:23 -07:00
Baizhou Zhang
606aa11ea8 [DSA] Enable all reduce fusion for DSA models (#22390) 2026-04-09 12:42:44 -07:00
Lawrence Wu
8eb235ab51 fix: do not strip whitespace from GLM tool call values (#20543) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-04-09 11:14:15 -07:00
Lishan H
8b991d98a1 [feature] asr: add chunk-based streaming ASR for Qwen3-ASR (#22089) 2026-04-10 01:49:03 +08:00
billishyahao
1df9f4e2f6 [AMD] Add prealloc token env for mori-ep (#22329) 2026-04-09 09:34:35 -07:00
Jonah Bernard
8216b921a1 Add MLX profiling to bench_one_batch.py (#22159) 2026-04-09 20:45:21 +08:00
Liangsheng Yin
9fed58805f [Doc] Clarify SWA HybridSWAPoolConfigurator comments on all-SWA vs hybrid semantics (#22443) 2026-04-09 03:02:16 -07:00
YMbmzy
8a67fb20ea [Speculative] Support penalty for spec v2 overlap scheduling (#22049) 2026-04-09 01:59:04 -07:00
Thomas Wang
628df31d08 [AMD] Use aiter CK layernorm2d for LayerNorm to reduce NSA indexer kernel launches (#22424) 2026-04-09 01:55:29 -07:00
xutizhou
57ffc55fb6 feat: [1/2] [DeepEP] Fuse shared expert into MoE dispatch under EP (#20089) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: AichenF <aichenf@nvidia.com>
 2026-04-09 01:48:28 -07:00
Mick
9709192ce9 [diffusion] feat: support FLUX.2-small-decoder (#22414) 2026-04-09 15:53:14 +08:00
Liangsheng Yin
de441ac6bb [core] Introduce MemoryPoolConfigurator class hierarchy (#22389) 2026-04-09 15:29:19 +08:00
Evgueni Petrov
b9c316917b fix AttributeError: 'LazyValue' object has no attribute 'keys' in eplb_manager.py for qwen3 moe (#21822) 2026-04-09 00:13:29 -07:00
Nicolas Castet
e379befbac Add symmetric debug mode to print stack trace of comm ops with unregistered tensors (#18569) 2026-04-08 22:34:58 -07:00
Bingxu Chen
6b96f8341d [AMD] Fix multimodal diffusion test crash on ROCm by falling back to SDPA (#22335) 2026-04-08 22:32:49 -07:00
Mick
355fcbcc17 [diffusion] fix: fix cache dit refresh none mask (#22374) 2026-04-09 11:58:24 +08:00
jsheng_Linkedin
6838a23226 [Feature] Add token embedding overrides for sparse embedding replacement (#20960) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 20:51:36 -07:00
Kurkur
a69be2e866 [Feature] Support eagle3 for qwen3-vl (#22230) 2026-04-09 11:45:36 +08:00