kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-11 17:59:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,308 Commits 157 Branches 0 Tags
ebd64f5d40e20a77b35ca6fae9feaf1096737d91
Commit Graph

8164 Commits

Author SHA1 Message Date
fzyzcjy
ebd64f5d40 Support user-supplied recv-side transform in dumper grafter (#24509) 2026-05-06 16:56:52 +08:00
fzyzcjy
9a65f0ac26 Support t2b direction and overlap protection in dumper grafter (#24508) 2026-05-06 16:56:24 +08:00
fzyzcjy
58487e68e5 Support cross-system tensor grafting in dumper (#24507) 2026-05-06 16:55:40 +08:00
fzyzcjy
61104d7d0a Add prefixed _log helper in dumper (#24506) 2026-05-06 16:54:20 +08:00
Mick
fbebfdec9a [diffusion] fix: fix diffusion FSDP sharding (#24431) 2026-05-06 14:55:51 +08:00
cctry
660a77f221 Silence noisy health-check race log in TokenizerManager (#24466) 2026-05-05 21:06:43 -07:00
ybyang
3da87902d7 [HiSparse] Support FP8 KV cache by routing to flashmla_kv backend (#23013) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-05-06 03:18:30 +00:00
Night
b2420d72ff [RL] DeepEP support for --enable-return-routed-experts (#16859) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Junrong Lin <33685709+ocss884@users.noreply.github.com>
 2026-05-05 20:01:07 -07:00
Xiaoyu Zhang
d7385b575f [Diffusion] Optimize Hunyuan3D shape denoising (#24287) 2026-05-06 10:10:09 +08:00
Jianhong Zhang
c7019ff33d [NIXL][XPU] Use np.uint64 for pointer/length arrays in disaggregation KV transfer (#24188) 2026-05-06 10:09:03 +08:00
Lianmin Zheng
b91b05ae27 Add --random-input-len to send_one.py (#24464) 2026-05-05 17:49:33 -07:00
cctry
22cf7d2b42 [Fix] Handle nixlRemoteDisconnectError in NixlKVSender (#24296) 2026-05-05 17:23:42 -07:00
Lianmin Zheng
64f80eabbe Register aten::rms_norm and aten::mm.dtype in batch invariant mode (#24459) 2026-05-05 17:21:34 -07:00
Lianmin Zheng
46bde1f426 Add fwd_occupancy metric to SchedulerStats and Prometheus collector (#24458) 2026-05-05 17:04:34 -07:00
Xinyuan Tong
1e404afec2 fix(req_pool): bump pool.size to match actual tensor row count after #24243 (#24439) 2026-05-05 16:58:26 -07:00
Lianmin Zheng
710fed10fb Revert "[fix] /pause_generation and /continue_generation wrong for --tokenizer-worker-num > 1" (#24461) 2026-05-05 16:44:34 -07:00
maocheng23
431ca54334 [fix] /pause_generation and /continue_generation wrong for --tokenizer-worker-num > 1 (#24445) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-05 16:27:58 -07:00
Liangsheng Yin
08d4c2072b move topk capturers to srt/state_capturer/ (#24450) 
Co-authored-by: Yueming Yuan <yym022502@gmail.com>
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
Co-authored-by: Ziang Li <ziangli@umich.edu>
 2026-05-05 15:54:01 -07:00
Liangsheng Yin
47a416fc62 add indexer-topk capture (V3.2 NSA + infra) (#24392) 2026-05-05 15:05:15 -07:00
Liangsheng Yin
c4c0376fcb consolidate routed-experts capturer onto reusable base (#24403) 2026-05-05 12:41:49 -07:00
Mick
d23ef408f7 [diffusion] fix: fix RowParallel LoRA merged forwarding (#24410) 2026-05-06 00:30:16 +08:00
Mick
cc54d8e8d0 [diffusion] chore: clean CUDA cache only at explicit release points (#24397) 2026-05-05 22:30:43 +08:00
Polisetty V R K Jyothendra Varma
fdfc46f3a5 [Intel GPU] Enable DeepSeek V3.2 inference on XPU (#24356) 
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
 2026-05-05 20:47:40 +08:00
Zhangheng
e299ec1bff [UnifiedRadixTree]: Fix flaky ci (#24421) 2026-05-05 20:22:19 +08:00
Khoa Pham
d22853480d Fix deterministic inference on models with SWAKVPool (#24395) 2026-05-05 20:20:46 +08:00
Bi Xue
9fb9a1cca6 [sgl] expose swa and mamba cache metrics (#24396) 2026-05-05 20:19:50 +08:00
Xiaoyu Zhang
67e8bd7a80 [codex] Optimize Helios fused norm modulation (#24059) 2026-05-05 19:28:37 +08:00
Xiaoyu Zhang
8c703f215e Add HunyuanVideo ModelOpt FP8 diffusion support (#23199) 2026-05-05 19:27:28 +08:00
billishyahao
80ccb6b93c [AMD] fix tbo specv2 seq_lens_cpu NoneType error (#24319) 2026-05-05 01:54:43 -07:00
Mick
177babcc38 [diffusion] optimize: fuse LTX2 split rotary embedding (#24411) 2026-05-05 16:07:40 +08:00
Hubert Lu
c2db19ffa4 [AMD] Enable EAGLE speculative decoding for Qwen3.5 FP8 and MXFP4 models with aiter's unified attention (#23146) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: sogalin <39478626+sogalin@users.noreply.github.com>
 2026-05-05 00:09:40 -07:00
Mick
04926e1d9f [diffusion] feat: cache encoder results for default negative prompt (#24304) 2026-05-05 11:56:01 +08:00
Mick
e483e60b72 [diffusion] CI: pin diffusion consistency GT revision (#24400) 2026-05-05 11:53:22 +08:00
Ethan (Yusheng) Su
2b769d37a4 (2/n - prefill optimize)perf(lora): remove GPU-CPU sync barrier (.item()) in MoE LoRA path and remove duplicate code (#24246) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-04 18:11:28 -07:00
Mick
2f7d99b7f7 [diffusion] cli: support component attention backend overrides (#24320) 2026-05-05 08:39:27 +08:00
Xiaoyu Zhang
078f84d80d [SKILL] Add diffusion benchmark presets for edit and Hunyuan3D models (#24288) 
Co-authored-by: BBuf Codex <bbuf-codex@users.noreply.github.com>
 2026-05-05 08:18:12 +08:00
Ji Zeng
4b487ca98b [Fix] NGRAMWorker.update_weights_from_tensor — delegate to target worker (#24344) 2026-05-04 16:23:17 -07:00
Liangsheng Yin
6a62eabed6 consolidate NSA pool construction (#24389) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-05-04 16:04:31 -07:00
Tarushii Goel
d7c93e183b [sgl] reduce specdec cpu overhead (#23321) 2026-05-04 15:02:03 -07:00
Liangsheng Yin
4743cf6051 misc: add marlin to moe runner choices; drop dead env var doc (#24384) 
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2026-05-04 15:01:47 -07:00
Lianmin Zheng
29dd3a36c0 Refactor device timer installation and rename prefill prealloc to bootstrap (#24341) 2026-05-04 13:57:13 -07:00
Vladislav Nosivskoy
60a1dacd89 [HiCache] return cached_tokens_details in sglext for streaming responses (#22055) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
 2026-05-04 12:30:17 -07:00
Ke Bao
6dd7aebb36 Minor scheduler fixes (#24359) 2026-05-05 02:01:23 +08:00
Sam Shleifer
e6f252e9b8 Cache FlashInfer autotune configs (#24156) 2026-05-05 02:00:40 +08:00
Yuan Luo
e5c58eb9d6 [VLM] Optimize Gemma4 VLM with PCG and fuse RMSNorm + residual add + scalar (#24048) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-05-04 09:36:26 -07:00
Mick
1be3163011 [diffusion] fix: use direct all-to-all for USP collectives (#24366) 2026-05-05 00:08:48 +08:00
Xiaoyu Zhang
4b6d44641b [diffusion] chore: enable channels-last 3D VAE convs by default (#23200) 2026-05-04 22:59:31 +08:00
Zhangheng
05aed5e1d5 [UnifiedRadixTree]: Add KL accuracy CI for UnifiedTree with HiCache (#24346) 2026-05-04 20:18:10 +08:00
Liangsheng Yin
84f3b44916 [tiny] misc cleanups across configs, attention, jit_kernel (#24350) 2026-05-04 03:17:14 -07:00
Linzhang Li
952b3caf18 feat: use structural tags to enable strict tool calling and reasoning for more models (#21722) 
Signed-off-by: Yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Ubospica <ubospica@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-05-04 02:30:28 -07:00