kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-11 09:47:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,342 Commits 157 Branches 0 Tags
42f8ea4030947bcb5431464370492daee6735e3d
Commit Graph

4233 Commits

Author SHA1 Message Date
Liangsheng Yin
42f8ea4030 [Test] Fix session control test (#12336) 2025-10-29 18:28:04 +08:00
Liangsheng Yin
14cbe42fd3 Refactor abortion in event loop (#12312) 2025-10-29 18:25:20 +08:00
Baizhou Zhang
685c06451f [ci] Try fixing broken CIs (#12317) 2025-10-29 01:13:51 -07:00
hlu1
42e1a72efb [Deepseek V3.2] Enable flashmla_auto with MTP (#12294) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-10-28 23:51:20 -07:00
b8zhong
83a7c89c3f followup fix for llama 4 trtllm flashinfer backend (#12314) 2025-10-28 22:17:08 -07:00
Yuzhen Zhou
0380ca82ef Add Batch‑Invariant RMSNorm (#12144) 2025-10-28 21:05:57 -07:00
Yingchun Lai
ec92b0cefe EPLB: prefer to use physical experts in the same gpu or node (#10874) 2025-10-28 21:01:11 -07:00
Liana Koleva
e03b6beeb1 doc: improve modelopt error description (#12269) 2025-10-28 20:57:55 -07:00
Yingchun Lai
5e36a0b455 [metrics][EPLB]: Support selected count of physical experts on each GPU (#9825) 2025-10-28 20:56:19 -07:00
Gao016
0297773a2f a tiny fix for support deepseek bf16 weights (#12313) 
Co-authored-by: gaochang <gaochang@U-19PX2WQ1-0350.local>
 2025-10-28 20:46:44 -07:00
Baizhou Zhang
587deb15a7 [hotfix] Fix pytest not found in CI (#12311) 2025-10-29 11:07:36 +08:00
Cheng Wan
83087247d1 [hotfix] missing w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization (#12259) 2025-10-28 19:10:38 -07:00
Xiaoyu Zhang
334543ff3b Add continuous_usage_stats support for streaming responses (#12241) 2025-10-29 10:01:23 +08:00
b8zhong
c143f416ce fix: Llama 4 BF16 load on Blackwell (#12308) 2025-10-28 18:59:01 -07:00
fzyzcjy
29195aaa6e Super tiny fix expert distribution dump error (#12271) 2025-10-28 15:20:55 -07:00
bmac3
8d6ab1cb88 fix seqlen bug for trtllm_mla's draft_extend (#12295) 2025-10-28 14:47:47 -07:00
b8zhong
77225d602a Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend (#11928) 2025-10-28 10:39:43 -07:00
Trevor Morris
fdd00295b5 Fix 'BypassedTopKOutput' object has no attribute 'topk_weights' for DeepEP (#12231) 2025-10-28 09:28:25 -07:00
Yineng Zhang
64cf868eba chore: cleanup quant deps (#12268) 2025-10-28 02:03:57 -07:00
Yineng Zhang
ea39952797 Revert "[Feature] PD-Multiplexing Context and Scheduler." (#12267) 2025-10-28 02:00:37 -07:00
Shangming Cai
41a113356a Fix potential eos bug on decode instance when PD is enabled (#12206) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-10-28 01:29:02 -07:00
Xuchun Shang
a1f2dc90e4 [Bug fix] [PP] fix wrong dtype for quantified model (#12247) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-10-28 01:27:24 -07:00
Feng Su
ea96106000 [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) 
Signed-off-by: Feng Su <sufeng@linux.alibaba.com>
 2025-10-28 01:25:46 -07:00
Cheng Wan
b1e13e7cea [hotfix] Incorrect CombineOverlapArgs in SBO (#12230) 2025-10-28 01:23:06 -07:00
Chenxi Li
cc7b04a29c Feature/Add GET endpoint to query loaded LoRA adapters (#12229) 2025-10-28 01:22:00 -07:00
fzyzcjy
691c8534cf Support releasing CUDA graph memory when paused (#7873) 
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
 2025-10-28 14:40:50 +08:00
Yongfei Xu
d2b8c4123e Opt fused triton moe: add tma for down proj kernel (#10567) 
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-10-28 14:26:17 +08:00
Scott Lee
bf8f7a944f Add per-request retraction count (#11177) 2025-10-27 23:22:34 -07:00
hlu1
81a632ace6 [DeepseekV32] Enable flashmla_prefill kernel with fp8 kvcache (#11655) 
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
 2025-10-27 23:11:48 -07:00
ishandhanani
285a8e6986 docker: add CUDA13 support in dockerfile and update GDRCopy/NVSHMEM for blackwell support (#11517) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-27 22:00:54 -07:00
Yuan Luo
813bd6f85c [2/2] Use moe_sum_reduce cuda kernel (#10654) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
 2025-10-28 12:01:57 +08:00
Xinyuan Tong
729f612dc6 Update openai package version to 2.6.1 (#12222) 2025-10-28 11:23:40 +08:00
jianan-gu
899453ac50 Use explicit uint64 dtype for Tensor data_ptr() to avoid overflow (#11994) 2025-10-27 19:05:57 -07:00
Lifu Huang
ce832d7034 Add env var to control custom Triton kernel cache and set CSGMV as default backend. (#12176) 2025-10-27 17:49:32 -07:00
weiliang
88596739a4 Support running FP4 Deepseek on SM120. (#11708) 2025-10-27 17:37:49 -07:00
Yineng Zhang
a6ea3add76 [Auto Sync] Update scheduler.py, spec_info.py, run_suite.py... (20251027) (#12235) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-10-27 17:21:08 -07:00
fzyzcjy
326c84c493 Compiling rope while preserving true on policy (#12161) 2025-10-28 08:02:17 +08:00
gongwei-130
8da608cce0 fix: AttributeError: 'NixlKVManager' object has no attribute 'prefill_tp_size_table' (#12234) 2025-10-27 15:57:33 -07:00
satyamk7054
9fc3e8aac7 Add support for Matryoshka embeddings (#126) (#11142) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-10-28 02:49:36 +08:00
Chunyuan WU
c11b34d599 rope xpu: fix missing argument 'fused_set_kv_buffer_arg' and replace native with sgl_kernel_xpu impl (#12006) 2025-10-28 01:02:18 +08:00
ykcombat
05ad28f25e [Feature] PD-Multiplexing Context and Scheduler. (#11592) 2025-10-28 00:54:43 +08:00
pansicheng
0cae873fcd check_offload_progress more frequently (#11656) 2025-10-28 00:37:38 +08:00
Haichao Zhu
a8b91f6b2d improve mimax-m2 rmsnorm precision (#12186) 2025-10-28 00:01:42 +08:00
Jimmy
959d1ab84b fix(metrics): double times add_latency for DECODE_BOOTSTRAP (#12209) 2025-10-27 23:59:48 +08:00
Muqi Li
6c1c193308 [Detokenizer Manager] Cleanup state when reqs are finished (#12205) 2025-10-27 23:59:09 +08:00
cctry
3029d30189 Fix crash after flush cache (#12107) 
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
 2025-10-27 23:52:27 +08:00
Yuan Luo
f389f01714 Optimize triton_mrope with torch compile (#12112) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-27 23:49:22 +08:00
Weiwei
caa4819bfc Add support for AutoRound quantized models (#10153) 2025-10-27 18:17:29 +08:00
Yuxuan Zhang
a88b006ecf GLM-4-0414 and GLM-4.1V Code Refactor (#12117) 2025-10-27 16:57:07 +08:00
sglang-bot
55d75e11bd chore: bump SGLang version to 0.5.4.post1 (#12169) 2025-10-27 09:35:20 +08:00