kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-07 07:47:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,459 Commits 157 Branches 0 Tags
c1d529c19605cbf1f9be8db6d6d225b1465ea2e0
Commit Graph

6113 Commits

Author SHA1 Message Date
Aurick Qiao
c1d529c196 Fix Session for multimodal and expose it through Engine (#18152) 2026-02-04 10:33:27 +08:00
wxy
da758ed601 [diffusion] fix: fix server cache-dit bug under continuous dynamic requests (#17140) 2026-02-04 09:03:37 +08:00
satyamk7054
793bf9fc06 Update weight rename check for Qwen3 Embeddings (#17535) 2026-02-03 13:55:11 -08:00
Hudson Xing
e867040fc6 add streaming parallel tool call test case (#18097) 2026-02-03 12:46:01 -08:00
R0CKSTAR
7de650c83c [diffusion] hardware: support diffusion models on MTGPU (doc, 6/N) (#17346) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-02-03 12:44:57 -08:00
R0CKSTAR
ec2461bc16 [diffusion] hardware: support diffusion models on MTGPU (multi-GPU, 5/N) (#17318) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-02-03 12:44:22 -08:00
R0CKSTAR
acf724b036 [Diffusion] Only import sgl_kernel in custom op cuda path (SiluAndMul and RMSNorm) (#15592) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-02-03 12:42:58 -08:00
Vladislav Nosivskoy
e166ca8758 [HiCache] feat: Add detailed cache hit breakdown for HiCache in sglext and Prometheus metrics (#17648) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-02-03 11:45:35 -08:00
Even Zhou
d48bbe3bed [CI][NPU] Bugfix import sgl-kernel error (#18173) 2026-02-03 11:39:38 -08:00
DiweiSun
495290aefd enable ut test for xpu devices (#11712) 
Co-authored-by: jundu <jun.du@intel.com>
Co-authored-by: Gao, Pengfei <pengfei.gao@intel.com>
 2026-02-03 11:15:14 -08:00
elvischenv
99fab2ce67 [Bugfix] Fix Mistral Large 3 NVFP4 TRTLLM MoE (#18065) 2026-02-03 20:32:49 +08:00
Lewis
a45647bce1 [PD] feat: support mooncake intra-node nvlink kv transfer (#17866) 
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
 2026-02-03 17:47:52 +08:00
Xiaowei Wang
cc69ac9e7a Warmup before profiling prefill latency for dynamic chunk sizing (#17198) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-03 17:45:23 +08:00
Mohammad Miadh Angkad
6f6b9c6e42 [Perf] Use safetensors load_file in multithread loader (#18124) 2026-02-02 23:21:13 -08:00
fatSheep
7a9d9c79d1 [HiCache] fix: apply extra_backend_tag in Mooncake batch_exists (#17265) 2026-02-02 22:54:56 -08:00
Viacheslav
74f716dbd7 Gigachat 3 tool parser and tests (#14765) 2026-02-02 22:28:34 -08:00
Kaixi Hou
4181290efd [NVIDIA] Add --top-k argument to run_eval.py (#18025) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-02 22:17:53 -08:00
b8zhong
78bf13db44 MoE Refactor: Refactor modelopt_quant.py -> flashinfer_trllm.py (#16685) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-02-02 20:45:14 -08:00
Xiaoyu Zhang
eedd472025 [Diffusion] fix serving image_edit get input image bug (#18109) 2026-02-03 12:17:16 +08:00
Hank Han
e484c90cc7 Add triton_fused_moe config for GLM-4.7-FP8 tp8 H20 H20-3e (#18091) 2026-02-03 12:08:23 +08:00
Linyu Wu
9b1619c148 [Move sgl-kernel Kernel to JIT] Add JIT concat MLA kernels (#17889) 2026-02-03 10:49:17 +08:00
Mick
62004fd2be [diffusion] UX: improve logging (#18122) 2026-02-03 10:35:05 +08:00
zhangheng
180594358b [HiCache]: Support DeepSeek v32 cpu offloading (#17415) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-02-02 18:07:37 -08:00
Xiaoyu Zhang
a1bbc892af [Diffsuion & JIT_kernel] QKNorm cross heads kernel (#18073) 2026-02-03 10:03:17 +08:00
EkiRui
fd983b09b6 [Performance] Optimize radix cache eviction performance (#14339) 
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
 2026-02-03 09:44:20 +08:00
Alison Shao
28e2340725 Fix HF hub race condition in CI by coordinating model downloads across TP ranks (#17787) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-02-02 14:57:45 -08:00
cctry
027f314050 [Fix] data race in req_to_token pool (#17850) 2026-02-02 14:38:15 -08:00
TZHelloWorld
cbf1500390 [MiMoV2Flash] [feat]: support two batch overlap (#17634) 2026-02-02 14:03:57 -08:00
khalilzhk
b0a6d5244c [NPU] support dsv32 radixcache on ascend (#17964) 2026-02-03 03:34:12 +08:00
Yongfei Xu
677f3c49da [DeepSeek V3.2] [Bugfix] slice indexer and padding fa3 when can not run cuda graph (#17076) 2026-02-03 01:32:20 +08:00
Yuhao Yang
980d2936cd model: support Step-3.5-Flash (#18084) 
Co-authored-by: ltd0924 <ltd0924@sina.com>
 2026-02-03 00:40:07 +08:00
Sugar920
c781db0f6c [NPU] update nightly tests (#17952) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: cy <chenyang08056032@163.com>
 2026-02-03 00:13:30 +08:00
Xuhao Zhang
0537232b05 [NPU]mindspore model support moe (#15363) 2026-02-02 17:52:49 +08:00
CHEN Xi
aa780a6258 [diffusion] fix: remove accelerate dependency for device mapping (#18026) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-02-02 17:24:19 +08:00
Xiaoyu Zhang
a0757c9624 [Diffusion] Fix Ring Parallel bug with FA4 (#18062) 2026-02-02 17:06:51 +08:00
陈一涵
86117dfe0e [diffusion] CI: deprecate WarmupRunner in CI (#18038) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-02-02 15:16:08 +08:00
Xiaoming Liu
522e13b4d2 fix: correct weight loading prefix mapping for Qwen3-VL (#18024) 
Co-authored-by: liuxiaoming <liuxiaoming@modelbest.cn>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-02-01 21:50:21 -08:00
RangerCD
a480ca7ead fix: zmq_to_tokenizer encoder transfer when host listens to 0.0.0.0 (#17929) 2026-02-02 13:27:27 +08:00
siyu
f1824a957b [EPD][refactor]: introduce BaseMMReceiver for gRPC transport integration (#17921) 2026-02-02 11:37:32 +08:00
Cheng Wan
ab8b99eb23 Refine logprob logic for request handling (#17986) 2026-02-01 19:11:52 -08:00
Simon (Jiyou) Li
8ed35df204 Add bootstrap_room validation to detect metadata corruption in PD disaggregation (#17430) 
Co-authored-by: 继优 <jiyou.ljy@alibaba-inc.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-02-02 10:43:50 +08:00
Mick
c84cd4b5ff [diffusion] fix: fix missing component names for VAELoader (#18069) 2026-02-02 09:48:17 +08:00
Mick
977096ae03 [diffusion] cli: introduce generic attention backend configuration in ServerArgs (#18036) 2026-02-02 09:47:40 +08:00
Yuhao Yang
d11ccc0a0a fix: avoid double reduce in VLM dp attention (#17991) 2026-02-02 09:44:32 +08:00
Yuan Luo
9227d4f748 [Fix] Remove no use code in MiMo-V2-Flash (#18051) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-01 15:34:09 -08:00
Glen Liu
99dad105fd [TestFix] rewrite LoRA overlap loading tests (#18047) 2026-02-01 14:52:08 -08:00
Koushik Dutta
993ec178ef [BUGFIX]: using language-only should not reserve space for the vision encoder (#18011) 
Co-authored-by: root <root@ubuntu-nvidia.localdomain>
 2026-02-01 14:49:00 -08:00
Lianmin Zheng
1acae30806 [Auto Sync] Update test_deterministic.py (20260131) (#18034) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-01 07:40:07 -08:00
Lianmin Zheng
fb609669ca [Auto Sync] Update elementwise.py (20260131) (#18033) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-01 07:39:37 -08:00
Yuan Luo
4ea4f2a20c [VLM] Optimize get_rope_index for GLM4v (#17420) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-01 18:59:15 +08:00