kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-07 07:47:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,593 Commits 157 Branches 0 Tags
a8eef53dc419f6575ca4aece11065d3ff7f463b9
Commit Graph

6208 Commits

Author SHA1 Message Date
Thomas Wang
a8eef53dc4 Fp8 prefill attn kernel integration (#18528) 
Co-authored-by: kkHuang-amd <wunhuang@amd.com>
 2026-02-10 23:23:48 -08:00
BourneSun0527
2cc235e795 Fix Bug on dsv3.2 (#18553) 
This PR affects only the NPU. If any issues arise, please contact iforgetmyname.
 2026-02-11 14:39:01 +08:00
Michael
d84d2063d3 [AMD] Fix Janus-Pro crash and add Kimi-K2.5 nightly test (#18269) 2026-02-10 22:33:13 -08:00
Liangsheng Yin
cd90346a2b Add cache hit rate UT (#18566) 2026-02-10 21:27:41 -08:00
cutetocute
8d2892330c chore: fix some typos (#18577) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-02-10 20:47:41 -08:00
Liangsheng Yin
93fca0bbc3 Fix wrong prefill log. (#18570) 2026-02-10 15:54:03 -08:00
Yi-Chia Chen
2bfab1bb67 Fix radix cache key to include generated tokens in multi-turn (regression) (#16521) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-02-10 14:08:34 -08:00
Thomas Wang
4262f5259b Tilelang sparse decode fwd for dsv32 mi355 (#18488) 
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2026-02-10 09:53:26 -08:00
Zheng Li
44603764d6 fix(config): Support setting Mamba state dtype via config file (#18532) 
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
 2026-02-11 00:20:06 +08:00
Mick
efcdda0176 [diffusion] fix: fix fsdp (#18187) 2026-02-10 20:22:20 +08:00
wxy
47978ee858 [diffusion] feat: support parallel wan-vae decode (#18179) 2026-02-10 18:32:00 +08:00
Zehuan Li
26f2b3798d [DLLM] Basic dLLM scheduling strategy and implementation (#17484) 
Signed-off-by: Zehuan Li <lizehuan.lzh@antgroup.com>
 2026-02-10 16:54:15 +08:00
shuwenn
8da14aea88 [HiCache] fix: StorageMetricsCollector was initialized twice (#18354) 2026-02-10 00:53:00 -08:00
Xinyuan Tong
398b81f78c Support GlmMoeDsaForCausalLM (#18521) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: BBuf <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-02-10 15:20:10 +08:00
Xinyuan Tong
e8a2c13380 Deepseekv32 compatibility with transformers v5 (#18297) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-02-10 14:50:40 +08:00
siyu
0b15f19927 [EPD] Add notification mechanism to fix server hang and add timeout env var (#18229) 2026-02-10 11:52:54 +08:00
maocheng23
1d366f1206 Make bench_one_batch_server compatible for more backends (#18512) 2026-02-10 10:36:39 +08:00
Qiaolin Yu
4a1b50bb2d Fix idle batch predict dtype in spec v2 (#18379) 2026-02-10 10:29:13 +08:00
Kartik Ramesh
26a006e47f Add cache_config_info metric. (#17273) 2026-02-09 16:09:09 -08:00
Lianmin Zheng
b027c5aca6 [Auto Sync] Update cache_init_params.py (20260209) (#18502) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 14:49:41 -08:00
Lianmin Zheng
ce95f203b0 [Auto Sync] Update logits_processor.py (20260209) (#18503) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-09 14:33:02 -08:00
ishandhanani
01e3f4682e feat(kv-events): Add medium field to KV event types for storage tier tracking (#18205) 2026-02-09 12:39:15 -08:00
Zheng Li
27c447653d model: support Qwen3.5 (#18489) 
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
 2026-02-10 00:27:59 +08:00
Kurt Shuster
006da22268 Pass quantize_config to _initialize_model (#18273) 2026-02-09 23:34:42 +08:00
brimon
ddbcfbaaab feature: support bidirectional attention for Gemma-3 (#10707) 2026-02-09 23:17:45 +08:00
Mick
4f7da5ad0f [diffusion] chore: fix unclean shutdown and resource leaks (#18477) 2026-02-09 22:32:08 +08:00
yrk111222
76eb1c8406 [diffusion] feat: add ModelScope support (#17924) 2026-02-09 19:23:45 +08:00
Baizhou Zhang
615a02dcd4 Revert "optimize get_topk_ragged by fusing get k and k_scale triton kernel" (#18471) 2026-02-09 16:37:19 +08:00
Liangsheng Yin
875ad6cf35 Tiny rename for spec related fileds. (#18468) 2026-02-09 00:10:39 -08:00
LHXuuu
107958a489 Make compressed-tensors MoEs support ignored layers (#17828) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 14:37:33 +08:00
Junlin Zhou
14652243bd [DLLM] Add JointThreshold algorithm for joint M2T and T2T decoding (#18171) 
Signed-off-by: Junlin Zhou <zhoujunlin.zjl@antgroup.com>
Co-authored-by: Tiwei Bie <tiwei.btw@antgroup.com>
 2026-02-09 14:20:45 +08:00
Bingxu Chen
3f3c201243 [AMD] Update aiter to v0.1.10.post2 (#18423) 
Co-authored-by: kkHuang-amd <wunhuang@amd.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-02-08 22:08:24 -08:00
Yingchun Lai
a1189068fa fix: fix the wrong return value type of draft model runner (#18105) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-02-08 20:51:35 -08:00
Zheng Wengang
68e31a3485 [BugFix][PD]Fix metadata_buffer_index leak when aborted in PD (#17483) 2026-02-09 11:34:29 +08:00
Shangming Cai
bffd765417 Refactoring Mooncake TE as a shared distributed component (#17810) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-09 10:53:11 +08:00
Yi Zhong
bf89cc3803 [ModelOPT] Support Qwen 3 Next Coder NVFP4 (#18224) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-02-08 22:29:07 +00:00
Mohammad Miadh Angkad
071bf2ce09 [Kimi-K2.5] Fix missing quant_config in KimiK25 (#18440) 2026-02-08 12:02:45 -08:00
Piotr Mazurek
656a3d742e Add tensor parallelism support to LFM2 ShortConv layers (#17777) 2026-02-09 00:52:47 +08:00
Mick
6601bc24da [diffusion] chore: revise process title (#18446) 2026-02-09 00:14:06 +08:00
debo3
031a652b93 Fix TRT-LLM MLA backend applying k_scale to BF16 KV cache in BMM1 (#18396) 2026-02-08 23:11:16 +08:00
Mick
a41aff1243 [diffusion] refactor: group component loaders under the component_loaders/ directory (#18438) 2026-02-08 23:02:27 +08:00
Yi Zhong
ca36d88fa6 [ModelOpt] Fix broken Qwen3-235B-A22B-Instruct-2507-NVFP4 launch (#18189) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-02-08 14:35:28 +00:00
wxy
43eecd8265 [diffusion] feat: support efficient sequence shard (#18161) 2026-02-08 21:09:39 +08:00
Zack Yu
d71ccd8860 fix: sync server_args.kv_cache_dtype when detecting FP8 KV cache (#18394) 2026-02-08 14:10:59 +08:00
DarkSharpness
8e2e835c2f [Fix] Fix backend selection after flashinfer version update (#18364) 2026-02-08 11:20:41 +08:00
Makcum888e
00248d85c7 [diffusion] platform: support WAN/FLUX/Qwen-Image/Qwen-Image-edit on Ascend (#13662) 
Co-authored-by: dhx98 <haox.dai@gmail.com>
Co-authored-by: DHX98 <haoxiand@andrew.cmu.edu>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: DHX98 <DHX98@noreply.gitcode.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-02-08 10:45:30 +08:00
Mohammad Miadh Angkad
7b83659310 fix: fix NVFP4 Kimi-K2.5 weight mapping and exclude list (#18370) 2026-02-08 10:23:48 +08:00
wxy
64950d8f97 [diffusion] feat: support saving videos directly on the server to avoid the overhead of tensor transfer (#18253) 2026-02-07 22:08:42 +08:00
Mick
31d4cd2ffd [diffusion] fix: respect dist_timeout option (#18386) 2026-02-07 20:56:04 +08:00
Mohammad Miadh Angkad
fddef76619 [Doc] Fix outdated --fp4-gemm-backend documentation (#18350) 2026-02-07 20:42:47 +08:00