kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 16:27:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,081 Commits 157 Branches 0 Tags
eff7df6d0a49811ce86edb5cb9e4e1bf70b91c09
Commit Graph

4680 Commits

Author SHA1 Message Date
jacky.cheng
eff7df6d0a [AMD] Enable fused shared expert append and flatten quant for fp8 deepseekR1 model (#13705) 
Co-authored-by: yctseng0211 <yctseng@amd.com>
 2025-11-21 02:48:28 -08:00
Mick
5e7f91d451 [diffusion] profile: support performance metric dumping and comparison (#13630) 2025-11-21 18:47:16 +08:00
Xiaoyu Zhang
a34d3abb54 [Clean code] Compressed_tensors_moe code clean (#13719) 2025-11-21 18:15:46 +08:00
Cheng Wan
6d0e0b9bfc [11/N] MoE Refactor: Simplifying SBO Implementation with Dispatcher Hooks (#13327) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-11-21 01:11:37 -08:00
Even Zhou
589d9ad55b [NPU] chore: bump to CANN 8.3.RC1 and Pytorch 2.8.0 (#13647) 2025-11-21 17:07:08 +08:00
Ziming Huang
a244c0309c [Fix] Qwen3Next lmhead dtype (#13708) 2025-11-21 16:01:33 +08:00
Teng Ma
1bb063aac8 [HiCache] fix unit test with changed new APIs (#13498) 2025-11-21 15:40:31 +08:00
Michele Marzollo
b30f63c40f [Bugfix] Fix hidden state size in EAGLE PD disaggregation buffers (#13590) 
Co-authored-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-11-21 15:32:40 +08:00
Lianmin Zheng
90a0133515 [Auto Sync] Update http_server.py, io_struct.py, scheduler_... (20251120) (#13679) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Zhuqi Li <zhli@x.ai>
 2025-11-20 23:17:15 -08:00
zyksir
4360279036 [diffusion] server: use meta to avoid Linear init for TextEncoder (#13564) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-21 15:14:24 +08:00
ishandhanani
b537ac0d1f Fix ZMQ bind error on non-zero rank nodes when using SGLANG_BLOCK_NONZERO_RANK_CHILDREN=0 (#13686) 2025-11-20 22:54:41 -08:00
Mick
eda2f70033 [diffusion] doc: minor update docs (#13177) 2025-11-21 14:35:29 +08:00
Yuhao Yang
8c212a2029 [difusion] CI: speed up multimodal_gen ci (#13665) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-21 14:26:33 +08:00
Stefan He
d754ce973e [Piecewise Cuda Graph] rename, refactor and add more logging (#13675) 
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
 2025-11-21 13:28:38 +08:00
Yuan Luo
475962a139 [VLM] Support Piecewise CUDA Graph for InternVL (#13640) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-21 13:21:51 +08:00
Xiaoyu Zhang
bfcf15a129 [opt kimi k2 4 / n] Delete useless pad kernel in sgl_moe_align_block_size (#13587) 2025-11-21 13:16:42 +08:00
Xiaoyu Zhang
fb04d43428 [kimi k2 thinking] Avoid useless torch.zeros_ (#13596) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-11-21 13:15:27 +08:00
Mike Qiu
6be65ae462 Fix target MLA with eagle3 support for PD disaggregation (#13555) 
Signed-off-by: Michael Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
 2025-11-21 12:16:05 +08:00
Glen Liu
750084ae08 remove unnecessary starvation check (#13619) 2025-11-20 19:10:51 -08:00
Mick
c0a2513b07 [diffusion] CI: improve validation method (#13627) 2025-11-21 09:13:13 +08:00
fzyzcjy
3f1cfd87b6 Super tiny remove unused MiniMaxM2MLP class (#13659) 2025-11-21 07:35:12 +08:00
Stefan He
b5344b31b8 [Piecewise CUDA Graph] Fix recompile issue for Mixtral and Grok2 (#13667) 
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
 2025-11-20 14:20:11 -08:00
Glen Liu
ada8ce1fd0 allow loras to be implicitly evicted and loaded based on max_loaded_loras (#11526) 2025-11-20 13:34:32 -08:00
b8zhong
42028af614 enable csgmv automatically on cuda (#13600) 2025-11-20 12:53:02 -08:00
hlu1
7291c72e57 [Deepseek V3.2] Change indexer weights_proj to fp32 (#13459) 2025-11-20 12:24:10 -08:00
Vincent Zhong
6bc3062894 Fix launch of Olmo3 (#13666) 
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2025-11-20 12:04:30 -08:00
YAMY
fa92441027 [DeepseekV3.2] Deepseek fp8 support for MHA path (#12964) 2025-11-20 11:13:36 -08:00
Fan Yin
2dec555d36 [10/n] decouple quantization impl from vllm dependency - fix import (#13524) 2025-11-21 01:45:51 +08:00
Lianmin Zheng
a352e833c4 CI: Kill zombie diffusion processes in CI & minor code style fix on rotary embedding fallback (#13637) 2025-11-20 21:57:01 +08:00
Lianmin Zheng
7af9b88c6c Revert "[Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel)" (#13644) 2025-11-20 02:11:12 -08:00
yctseng0211
c8ede0e93c [ROCM] Optimized deepseek-r1 fp8 model with + triton_gemm_a8w8 + batch_gemm_a8w8 + fused set_mla_kv_buffer kernel (#13617) 
Co-authored-by: root <root@smci355-ccs-aus-m12-17.cs-aus.dcgpu>
Co-authored-by: jacky.cheng <yichiche@amd.com>
 2025-11-20 00:29:56 -08:00
DarkSharpness
b51f9bbee7 [Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel) (#13453) 2025-11-20 00:03:32 -08:00
Mick
4a8442af1b diffusion: improve baseline performance monitor (#13614) 2025-11-20 14:29:56 +08:00
Thomas Wang
7dcf910d53 Add support for new aiter version (AR accuracy, is_shuffled PR) (#13554) 
Co-authored-by: sogalin <39478626+sogalin@users.noreply.github.com>
 2025-11-19 22:17:57 -08:00
joesun
c7b37b7074 [diffusion] fix: remove multimodal_gen redundant get_bool_env_var func (#13583) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-20 14:05:29 +08:00
iLeGend
10e0b83a4c Add FP32 dtype support for RoPE - Part2 (#13328) 2025-11-19 21:19:53 -08:00
Mick
bc42c8c415 [diffusion] refactor: refactor pipeline folders (#13253) 2025-11-20 12:56:51 +08:00
Mick
2e3a69ae05 [diffusion] refactor: remove PreprocessorConfig (#13248) 2025-11-20 12:53:26 +08:00
Kaixi Hou
c3c4da71fb [NVIDIA] Add fp8 gemm benchmark on blackwell (#13528) 2025-11-19 19:35:00 -08:00
Xiaoyu Zhang
dc69462456 [CI fix] Fix image download failures in VLM CI tests (#13613) 2025-11-20 11:18:06 +08:00
StonyPort
48ca9f7518 feat: support external custom models (#13429) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-20 11:16:58 +08:00
Mick
127d59cd2c [diffusion] CI: improve diffusion CI (#13562) 
Co-authored-by: Adarsh Shirawalmath <114558126+adarshxs@users.noreply.github.com>
 2025-11-20 10:54:13 +08:00
Yuan Luo
af6bcadcf7 [VLM] Support Piecewise CUDA Graph for Qwen2.5-VL (#13055) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Yuhao Yang <yhyang201@gmail.com>
 2025-11-20 10:23:44 +08:00
Binyao Jiang
67fca6b297 [GDN] Remove unnecessary conv state clone (#13603) 2025-11-19 16:09:12 -08:00
Binyao Jiang
f88b2aa6af [GDN] Remove unnecessary contiguous() (#13604) 2025-11-19 16:07:49 -08:00
Lianmin Zheng
17b24aca6e [Auto Sync] Update base_grammar_backend.py, collector.py (20251116) (#13357) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-11-19 16:05:50 -08:00
sglang-bot
bfaf0b8607 chore: bump sgl-kernel version to 0.3.17.post2 (#13570) 2025-11-19 14:02:57 -08:00
Xinyuan Tong
a355794905 Expend compatibility check for all quantized MoE models (#13465) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-20 01:24:27 +08:00
Liangsheng Yin
196b940aed [3/N] CI refactor: move some manually triggered tests. (#13448) 2025-11-19 23:06:53 +08:00
Zilin Zhu
e197bef5ce [RL] Allow passing tensors of different dtypes for FlattenedTensorBucket (#13413) 2025-11-19 17:24:48 +08:00