kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-09 08:47:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,048 Commits 157 Branches 0 Tags
6bc3062894652e5fe8a4357110fa1d18ce360a2e
Commit Graph

4655 Commits

Author SHA1 Message Date
Vincent Zhong
6bc3062894 Fix launch of Olmo3 (#13666) 
Signed-off-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2025-11-20 12:04:30 -08:00
YAMY
fa92441027 [DeepseekV3.2] Deepseek fp8 support for MHA path (#12964) 2025-11-20 11:13:36 -08:00
Fan Yin
2dec555d36 [10/n] decouple quantization impl from vllm dependency - fix import (#13524) 2025-11-21 01:45:51 +08:00
Lianmin Zheng
a352e833c4 CI: Kill zombie diffusion processes in CI & minor code style fix on rotary embedding fallback (#13637) 2025-11-20 21:57:01 +08:00
Lianmin Zheng
7af9b88c6c Revert "[Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel)" (#13644) 2025-11-20 02:11:12 -08:00
yctseng0211
c8ede0e93c [ROCM] Optimized deepseek-r1 fp8 model with + triton_gemm_a8w8 + batch_gemm_a8w8 + fused set_mla_kv_buffer kernel (#13617) 
Co-authored-by: root <root@smci355-ccs-aus-m12-17.cs-aus.dcgpu>
Co-authored-by: jacky.cheng <yichiche@amd.com>
 2025-11-20 00:29:56 -08:00
DarkSharpness
b51f9bbee7 [Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel) (#13453) 2025-11-20 00:03:32 -08:00
Mick
4a8442af1b diffusion: improve baseline performance monitor (#13614) 2025-11-20 14:29:56 +08:00
Thomas Wang
7dcf910d53 Add support for new aiter version (AR accuracy, is_shuffled PR) (#13554) 
Co-authored-by: sogalin <39478626+sogalin@users.noreply.github.com>
 2025-11-19 22:17:57 -08:00
joesun
c7b37b7074 [diffusion] fix: remove multimodal_gen redundant get_bool_env_var func (#13583) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-20 14:05:29 +08:00
iLeGend
10e0b83a4c Add FP32 dtype support for RoPE - Part2 (#13328) 2025-11-19 21:19:53 -08:00
Mick
bc42c8c415 [diffusion] refactor: refactor pipeline folders (#13253) 2025-11-20 12:56:51 +08:00
Mick
2e3a69ae05 [diffusion] refactor: remove PreprocessorConfig (#13248) 2025-11-20 12:53:26 +08:00
Kaixi Hou
c3c4da71fb [NVIDIA] Add fp8 gemm benchmark on blackwell (#13528) 2025-11-19 19:35:00 -08:00
Xiaoyu Zhang
dc69462456 [CI fix] Fix image download failures in VLM CI tests (#13613) 2025-11-20 11:18:06 +08:00
StonyPort
48ca9f7518 feat: support external custom models (#13429) 
Co-authored-by: qiuxuan.lzw <qiuxuan.lzw@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-20 11:16:58 +08:00
Mick
127d59cd2c [diffusion] CI: improve diffusion CI (#13562) 
Co-authored-by: Adarsh Shirawalmath <114558126+adarshxs@users.noreply.github.com>
 2025-11-20 10:54:13 +08:00
Yuan Luo
af6bcadcf7 [VLM] Support Piecewise CUDA Graph for Qwen2.5-VL (#13055) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Yuhao Yang <yhyang201@gmail.com>
 2025-11-20 10:23:44 +08:00
Binyao Jiang
67fca6b297 [GDN] Remove unnecessary conv state clone (#13603) 2025-11-19 16:09:12 -08:00
Binyao Jiang
f88b2aa6af [GDN] Remove unnecessary contiguous() (#13604) 2025-11-19 16:07:49 -08:00
Lianmin Zheng
17b24aca6e [Auto Sync] Update base_grammar_backend.py, collector.py (20251116) (#13357) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-11-19 16:05:50 -08:00
sglang-bot
bfaf0b8607 chore: bump sgl-kernel version to 0.3.17.post2 (#13570) 2025-11-19 14:02:57 -08:00
Xinyuan Tong
a355794905 Expend compatibility check for all quantized MoE models (#13465) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-20 01:24:27 +08:00
Liangsheng Yin
196b940aed [3/N] CI refactor: move some manually triggered tests. (#13448) 2025-11-19 23:06:53 +08:00
Zilin Zhu
e197bef5ce [RL] Allow passing tensors of different dtypes for FlattenedTensorBucket (#13413) 2025-11-19 17:24:48 +08:00
ykwd
075ba74dd4 [Doc] Update HiCache and Mooncake docs & Mooncake Setup Error Checking (#12740) 2025-11-19 12:03:27 +08:00
Chen Haozhe
6c2e5fcd91 [feat][Ascend][Mindspore]: support model-impl of mindspore (#9234) 2025-11-19 09:17:47 +08:00
Baizhou Zhang
9f59194f29 [Fix] Fix DeepSeek V3 MTP on B200 (#13548) 2025-11-18 16:30:26 -08:00
b8zhong
92ad2ff9ce Flashinfer TRTLLM-GEN-MoE + Qwen3 (#13489) 2025-11-18 14:18:29 -08:00
Baidu-AIAK
9b64f6f359 [fix] Fixes accuracy issues caused by incorrect use of rope (#13495) 2025-11-18 14:17:33 -08:00
Kangyan-Zhou
c0d1a3383b Remove jet-ai/Jet-Nemotron-2B in nightly text tests as this is constantly failing (#13540) 2025-11-18 13:58:16 -08:00
Glen Liu
d79e12941c Small cleanups related to LoRA weight loading (#13474) 2025-11-18 09:14:34 -08:00
Junrong Lin
518467be64 [Feature] Re:Enable hybrid mem saver (#12962) 2025-11-18 21:29:15 +08:00
Morpheus Guo
7e88b9c128 [BugFix] Accuracy and function Issue when run ptpc quant model (#13157) 
Co-authored-by: yuechguo <yuechguo@amd.com>
 2025-11-18 01:54:58 -08:00
Zhiqiang Xie
cfcf2758aa [HiCache] Critical fix to host memory double free (#13501) 
Co-authored-by: Hao Chen <cighao@gmail.com>
 2025-11-18 01:34:23 -08:00
Nicholas
ac81db66c2 [VLM][feat] Support encoder DP for Qwen2.5-VL (#13126) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: liusy58 <xiehang.lsy@alibaba-inc.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
 2025-11-18 16:13:18 +08:00
Cheng Wan
a5ad0069b2 fix: change performance log directory to cache path (#13482) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-18 15:18:43 +08:00
alisonshao
0c96677902 CI: fix NFS EBUSY error in PR test workflow (#13460) 
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-11-18 13:25:29 +08:00
b8zhong
f33860777c [Piecewise CUDA Graph] Support ModelOpt FP8 (#13094) 2025-11-18 12:46:24 +08:00
Shangming Cai
4c3573e493 [PD] Clarify init method docstrings for kvsender and kvreceiver (#13476) 2025-11-18 12:20:19 +08:00
wingedge
f1be8aa0f2 chore: add an unified server arg for multimodal inputs preprocess config(#12149) 
Co-authored-by: bianfeng <bianfeng@pinduoduo.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-11-18 12:18:50 +08:00
Zijian Zhang
aa8ecbda7a model: support JetVLM (#13289) 2025-11-18 12:02:03 +08:00
Binyao Jiang
26ca07469b [GLM4.6v] Relax the constraint of non-user role chat completion message schema for new GLM-v release (#13258) 2025-11-18 10:58:17 +08:00
Binyao Jiang
90c18a16cb [GLM4.6v] Required changes for bumping up to transformer 5.x (#13229) 2025-11-18 10:58:00 +08:00
Minglei Zhu
9846f8ed72 fix MambaPool clear method after refactoring (#13449) 2025-11-18 10:03:45 +08:00
Baizhou Zhang
85ae508e8b Add bfloat16 tuned fused moe config for Dpsk-MTP layer on B200 (#13455) 2025-11-17 17:44:31 -08:00
pansicheng
d879e37f1b Add interface_v1 option for dynamic HiCache backend (#13140) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-11-17 17:42:17 -08:00
Kaixi Hou
e389f91dec [NVIDIA] Fix broken fp8 MoE of deepseek v3 (#13264) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2025-11-17 16:13:28 -08:00
Vedant V Jhaveri
aac07bf7fd [Embeddings Performance Testing] Add performance test for embedding models (#12359) 2025-11-17 15:35:18 -08:00
Simo Lin
25acbbc612 Remove verbs from GET endpoint paths to follow REST standards (#13273) 2025-11-17 12:34:36 -08:00