kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-10 17:27:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,212 Commits 157 Branches 0 Tags
f4e835af2faa18220e327264efaa02e74dc2f60a
Commit Graph

5360 Commits

Author SHA1 Message Date
Liangsheng Yin
f4e835af2f Cleanup ModelRunner (#15802) 2025-12-25 18:13:30 +08:00
Xiaoyu Zhang
e6ce16a4c2 [diffusion] feat: support TP for Flux.1.dev (#15666) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-25 18:00:44 +08:00
Xiaoyu Zhang
de2f2880b5 [JIT sgl-kernel] Jit support per tensor quant (#15709) 2025-12-25 16:24:37 +08:00
zhangheng
a89e85e739 [1/N][Sparse With Hicache]: Add Sparse Interface (#14741) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: MagicYang1573 <1328657938@qq.com>
 2025-12-25 00:12:47 -08:00
Ke Bao
cbf9f13493 Adjust server args for Mimo-v2-flash model (#15803) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-25 16:04:39 +08:00
Yi Zhang
eb3da9c1dd fuse ssm state store into chunk_gated_delta_rule_fwd_h (#15409) 2025-12-25 16:03:15 +08:00
Yuan Luo
b9af8d2eb9 [VLM] Support apply qk norm in multi cuda streams (#15720) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-25 14:35:07 +08:00
Prozac614
10a9573efd [diffusion] CI: add tests to validate the size, extension, and format of output images/videos. (#15736) 2025-12-25 14:00:22 +08:00
Prozac614
49ab72f837 [diffusion] fix: remove unused LoRA application logic from RowParallelLinearWithLoRA class in linear.py (#15801) 2025-12-25 13:43:38 +08:00
R0CKSTAR
8865424f14 [diffusion] refactor: refactor attention backend checking to use backend enum (#15555) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2025-12-25 11:51:40 +08:00
Liangsheng Yin
b311c43d13 Clarify None handling in sglang's environ (#15770) 2025-12-25 11:40:17 +08:00
Huaixin Chang
0c39730b18 DP: support piggyback server load report (#11469) 
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com>
 2025-12-25 11:35:05 +08:00
satyamk7054
38dd4fbb66 Add overlap scheduling for embeddings code path (#14032) 
Co-authored-by: Satyam Kumar <satyamk@linkedin.com>
 2025-12-24 18:24:18 -08:00
Lianmin Zheng
92ddc46824 [Auto Sync] Update server_args.py (20251223) (#15700) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-12-24 14:55:14 -08:00
Chenxi Li
17b38f88bc Add LoRA metrics for potential auto scaling (#15149) 2025-12-24 12:13:52 -08:00
Xinyuan Tong
ae434f7821 Move limit-mm-data-per-request to make code clean (#15775) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-24 10:59:39 -08:00
Douglas Yang
17e65466de fix: nightly fix b200 gpqa (#15745) 2025-12-24 10:29:45 -08:00
Simo Lin
b27b5a834e [bug] fix code formatting which blocks ci (#15780) 2025-12-24 09:14:13 -08:00
Mayyyy
2fb3160598 [NPU] Bug fix in device detect (#14137) 2025-12-25 00:18:39 +08:00
Liwansi
8bf7f240b6 [NPU]qwen3 pp bugfix (#15390) 2025-12-25 00:14:40 +08:00
Mick
2c5679f314 [diffusion] refactor: unify the profiling api for all executors (#15718) 2025-12-24 23:26:09 +08:00
Liangsheng Yin
159b128357 Tiny add flush for CI crash locating (#15769) 2025-12-24 22:47:10 +08:00
fzyzcjy
9338f63f7d Add request counter in addition to existing response counter (#15768) 2025-12-24 22:22:38 +08:00
fzyzcjy
8196998a03 Tiny add num retracted tokens metric (#15653) 2025-12-24 22:15:23 +08:00
fzyzcjy
aa21c6e3f1 Super tiny code cleanup (#15652) 2025-12-24 22:15:00 +08:00
fzyzcjy
fd4a558e71 Add metrics for having prefill and decode in different ranks (#15752) 2025-12-24 21:35:35 +08:00
fzyzcjy
b3b818fd86 Tiny unify realtime_tokens_total metric (#15747) 2025-12-24 21:29:43 +08:00
Liangsheng Yin
d5fbbfd988 Retry removing wrong logic about max total token in spec decoding (#15748) 2025-12-24 21:13:26 +08:00
fzyzcjy
e245cac0c5 Support JSON format request logging for easier parsing (#15743) 2025-12-24 19:52:43 +08:00
fzyzcjy
c6a6ba4368 Tiny refactor request logger (#15740) 2025-12-24 19:51:26 +08:00
vincentzed
ddb3970eb4 Tiny env cleanup in deepgemm (#15706) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2025-12-24 00:58:46 -08:00
Liangsheng Yin
cb719c74ad [2/N] clean duplicate code of logprob processing in spec. (#15593) 2025-12-24 15:21:13 +08:00
Lianmin Zheng
ff903a7eea Simplify server args (#15704) 2025-12-23 22:46:12 -08:00
Prozac614
eee3700d84 [diffusion] feat: support lora strength (#15691) 2025-12-24 13:31:23 +08:00
Mick
dfb5357448 [diffusion] http-server: relax openai image endpoint's strict content_type limit (#15717) 2025-12-24 13:26:13 +08:00
Xuchun Shang
3bf07c684f [Feature][MM] split the images of one request into multiparts (#11828) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Kun(llfl) <llfl@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
 2025-12-24 11:22:04 +08:00
michael-amd
e7b09efc0a [AMD] Add AMD Nightly Performance & VLMs Accuracy Tests (#15500) 2025-12-23 19:03:27 -08:00
Xuchun Shang
fe712aa3df [bug fix] fix hicache jit kernel (#15177) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-24 10:31:21 +08:00
Roger Young
5c64a20da7 Update MiniMax-M2 ToolCall and add MiniMax-M2.1 in Docs (#15538) 
Co-authored-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-12-23 15:11:52 -08:00
Lianmin Zheng
cf81737693 Add kv_transfer_total_mb to metrics (#15667) 2025-12-23 11:16:55 -08:00
Liangsheng Yin
bd572360f3 Tiny apply gsm8k mixin to ngram test (#15606) 2025-12-24 01:30:26 +08:00
ratish
5f3a47d8a7 [model-gateway]: add gRPC router embeddings endpoint implementation (#15273) 2025-12-23 09:12:22 -08:00
Liangsheng Yin
705287b2e5 Tiny add more information in retract logging. (#15694) 2025-12-24 00:43:36 +08:00
Yubo Wang
762846531f Fix Illegal Memory Access when fa3 + spec + topk + page_size > 1 (#15469) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-12-24 00:13:57 +08:00
shuwenn
53f974b973 fix: potential crash for missing stream attribute (#15644) 2025-12-23 23:24:31 +08:00
DarkSharpness
291f11ae39 [Minor] Enhance JIT kernel and add dev docs (#14570) 2025-12-23 22:34:59 +08:00
Teng Ma
d7301c89ba [Feature] support fastsafetensors (#15091) 
Signed-off-by: Xuchun Shang <xuchun.shang@gmail.com>
Co-authored-by: Xuchun Shang <xuchun.shang@gmail.com>
 2025-12-23 22:33:56 +08:00
ryang
ffc23ef877 [diffusion] feat: generalize layerwise offloader to flux1 (#15633) 2025-12-23 22:22:40 +08:00
Tianyu Guo
fa2966983a Support PP for zmq_to_scheduler (#15312) 2025-12-23 17:07:55 +08:00
Douglas Yang
f9dd90ac35 fix: increasing H200 test timeout (#15600) 2025-12-23 01:00:37 -08:00