kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-08 00:07:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,931 Commits 157 Branches 0 Tags
e7dc85c50baf434fcd5faeee30a4e1d0cd9ddaed
Commit Graph

5760 Commits

Author SHA1 Message Date
Lianmin Zheng
e7dc85c50b Fix grammar sync across TP ranks (#17100) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2026-01-15 18:38:01 -08:00
Ratish P
c81bad1bf7 [diffusion] feat: add cloud storage support for API (#14579) 2026-01-16 10:29:38 +08:00
hlu1
0e86de7c0b Remove deepseek-r1 from THINKING_MODE_CHOICES in run_eval.py (#17178) 2026-01-15 16:53:06 -08:00
PiteXChen
8b22deef5b fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369) 
Signed-off-by: CLFutureX <chenyongqyl@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 15:14:38 -08:00
JinYan Su
72e2f70ef7 feat(hicache): support numa detect to reduce long tail latency (#11028) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-15 14:11:49 -08:00
huangtingwei
77fc4c4a53 Add mooncake store read/write bandwidth logs (#10598) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Lei Huang <lei.huang@novita.ai>
 2026-01-15 12:15:41 -08:00
cctry
dda35ccbd8 Fix gid calculation in per_tensor_absmax_kernel (#17126) 2026-01-15 23:21:41 +08:00
wxy
d11e2dc6f4 [diffusion] chore: improve the output_path config and enable the server to return inference duration (#16965) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-01-15 22:31:50 +08:00
Mick
16831ab6d7 [diffusion] fix: fix using upstream flash_attn on blackwell (#17111) 2026-01-15 22:30:48 +08:00
R0CKSTAR
c9a45b7e3c [diffusion] fix: fix UMA detection (#17113) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-01-15 22:29:46 +08:00
HuangJi
e7df8bdc5c [diffusion] refactor: move SLA to attention_backend folder (#17020) 2026-01-15 21:36:48 +08:00
Lancer
e997995037 [diffusion] fix: optimize text encoder CPU offload initialization to address OOM (#17064) 
Signed-off-by: Lancer <maruxiang6688@gmail.com>
Co-authored-by: Lancer <maruxiang6688@gmail.com>
 2026-01-15 21:28:57 +08:00
Mick
68e8d0f68d [diffusion] CI: add testcase for cfg parallel (#17056) 2026-01-15 13:13:31 +08:00
sglang-bot
000ad42225 chore: bump sgl-kernel version to 0.3.21 (#17075) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-01-15 12:41:17 +08:00
Hanming Lu
9d5f16d456 [SWA] fix swa radix cache match_len_since_tombstone update when hits swa_tombstone (#17061) 2026-01-15 11:28:33 +08:00
Ke Bao
7f8a58fffb Refactor prefix cache type checking (#17028) 2026-01-15 11:28:13 +08:00
b8zhong
4346db5faf [Fix] Remove assertion for padding for NVFP4 weight scales to fix GLM 4.5 NVFP4 (#12497) 2026-01-15 08:57:14 +08:00
Артем Савкин
424a380077 [NPU] NPU quantization refactoring & more quantization formats support (#14504) 
Co-authored-by: TamirBaydasov <mr.jeijy@gmail.com>
Co-authored-by: Tamir Baydasov <41994229+TamirBaydasov@users.noreply.github.com>
Co-authored-by: Савкин Артем <savkinartem@MacBook-Air-Viktoria.local>
Co-authored-by: Edward Shogulin <edward.shogulin@gmail.com>
 2026-01-15 04:25:15 +08:00
Simo Lin
f091858304 [gRPC] Add GetLoads RPC for comprehensive load metrics (#17087) 2026-01-14 12:05:10 -08:00
Aurick Qiao
5b1215d9da fix session request with None tokenizer (#16278) 2026-01-14 10:41:08 -08:00
Simo Lin
b3a3f51320 [API] Add /v1/loads endpoint for load metrics (#16976) 2026-01-14 09:13:20 -08:00
shuwenn
de94d793ad feat: support qwen3(-VL) rerank scoring&chat template (#16403) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-15 00:45:46 +08:00
Xiaoyu Zhang
0d904ef44c [diffusion] fix: fix fsdp tp load make param miss parallel meta data (#17058) 2026-01-15 00:12:33 +08:00
Yuan Luo
969faaa410 [diffusion] fix: revise fa4 backend to support blackwell (#17077) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-14 23:31:46 +08:00
shuwenn
48c2aca9ba [Env] centralize pd vars in environ.py (#16264) 2026-01-14 23:06:01 +08:00
fxmarty-amd
5af84c8af5 [AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392) 
Co-authored-by: Dehua Tang <dehtang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: YC Tseng <yctseng@amd.com>
 2026-01-14 01:44:40 -08:00
Yuan Luo
feae615b11 [VLM] Support ViT CUDA Graph for InternVL (#16732) 2026-01-14 17:29:23 +08:00
Netanel Haber
e75299a111 Fix issues/16714: Revert comment out of tl.debug_barrier() in causal_conv1d_triton (#16899) 
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2026-01-14 17:26:48 +08:00
roikoren755
72bacc88c8 [NemotronH] Use ReplicatedLinear for fc1_latent_proj (#16569) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-01-14 17:08:27 +08:00
shaharmor98
ba625c2d90 Feat/support nemotron h mtp (#17013) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2026-01-14 16:30:35 +08:00
HuangJi
030496eb06 [diffusion] fix: fix --warmup-resolutions' conflict with CacheDiT (#16962) 2026-01-14 14:44:10 +08:00
shuwenn
cd33694585 feat: add --admin-api-key for finer-grained endpoint auth (#15908) 
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
 2026-01-13 20:21:55 -08:00
Mick
a5348eac4c [diffusion] chore: avoid raising error when output resolution is not optimal (#17030) 2026-01-14 11:36:27 +08:00
Mick
9524040220 [diffusion] chore: refactor warmup logic (#17027) 2026-01-14 11:35:06 +08:00
Liangsheng Yin
e2c8a50b38 fix grammar timeout sync across tp ranks. (#16898) 2026-01-14 10:26:31 +08:00
Lianmin Zheng
a4825ed588 Fix kernel type annotations for fp8 quant and logging (#16994) 2026-01-13 18:14:32 -08:00
Hubert Lu
afe285f7bd [AMD] enable CUDA graph for NSA backend and fix NSA FP8 fused RMSNorm group quant (#16841) 
Co-authored-by: wufann <715544327@qq.com>
 2026-01-13 17:36:01 -08:00
Byron Hsu
339915ce2b [logprob] Fix logprob + streaming for long concurrent decode by caching already processed logprob (#17005) 
Co-authored-by: root <root@memx-cge-29-sr1.xpop.twttr.net>
 2026-01-13 12:39:14 -08:00
Lianmin Zheng
075c5a5789 Code clean up for fp8 quantization (#16982) 2026-01-13 12:38:39 -08:00
Yuhao Yang
a0b4ba9032 [diffusion] model: GLM-Image (#16894) 
Co-authored-by: jianyingzhu <53300651@qq.com>
 2026-01-14 02:02:03 +08:00
Mick
7a869045b6 [diffusion] chore: clean excessive document (#16986) 2026-01-13 21:33:50 +08:00
ahb13
888d7e54d1 [diffusion] fix: fix compatibility issue with torch.compile and flash attention v4 (#16790) 
Co-authored-by: root <root@nimrodb200.tail5cbfe.ts.net>
Co-authored-by: ahb13 <ahb13@qq.com>
 2026-01-13 21:11:19 +08:00
qichu-yun
3cb1fbaee4 [diffusion] fix: fix Qwen-Image-Edit Lightning LoRA alpha/rank scaling (read per-layer *.alpha) (#16935) 2026-01-13 21:09:11 +08:00
DarkSharpness
ba9f6d8f26 [Refactor] Clean up JIT kernel utilites (#16884) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-01-13 17:54:16 +08:00
Xiaoyu Zhang
740d3c0b39 [Diffusion] Remove useless dependency in diffusion (#16967) 2026-01-13 17:25:53 +08:00
Hubert Lu
8716589826 [AMD][Diffusion] support timestep embedding kernel for AMD GPUs (#16766) 2026-01-12 22:17:07 -08:00
HuangJi
a83484275d [diffusion] perf: optimize linear calculation in SLA (#16648) 2026-01-13 09:38:35 +08:00
Mick
47d485f35f [diffusion] fix: fix not respecting dit_layerwise_offload server arg (#16252) 2026-01-13 09:29:07 +08:00
Mick
2b42309955 [diffusion] UX: provide solutions for OOM (#16940) 2026-01-13 09:25:27 +08:00
James
ae0baefb94 [NPU] upgrade npu mf_apater plugin (#15853) 2026-01-13 09:02:10 +08:00