kvcache-ai/sglang
1
0
Fork 0
mirror of https://github.com/kvcache-ai/sglang.git synced 2026-07-18 09:47:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

perf: reduce ttft and itl with stream_interval 1 (#658)

Browse Source
This commit is contained in:
zhyncs
2024-07-20 02:14:22 +10:00
committed by GitHub
parent e1792cca24
commit ac971ff633
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/sglang/srt/server_args.py
View File

@@ -33,7 +33,7 @@ class ServerArgs:
# Other runtime options
tp_size: int = 1
stream_interval: int = 8
stream_interval: int = 1
random_seed: Optional[int] = None
# Logging