ybyang fa8993111d Fix: Add token heuristic increment in total_tokens load balancing (#22614)

2026-04-21 01:29:33 -07:00

..

async_dynamic_batch_tokenizer.py

[tokenizer] improve non streaming request processing + some small fixes. (#20310)

2026-04-10 15:46:12 -07:00

cache_controller.py

[HiCache]Fix hybrid model move_indices (#22940)

2026-04-21 00:15:39 -07:00

communicator.py

refactor: extract FanOutCommunicator and use declarative spec table (#22967)

2026-04-16 15:37:19 -07:00

configure_logging.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

data_parallel_controller.py

Fix: Add token heuristic increment in total_tokens load balancing (#22614)

2026-04-21 01:29:33 -07:00

detokenizer_manager.py

[tokenizer] improve non streaming request processing + some small fixes. (#20310)

2026-04-10 15:46:12 -07:00

disagg_service.py

[PD] Cleanup BootstrapServer init and ready check (#19551)

2026-02-28 16:41:42 +08:00

embed_types.py

[Feature] Add token embedding overrides for sparse embedding replacement (#20960)

2026-04-08 20:51:36 -07:00

hisparse_coordinator.py

[HiSparse] Clarify decode token usage logs (#22331)

2026-04-13 18:03:25 -07:00

io_struct.py

[Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544)

2026-04-20 22:50:40 -07:00

mm_utils.py

[VLM] Chunk-aware ViT encoding with per-image cache and lazy device transfer (#22038)

2026-04-04 16:55:17 +08:00

multi_tokenizer_mixin.py

refactor: extract FanOutCommunicator and use declarative spec table (#22967)

2026-04-16 15:37:19 -07:00

multimodal_processor.py

[Feature] Stronger transformers modeling backend with TP, PP, MoE, VLMs, and torch compile (#19163)

2026-04-02 16:02:33 -07:00

overlap_utils.py

Enable JIT clamp_position and resolve_future_token_ids on ROCm (#21116)

2026-03-22 22:33:54 -07:00

prefill_delayer.py

[bugfix] fix bug when enable prefill delay and DP (#20134)

2026-03-13 08:51:09 +08:00

schedule_batch.py

Opt-in strip of thinking tokens from radix cache (#23315)

2026-04-20 22:59:50 -07:00

schedule_policy.py

Fix hybrid swa chunked prefill oom (#23174)

2026-04-21 10:46:45 +08:00

scheduler_dp_attn_mixin.py

Optimization: Reduce the number of D2H operations (#19424)

2026-03-04 16:32:42 +08:00

scheduler_input_blocker.py

Tiny move files to utils folder (#11166)

2025-10-03 22:40:06 +08:00

scheduler_output_processor_mixin.py

[Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544)

2026-04-20 22:50:40 -07:00

scheduler_pp_mixin.py

[mem] Flatten memory checkers into composable per-pool invariant checks (#22562)

2026-04-11 02:56:22 -07:00

scheduler_profiler_mixin.py

Add XPU profiler activity support in benchmark code (#12981)

2026-03-04 23:22:56 -08:00

scheduler_recv_skipper.py

[Feat]Add scheduler recv skipper weights to environment configuration (#13855)

2025-11-27 18:16:11 +08:00

scheduler_runtime_checker_mixin.py

integrate streaming session into UnifiedRadixCache (#23145)

2026-04-19 20:47:41 -07:00

scheduler_update_weights_mixin.py

[sgl] add support for weight update function in spedec (#22088)

2026-04-20 16:26:20 -07:00

scheduler.py

[Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544)

2026-04-20 22:50:40 -07:00

template_manager.py

fix: fix the bug blocking completion template application (#17010)

2026-02-24 18:50:08 +08:00

tokenizer_control_mixin.py

Merge /get_load into /v1/loads (#23010)

2026-04-17 13:36:51 -07:00

tokenizer_manager_score_mixin.py

[Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544)

2026-04-20 22:50:40 -07:00

tokenizer_manager.py

[Score API] Add Multi-Item Scoring with pre-computed delimiter indices (#22544)

2026-04-20 22:50:40 -07:00

tp_worker.py

[Score API] Add return_pooled_hidden_states to Scoring API for SequenceClassification / RewardModel (#22427)

2026-04-15 14:58:56 -07:00

utils.py

Support return_logprob for spec v2 (overlap safe) (#19801)

2026-03-10 15:38:27 -07:00