[PR #1] [CLOSED] WIP: Support multi prefill instances on one node #5

New issue

Closed

opened 2026-10-29 21:06:24 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:06:24 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/sglang/pull/1
Author: @yuan-luo
Created: 4/7/2025
Status: ❌ Closed

Base: mooncake_transfer_engine ← Head: sgl_multi_prefill

📝 Commits (1)

9b4b682 Support multi prefill on one node

📊 Changes

9 files changed (+130 additions, -21 deletions)

View changed files

📝 python/sglang/srt/disaggregation/conn.py (+100 -10)
📝 python/sglang/srt/disaggregation/decode.py (+3 -1)
📝 python/sglang/srt/disaggregation/mini_lb.py (+6 -0)
📝 python/sglang/srt/disaggregation/prefill.py (+3 -3)
📝 python/sglang/srt/managers/io_struct.py (+6 -0)
📝 python/sglang/srt/managers/schedule_batch.py (+6 -0)
📝 python/sglang/srt/managers/scheduler.py (+3 -0)
📝 python/sglang/srt/managers/tokenizer_manager.py (+3 -0)
📝 python/sglang/srt/server_args.py (+0 -7)

📄 Description

Motivation

Modifications

Checklist

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/sglang/pull/1 **Author:** [@yuan-luo](https://github.com/yuan-luo) **Created:** 4/7/2025 **Status:** ❌ Closed **Base:** `mooncake_transfer_engine` ← **Head:** `sgl_multi_prefill` --- ### 📝 Commits (1) - [`9b4b682`](https://github.com/kvcache-ai/sglang/commit/9b4b68268d64a51e7082ae675f9b1a11236d6955) Support multi prefill on one node ### 📊 Changes **9 files changed** (+130 additions, -21 deletions) <details> <summary>View changed files</summary> 📝 `python/sglang/srt/disaggregation/conn.py` (+100 -10) 📝 `python/sglang/srt/disaggregation/decode.py` (+3 -1) 📝 `python/sglang/srt/disaggregation/mini_lb.py` (+6 -0) 📝 `python/sglang/srt/disaggregation/prefill.py` (+3 -3) 📝 `python/sglang/srt/managers/io_struct.py` (+6 -0) 📝 `python/sglang/srt/managers/schedule_batch.py` (+6 -0) 📝 `python/sglang/srt/managers/scheduler.py` (+3 -0) 📝 `python/sglang/srt/managers/tokenizer_manager.py` (+3 -0) 📝 `python/sglang/srt/server_args.py` (+0 -7) </details> ### 📄 Description  ## Motivation  ## Modifications  ## Checklist - [ ] Format your code according to the [Code Formatting with Pre-Commit](https://docs.sglang.ai/references/contribution_guide.html#code-formatting-with-pre-commit). - [ ] Add unit tests as outlined in the [Running Unit Tests](https://docs.sglang.ai/references/contribution_guide.html#running-unit-tests-adding-to-ci). - [ ] Update documentation / docstrings / example tutorials as needed, according to [Writing Documentation](https://docs.sglang.ai/references/contribution_guide.html#writing-documentation-running-docs-ci). - [ ] Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to [Benchmark and Profiling](https://docs.sglang.ai/references/benchmark_and_profiling.html) and [Accuracy Results](https://docs.sglang.ai/references/accuracy_evaluation.html). - [ ] For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR. - [ ] Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:06:24 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/sglang#5

No description provided.

Rows
Columns