[PR #78] [CLOSED] [feat](lora): add KT composite multi-LoRA serving #77

New issue

Closed

opened 2026-10-29 21:07:01 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:07:01 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/sglang/pull/78
Author: @Illumination111
Created: 8/10/2026
Status: ❌ Closed

Base: main ← Head: agent/multi-lora-serving

📝 Commits (1)

0b15f39 feat: add KT composite multi-LoRA serving

📊 Changes

10 files changed (+963 additions, -102 deletions)

View changed files

📝 python/sglang/srt/layers/moe/kt_ep_wrapper.py (+269 -66)
➕ python/sglang/srt/lora/kt_composite_lora_manager.py (+470 -0)
📝 python/sglang/srt/lora/lora_registry.py (+10 -0)
📝 python/sglang/srt/managers/scheduler.py (+7 -0)
📝 python/sglang/srt/model_executor/cuda_graph_runner.py (+3 -0)
📝 python/sglang/srt/model_executor/forward_batch_info.py (+7 -0)
📝 python/sglang/srt/model_executor/model_runner.py (+54 -1)
📝 python/sglang/srt/model_executor/piecewise_cuda_graph_runner.py (+3 -0)
📝 python/sglang/srt/models/qwen3_5.py (+16 -3)
📝 python/sglang/srt/server_args.py (+124 -32)

📄 Description

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review Process

Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
After green CI and required approvals, ask Merge Oncalls to merge.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/sglang/pull/78 **Author:** [@Illumination111](https://github.com/Illumination111) **Created:** 8/10/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `agent/multi-lora-serving` --- ### 📝 Commits (1) - [`0b15f39`](https://github.com/kvcache-ai/sglang/commit/0b15f3966d0b178c23c3e73f74ee437529da69a4) [feat](lora): add KT composite multi-LoRA serving ### 📊 Changes **10 files changed** (+963 additions, -102 deletions) <details> <summary>View changed files</summary> 📝 `python/sglang/srt/layers/moe/kt_ep_wrapper.py` (+269 -66) ➕ `python/sglang/srt/lora/kt_composite_lora_manager.py` (+470 -0) 📝 `python/sglang/srt/lora/lora_registry.py` (+10 -0) 📝 `python/sglang/srt/managers/scheduler.py` (+7 -0) 📝 `python/sglang/srt/model_executor/cuda_graph_runner.py` (+3 -0) 📝 `python/sglang/srt/model_executor/forward_batch_info.py` (+7 -0) 📝 `python/sglang/srt/model_executor/model_runner.py` (+54 -1) 📝 `python/sglang/srt/model_executor/piecewise_cuda_graph_runner.py` (+3 -0) 📝 `python/sglang/srt/models/qwen3_5.py` (+16 -3) 📝 `python/sglang/srt/server_args.py` (+124 -32) </details> ### 📄 Description  ## Motivation  ## Modifications  ## Accuracy Tests  ## Benchmarking and Profiling  ## Checklist - [ ] Format your code according to the [Format code with pre-commit](https://docs.sglang.io/developer_guide/contribution_guide.html#format-code-with-pre-commit). - [ ] Add unit tests according to the [Run and add unit tests](https://docs.sglang.io/developer_guide/contribution_guide.html#run-and-add-unit-tests). - [ ] Update documentation according to [Write documentations](https://docs.sglang.io/developer_guide/contribution_guide.html#write-documentations). - [ ] Provide accuracy and speed benchmark results according to [Test the accuracy](https://docs.sglang.io/developer_guide/contribution_guide.html#test-the-accuracy) and [Benchmark the speed](https://docs.sglang.io/developer_guide/contribution_guide.html#benchmark-the-speed). - [ ] Follow the SGLang code style [guidance](https://docs.sglang.io/developer_guide/contribution_guide.html#code-style-guidance). ## Review Process 1. Ping Merge Oncalls to start the PR flow. See the [PR Merge Process](https://github.com/sgl-project/sglang/blob/main/.github/MAINTAINER.md#pull-request-merge-process). 2. Get approvals from [CODEOWNERS](https://github.com/sgl-project/sglang/blob/main/.github/CODEOWNERS) and other reviewers. 3. Trigger CI tests with [comments](https://docs.sglang.io/developer_guide/contribution_guide.html#how-to-trigger-ci-tests) or contact authorized users to do so. - `/tag-run-ci-label`, `/rerun-failed-ci`, `/tag-and-rerun-ci` 4. After green CI and required approvals, ask Merge Oncalls to merge. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:07:01 +00:00